Monitoring system, electronic equipment and computer readable storage medium

Abstract

The invention provides a monitoring system, electronic equipment and a computer readable storage medium. The system comprises an acquisition and reporting module, a buffer storage module, a data consumption module and a scheduling module, and the monitoring system can obtain coarse-grained monitoring data from an application node side through the acquisition and reporting module, so that excessive network bandwidth and system storage resources are prevented from being occupied; fine-grained data is stored in a local database on an application node side, so that missing of collected data can be avoided, even if data received by a monitoring system is missing due to network fluctuation and the like, the missing part can be re-scheduled and uploaded through a scheduling module, and a data consumption module supports data visual display, so that the data consumption efficiency is improved. Therefore, the integrity and observability of the monitoring data can be improved. Besides, the scheduling module can support a user-defined data scheduling task, so that the monitoring data with different granularities required by the reporting module to the reporting system can be flexibly scheduled and collected, and the availability of the monitoring system is enhanced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a monitoring system, an electronic device, and a computer-readable storage medium. Background Art

[0002] The monitoring system plays a vital role in the entire operation and maintenance process and even the entire life cycle of the software product, providing strong data support for fault warning and problem analysis. As the smallest data unit of the monitoring system, the collection granularity of the monitoring indicator greatly determines the difficulty of abnormal location.

[0003] At present, monitoring systems mainly adopt coarse-grained collection or fine-grained collection solutions. Coarse-grained collection mostly collects application-related monitoring data at intervals of minutes, while fine-grained collection mostly collects application-related monitoring data at intervals of seconds.

[0004] However, as the volume of application services continues to grow, the monitoring system has gradually exposed some defects. For example, the long interval of the coarse-grained collection solution easily leads to the loss of key data, the fine-grained collection solution requires a huge amount of bandwidth and storage resources, and the data integrity and observability of the monitoring system are poor. Summary of the invention

[0005] The embodiments of the present application provide a monitoring system, an electronic device, and a computer-readable storage medium, which are used to improve the data integrity and observability of the monitoring system and reduce resource usage.

[0006] In a first aspect, an embodiment of the present application provides a monitoring system, including: a collection and reporting module, a buffer storage module, a data consumption module and a scheduling module;

[0007] The collection and reporting module is deployed on the application node, and is used to collect the monitoring data of the application node at a first time interval and write the monitoring data into a local database of the application node, and is also used to read part of the monitoring data from the local database at a second time interval and transmit it to the buffer storage module, wherein the first time interval is smaller than the second time interval;

[0008] The buffer storage module is used to write the monitoring data transmitted by the collection and reporting module into the system database in an asynchronous manner;

[0009] The data consumption module is used to display the monitoring data in the system database in a visual manner;

[0010] The scheduling module is used to create a data scheduling task and send a reporting instruction to the collection and reporting module according to the data scheduling task. The data scheduling task is used to indicate the monitoring data that needs to be reported repeatedly, and the reporting instruction is used to enable the collection and reporting module to transmit the monitoring data indicated by the data scheduling task to the buffer storage module.

[0011] In a second aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0012] The memory stores computer-executable instructions;

[0013] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementations of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] Figure 1 A schematic diagram of an application scenario provided for this application;

[0018] Figure 2 A schematic diagram of the structure of a monitoring system provided for this application;

[0019] Figure 3 A schematic diagram of the working principle of a collection and reporting module provided for this application;

[0020] Figure 4 A schematic diagram of the workflow of a data reporting submodule provided for this application;

[0021] Figure 5 A schematic diagram of the working principle of a buffer storage module provided in this application;

[0022] Figure 6 A schematic diagram of the structure of an electronic device provided in this application.

[0023] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0025] At present, the scale of application services continues to expand. Complex architecture design and high management requirements have greatly increased the difficulty of operation and maintenance, and the stability of system operation has been severely challenged. As an important guarantee for the stable operation of services, the monitoring system plays an important role in fault prevention and problem analysis. However, due to the limitations of data center construction and existing monitoring systems, the monitoring system still has defects in key aspects such as observability, which can be manifested in the following aspects:

[0026] First, key monitoring data is missing. In the past, the application system design was relatively simple, with less concurrent pressure, and the use of coarse-grained collection solutions to report minute-level monitoring data was sufficient to meet the observability requirements. However, with the rapid expansion of service scale, the business traffic in one second may exceed dozens of times before, and the system is facing great pressure. At this time, if the monitoring data is still collected at intervals of 60 seconds, it is very likely that key indicators during this period will be missing, affecting the status observation of the system at abnormal times.

[0027] Second, high resource consumption. Based on the need for real-time monitoring indicator observation, some monitoring systems support fine-grained second-level monitoring solutions, collecting and reporting data at intervals of 1 second or 5 seconds. However, compared with the previous coarse-grained collection once a minute, the use of network bandwidth, storage and other resources in the second-level monitoring solution has also increased by dozens of times, greatly increasing the cost of system construction, and may be limited by the deployment and capacity of data center infrastructure, making it impossible to implement this solution.

[0028] Third, the integrity and observability of monitoring data are poor. When the monitoring system is collecting monitoring data of application services, if abnormal network fluctuations occur, the data that should have been reported will be lost, and the integrity of the monitoring data will be poor. In addition, most of the data in the monitoring system is raw data that has not been cleaned, with high dimensions, which is difficult for users to interpret and analyze, and has poor observability.

[0029] Based on this, the present application provides a monitoring system, an electronic device and a computer-readable storage medium to solve the above-mentioned technical problems.

[0030] Figure 1 A schematic diagram of an application scenario provided by this application. Figure 1 As shown, the targets to be monitored involve multiple application services, which can run on one or more application nodes, and the monitoring system needs to collect monitoring data of multiple monitoring indicators related to the application services. The application nodes can be physical servers or virtual machines.

[0031] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0032] The application scenarios mentioned above are only some examples. Those skilled in the art can expand the application to specific business scenarios, and the embodiments of this application do not impose specific limitations on this. Figure 1 The monitoring system provided by the present application is described with reference to other drawings for application scenarios.

[0033] Figure 2 This is a schematic diagram of the structure of a monitoring system provided by this application. Figure 2 As shown, the monitoring system provided by the embodiment of the present application may include Figure 2 The modules in the dotted box may specifically include a buffer storage module, a data consumption module, a scheduling module, and a collection and reporting module deployed on application nodes such as a server or a virtual machine.

[0034] The collection and reporting module is deployed on the application node, and is used to collect the monitoring data of the application node at a first time interval and write the monitoring data into the local database of the application node, and is also used to read part of the monitoring data from the local database at a second time interval and transmit it to the buffer storage module, and the first time interval is smaller than the second time interval.

[0035] The buffer storage module is used to write the monitoring data transmitted by the collection and reporting module into the system database in an asynchronous manner.

[0036] The data consumption module is used to display the monitoring data in the system database in a visual manner.

[0037] The scheduling module is used to create a data scheduling task and send a reporting instruction to the collection and reporting module according to the data scheduling task. The data scheduling task is used to indicate the monitoring data that needs to be reported repeatedly, and the reporting instruction is used to enable the collection and reporting module to transmit the monitoring data indicated by the data scheduling task to the buffer storage module.

[0038] The application node may be a server or a virtual machine, and one or more application services may run on each application node.

[0039] The collection and reporting module can continuously collect monitoring data related to the application service running on the application node (such as CPU usage, memory usage, etc.) at a first time interval based on the pre-configured monitoring items, and the collected data can be directly written into the local database of the application node ( Figure 2 In addition, the collection and reporting module can read the written data from the local database at a second time interval and report it to the buffer storage module, so that the buffer storage module passes the data to other modules for further use. The first time interval and the second time interval can be customized, but the first time interval must be less than the second time interval. For example, the second time interval can be a multiple of the first time interval.

[0040] Exemplarily, the first time interval can be set based on a fine-grained collection scheme, and the second time interval can be set based on a coarse-grained collection scheme. For example, the first time interval can be set to 1 second or 5 seconds, and the second time interval can be set to 1 minute. On the one hand, the collection and reporting module collects monitoring data such as the CPU usage and memory occupancy value of the application service running on the server in units of seconds and writes these data into the local database of the server. On the other hand, the collection and reporting module extracts part of the data written into the database in units of minutes and reports it to the buffer storage module. The local database can store these monitoring data according to the data collection time. For example, the CPU usage collected at 13:00:00 (13:00:00), 13:00:01, ..., 13:01:00, etc. can be recorded in the database as 30%, 31%, ..., 20% respectively. Then, the data read and transmitted to the buffer storage module at the second time interval can be 30% collected at 13:00:00 and 20% collected at 13:01:00.

[0041] The buffer storage module can put the monitoring data transmitted by the collection and reporting module into a queue, read the data in the queue and write it into the system database by enabling asynchronous methods such as asynchronous threads. In some possible implementations, the buffer storage module can also perform missing detection on the data before writing the read data into the system database to determine whether there are missing parts in the monitoring data.

[0042] The data consumption module can provide monitoring data display services. Specifically, it can provide a variety of visual display methods such as charts, reports and dashboards based on the monitoring data stored in the system database, so that users can query and analyze the data collected by the monitoring system.

[0043] The scheduling module can provide the function of re-reporting monitoring data based on user customization or the needs of other modules.

[0044] Exemplarily, if the buffer storage module detects that there are missing parts in the monitoring data, a data scheduling task can be created in the scheduling module, and a reporting instruction generated by the task is sent to the collection and reporting module, so that the collection and reporting module reports the missing monitoring data again.

[0045] For example, if it is found that more fine-grained monitoring data needs to be obtained during the analysis based on the monitoring data in the system database, a corresponding data scheduling task can be created in the scheduling module to enable the collection and reporting module to report the fine-grained monitoring data required for analysis. For example, the collection and reporting module collects monitoring data from the application service in units of 1 second and transmits the monitoring data to the buffer storage module in units of 1 minute. The amount of monitoring data stored in the system database is one-sixtieth of the amount of data collected by the collection and reporting module. If the data consumption module finds that the minute-level monitoring data of a certain period may be abnormal when analyzing the data in the system database, it can be scheduled through the scheduling module to enable the collection and reporting module to report the second-level monitoring data of the period to the buffer storage module and further store it in the system database.

[0046] The monitoring system provided by the embodiment of the present application includes a collection and reporting module, a buffer storage module, a data consumption module and a scheduling module. The collection and reporting module can store the monitoring data collected at a first time interval into a local database and report the monitoring data at a second time interval. By setting different time intervals, fine-grained monitoring data can be stored in the local database on the application node side, and the coarse-grained monitoring data can be reported to the system database through the buffer storage module for analysis and utilization by the data consumption module, and the scheduling module can control the collection and reporting module to repeatedly report monitoring data of different granularities through instructions. In this way, the monitoring system can obtain coarse-grained monitoring data with a small amount of data from the application node side, avoiding occupying too much network bandwidth and system storage resources, and the fine-grained data with a large amount of data is stored in the local database on the application node side, thereby avoiding the loss of collected data. Even if the data received by the monitoring system is missing due to network fluctuations and other reasons, the missing part can be rescheduled and uploaded through the scheduling module, and the data consumption module supports data visualization display, thereby improving the integrity and observability of the monitoring data. In addition, the scheduling module can support custom data scheduling tasks, and then can flexibly schedule the collection and reporting module to report monitoring data of different granularities required by the system, thereby enhancing the availability and scalability of the monitoring system.

[0047] Figure 3 A schematic diagram of the working principle of a collection and reporting module provided in this application. Figure 3 As shown, the collection and reporting module includes an indicator collection submodule, a data scanning submodule, a local database and a data reporting submodule.

[0048] The indicator collection submodule is used to collect monitoring data of the application node under multiple monitoring indicators at a first time interval and store the monitoring data in a local database of the application node.

[0049] The data scanning submodule is used to scan the monitoring data stored in the local database and send a reporting instruction to the data reporting submodule when abnormal data is scanned.

[0050] The data reporting submodule is used to read the corresponding monitoring data from the local database according to the received reporting instruction and transmit the read data to the buffer storage module.

[0051] Among them, the local database can be deployed on various application nodes such as servers. Optionally, the local database can use the SQLite database to store the collected monitoring data, ensure that the fine-grained monitoring data is written to the local disk, and ensure that recent data will not be lost when the server fails, such as power outages or downtime. In addition, users can set the retention time of monitoring data in the SQLite database according to actual conditions (such as local storage space, amount of monitoring data generated per unit time) to save storage resources. SQLite is an embedded relational database that can be used for server-side storage and client-side storage of software.

[0052] The indicator collection submodule can collect the monitoring data related to the application service on the server into the local database according to the pre-set monitoring collection strategy. The monitoring collection strategy can include the granularity of the collected data, such as setting the interval of 1 second to continuously collect monitoring data into the local database to ensure that fine-grained real-time monitoring data is obtained locally on the server. The monitoring collection strategy can also include the definition of the collected monitoring indicators, for example, some commonly used monitoring indicators can be predefined, and monitoring indicators can also be customized according to the needs of different business scenarios. Predefined indicators: commonly used monitoring indicators (such as CPU utilization, memory utilization, disk I / O, network bandwidth, etc.) are used to analyze the operating status of the operating system. Custom indicators: support users to customize monitoring indicators according to actual needs to meet the monitoring needs in different business scenarios. Custom indicators can be based on script extensions, and users can write custom scripts to collect monitoring indicators for specific software or businesses.

[0053] In some possible implementations, the data scanning submodule can be specifically used to: obtain the abnormal conditions of the monitoring indicators; determine whether the monitoring data in the local database meets the corresponding abnormal conditions according to the monitoring indicators, and if so, record the abnormal situation in the local database and send a first reporting instruction to the data reporting submodule.

[0054] The abnormal condition includes that the data falls into a preset abnormal interval or the change amount of the data within a preset period is greater than a preset abnormal threshold. The first reporting instruction is used to instruct the data reporting submodule to transmit the monitoring data that meets the abnormal condition to the buffer storage module.

[0055] The data scanning submodule can integrate the abnormal point scanning function to sequentially scan the fine-grained monitoring data in the local database. If the monitoring indicators in a certain time period are found to be abnormal, a request can be sent to the data reporting submodule to report the fine-grained monitoring data in the abnormal period. The data scanning submodule can predefine the abnormal conditions of the monitoring indicators, and can also support custom abnormal conditions. The following is an explanation of the custom abnormal point judgment conditions and abnormal point processing mechanism:

[0056] (1) Customized abnormal point judgment conditions: Users can define abnormal point judgment conditions based on different monitoring indicators. For example: (a) Threshold over-limit: Set the upper and lower thresholds of the monitoring indicator. When the indicator exceeds the threshold range, it is judged as an abnormal point; Example: CPU usage exceeding 90% or less than 5% can be judged as an abnormal point. (b) Mutation detection: Detect the sharp change of indicators in a short period of time; Example: Disk I / O increases by more than 50% within one minute can be judged as an abnormal point.

[0057] (2) Abnormal point processing mechanism: When the data scanning submodule detects an abnormal point, the following operations can be performed: (a) Record abnormal events: Record abnormal events in the local database for subsequent analysis and tracing. (b) Trigger reporting: Send instructions to the data reporting submodule to report the monitoring data at the abnormal point to the buffer storage module.

[0058] The above scanning function is designed to detect anomalies in monitoring data and report them in a timely manner, thereby improving the system's ability to respond to potential problems.

[0059] The data reporting submodule can be used to report the data in the local database to the buffer storage module, which can include multiple reporting methods such as timed reporting and triggered reporting. The following is an explanation of the timed reporting and triggered reporting mechanisms:

[0060] (1) Regular data reporting function:

[0061] The scheduled data reporting service reports the data in the local database to the buffer storage module according to the pre-set reporting time interval (such as the second time interval). For example, if the user sets the data reporting interval to 30 seconds, the data reporting submodule will only report the monitoring data every 30 seconds (such as the data at 00:00:00, 00:00:30, 00:01:00, etc.).

[0062] (2) Triggered data reporting function:

[0063] The data reporting submodule can report the fine-grained monitoring data of the specified time period stored in the local database according to the instructions of the data scanning submodule or the scheduling module. For example, if the monitoring data at 00:00:30 is abnormal, the data reporting submodule can be instructed to report the monitoring data every second from 00:00:25 to 00:00:35. Triggered reporting can transmit relevant data in a timely manner in specific circumstances, such as when an abnormal point is found, to facilitate rapid response and problem handling.

[0064] In some possible implementations, such as Figure 4 As shown, the data reporting submodule is also used to perform the following steps:

[0065] Step S401, obtaining reporting levels of different monitoring indicators.

[0066] Step S402: determining a second time interval corresponding to the monitoring indicator according to the reporting level of the monitoring indicator.

[0067] Step S403: reading the monitoring data corresponding to the monitoring indicators from the local database according to the second time intervals corresponding to the monitoring indicators, and transmitting the monitoring data to the buffer storage module.

[0068] Specifically, different data reporting time intervals can be customized according to the importance of the monitoring indicators. For the corresponding monitoring indicators, the data reporting submodule only reports the monitoring data at the preset time intervals. For example, the reporting time interval for CPU usage can be set to 30 seconds, and the reporting time interval for memory usage data can be set to 60 seconds.

[0069] The above reporting method can ensure that data at critical moments are reported, and can also reduce the network burden caused by too frequent reporting and save bandwidth. In addition, through customized trigger reporting, you can accurately locate and obtain fine-grained monitoring data, improving the observability of monitoring data.

[0070] Figure 5 A schematic diagram of the working principle of a buffer storage module provided in this application. Figure 5 As shown, the buffer storage module includes an asynchronous buffer submodule, a data storage submodule and a system database.

[0071] The asynchronous buffer submodule is used to receive the monitoring data transmitted by the collection and reporting module, and send the received monitoring data to the message queue.

[0072] The data storage submodule is used to read the monitoring data from the message queue and store the read data into the system database.

[0073] Optionally, the asynchronous buffer submodule can use Kafka to build a message queue. This submodule can aggregate and buffer the monitoring data from the collection and reporting module, put these monitoring data into the message queue, and enable the data storage submodule to asynchronously receive the data in the message queue. If the amount of monitoring data reported is large, the data can be written into different partitions in a multi-partition manner.

[0074] The data storage submodule can read different partitions in the Kafka message queue to achieve asynchronous reception of monitoring data. Optionally, the data storage submodule can clean the received monitoring data, remove invalid and noisy data, and ensure data quality. Optionally, the data storage submodule can also aggregate and perform statistical analysis (such as taking average values, maximum values, etc.) on the monitoring data of each application node according to pre-defined rules to improve the observability of the system. The calculated monitoring data can be stored in the system database to provide functions such as fast retrieval and aggregation calculation for the data consumption module.

[0075] In some possible implementations, the data storage submodule is also used to: detect whether there are missing values ​​in the read monitoring data; if so, send a data scheduling request to the scheduling module based on the missing values, and the data scheduling request is used to instruct the scheduling module to create a corresponding data scheduling task.

[0076] If there are missing values ​​in the data read by the data storage sub-module, it may be that the collection and reporting module failed to successfully report the data in full due to network fluctuations and other reasons. A request can be initiated to the scheduling module based on the data missing situation, so that the scheduling module creates a data scheduling task and further sends a reporting instruction to the collection and reporting module, so that the collection and reporting module can re-report the missing monitoring data.

[0077] Optionally, the system database can use the distributed architecture Druid database. Druid is a high-performance real-time analytical database. The distributed architecture Druid database can realize functions such as efficient storage and fast retrieval of monitoring data, and can also support online analytical processing (OLAP), combined with modules such as data consumption to perform complex query and data analysis operations.

[0078] In the above embodiment, the asynchronous buffer submodule can be used to apply the message queue as an intermediate bridge between data reporting and data storage, provide data buffering, prevent the problem of data loss caused by the mismatch between data writing and consumption speed, and improve the horizontal expansion capability of consumer nodes under high concurrency pressure through multi-partition mode to ensure that data is written to the database in time. The overall solution has high availability and scalability, can ensure the stability and timeliness of data processing under high concurrency pressure, and provide strong technical support for the monitoring system.

[0079] In one embodiment, the data consumption module may include a data analysis submodule, which is used to analyze and process the monitoring data stored in the system database using a built-in data analysis algorithm and to display the analysis and processing results in a visual manner.

[0080] Specifically, the data analysis submodule can have built-in multiple data mining and analysis algorithms to conduct in-depth analysis of historical monitoring data stored in the system database, helping users discover potential problems and anomalies and improve the accuracy of the monitoring system.

[0081] In some possible implementation methods, some machine learning algorithms and statistical analysis methods, such as cluster analysis, regression analysis, time series analysis, etc., can be integrated to conduct in-depth mining of historical monitoring data. Anomaly detection algorithms can also be used to detect abnormal patterns in system operation and provide timely warnings of possible problems.

[0082] In some possible implementations, algorithms can be used to perform trend analysis on monitoring data, predict future system operating conditions, help users perform preventive maintenance and optimize decisions, and analyze the correlation between different monitoring indicators to improve the accuracy of problem location.

[0083] In one embodiment, the data consumption module may further include a custom scheduling submodule, which may be used to: provide a human-computer interaction interface, and obtain target monitoring data selected by a user based on the human-computer interaction interface; and generate a data scheduling request based on the target monitoring data. The data scheduling request is used to instruct the scheduling module to create a corresponding data scheduling task.

[0084] Specifically, the custom scheduling submodule can support the user's active collection of application node data. By submitting a data scheduling request to the scheduling module and reporting the monitoring data of the specified time period, the observability of the system can be improved. First, the custom scheduling submodule can provide a data request interface and a human-computer interaction interface, allowing users to customize data scheduling requests, including specifying application nodes, data types, and time ranges. After the user submits the data scheduling request, the system sends the request to the scheduling module, which can create the corresponding data scheduling task and trigger the corresponding data collection operation. After the data collection is completed, the results can also be fed back to the user to support the user's further analysis and processing of the collection results.

[0085] In some possible implementations, the data consumption module can provide a variety of data visualization methods to enable users to quickly obtain key information. The following is a detailed description of some visualization methods: (1) Chart display: Display monitoring data in various forms such as line charts, bar charts, and pie charts to intuitively present the system operation status. (2) Report generation: Generate reports regularly to provide system operation reports, including key indicators, performance data, and abnormal conditions. (3) Dashboard: Design a customizable dashboard where users can select and arrange different monitoring indicators according to their needs to display the system health status in real time.

[0086] Through the above embodiments, the present application provides a monitoring system solution with high observability, strong data integrity, low bandwidth occupancy, low storage consumption, and high availability and high scalability, which can be applied to large-scale data processing and analysis scenarios. Specifically, the monitoring system of the present application proposes a second-level monitoring data collection solution and monitoring data reporting solutions of different granularities to ensure that anomalies with a short time in the application service can be captured and reported in time, thereby improving the observability of the system. The collection and reporting module stores the monitoring data in the local database on the application node, and reports it through a preset strategy, which can achieve the display effect of second-level monitoring. This mechanism greatly improves the observability of monitoring data and can save network bandwidth and central storage pressure. The monitoring system supports user-defined monitoring indicators and customized monitoring collection script extensions to meet specific monitoring requirements in different business scenarios, and can improve the adaptability and flexibility of the system. In the process of data reporting, real-time data verification is performed, incomplete or erroneous data is marked and repaired, and the final stored data is ensured to be complete and correct. The scheduling module supports a secondary reporting mechanism for data, which can retrieve lost data when the network fluctuates or node fails to ensure the integrity of the monitoring data.

[0087] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0088] In the specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 implements part or all of the functions of the above-mentioned monitoring system.

[0089] The specific implementation process of the processor 601 can be found in the above embodiment. The implementation principle and technical effect are similar and will not be repeated in this embodiment.

[0090] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The functions of the system disclosed in the invention can be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0091] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0092] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0093] The present application also provides a computer program product, including a computer program, which implements the functions of the above-mentioned monitoring system when executed by a processor.

[0094] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the functions of the above-mentioned monitoring system are implemented.

[0095] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0096] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0097] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0100] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0101] Those skilled in the art can understand that all or part of the functions of the above embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the functions of the above embodiments; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0102] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A monitoring system, characterized in that: include: Collection and reporting module, buffer storage module, data consumption module and scheduling module; The collection and reporting module is deployed on the application node, and is used to collect the monitoring data of the application node at a first time interval and write the monitoring data into a local database of the application node, and is also used to read part of the monitoring data from the local database at a second time interval and transmit it to the buffer storage module, wherein the first time interval is smaller than the second time interval; The buffer storage module is used to write the monitoring data transmitted by the collection and reporting module into the system database in an asynchronous manner; The data consumption module is used to display the monitoring data in the system database in a visual manner; The scheduling module is used to create a data scheduling task and send a reporting instruction to the collection and reporting module according to the data scheduling task. The data scheduling task is used to indicate the monitoring data that needs to be reported repeatedly, and the reporting instruction is used to enable the collection and reporting module to transmit the monitoring data indicated by the data scheduling task to the buffer storage module.

2. The monitoring system according to claim 1, characterized in that: The collection and reporting module includes an indicator collection submodule, a data scanning submodule, a local database and a data reporting submodule; The indicator collection submodule is used to collect monitoring data of the application node under multiple monitoring indicators at a first time interval and store it in a local database of the application node; The data scanning submodule is used to scan the monitoring data stored in the local database and send a reporting instruction to the data reporting submodule when abnormal data is scanned; The data reporting submodule is used to read the corresponding monitoring data from the local database according to the received reporting instruction and transmit the read data to the buffer storage module.

3. The monitoring system according to claim 2, characterized in that: The data scanning submodule is specifically used for: Acquire abnormal conditions of the monitoring indicator, wherein the abnormal conditions include that the data falls into a preset abnormal interval or the amount of change of the data within a preset time period is greater than a preset abnormal threshold; Determine whether the monitoring data in the local database meets the corresponding abnormal conditions according to the monitoring indicators. If so, record the abnormal situation in the local database and send a first reporting instruction to the data reporting submodule. The first reporting instruction is used to instruct the data reporting submodule to transmit the monitoring data that meets the abnormal conditions to the buffer storage module.

4. The monitoring system according to claim 2, characterized in that: The data reporting submodule is also used for: Get the reporting level of different monitoring indicators; Determining a second time interval corresponding to the monitoring indicator according to the reporting level of the monitoring indicator; The monitoring data corresponding to the monitoring indicators are read from the local database according to the second time interval corresponding to each monitoring indicator, and transmitted to the buffer storage module.

5. The monitoring system according to any one of claims 1 to 3, characterized in that: The buffer storage module includes an asynchronous buffer submodule, a data storage submodule and a system database; The asynchronous buffer submodule is used to receive the monitoring data transmitted by the collection and reporting module, and send the received monitoring data to the message queue; The data storage submodule is used to read the monitoring data from the message queue and store the read data into the system database.

6. The monitoring system according to claim 5, characterized in that: The data storage submodule is also used for: Detect whether there are missing values ​​in the read monitoring data. If so, send a data scheduling request to the scheduling module according to the missing values, and the data scheduling request is used to instruct the scheduling module to create a corresponding data scheduling task.

7. The monitoring system according to any one of claims 1 to 3, characterized in that: The data consumption module includes a data analysis submodule; The data analysis submodule is used to analyze and process the monitoring data stored in the system database through a built-in data analysis algorithm, and to display the analysis and processing results in a visual manner.

8. The monitoring system according to any one of claims 1 to 3, characterized in that: The data consumption module includes a custom scheduling submodule, and the custom scheduling submodule is used to: Providing a human-computer interaction interface and obtaining target monitoring data selected by a user based on the human-computer interaction interface; A data scheduling request is generated according to the target monitoring data, and the data scheduling request is used to instruct the scheduling module to create a corresponding data scheduling task.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor implements the functions of the monitoring system according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the functions of the monitoring system according to any one of claims 1 to 8.

Images