Monitoring data processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a monitoring data processing method, which can be applied to the field of server management and operation and maintenance, big data analytics and algorithm development. The monitoring data processing method comprises: in response to a monitoring request for a target device, determining a data acquisition rule on the basis of a request processing type contained in the monitoring request; acquiring target monitoring data for the target device according to the data acquisition rule, wherein the target monitoring data comprises at least one type of index data; for each type of index data in the target monitoring data, invoking a target model on the basis of the data type of the index data and the request processing type; and using the target model to process the index data, to obtain a processing result corresponding to the data type of the index data. The present disclosure further provides a monitoring data processing apparatus, a device, and a storage medium.

Description

Monitoring data processing methods, devices, electronic equipment and storage media Technical Field

[0001] This disclosure relates to the fields of server management and maintenance, big data analysis and algorithms, and more specifically, to a monitoring data processing method, apparatus, electronic device, medium and program product. Background Technology

[0002] Current methods for monitoring equipment anomalies typically require manual rule development and data analysis, which is time-consuming and labor-intensive. Furthermore, manual monitoring is prone to omissions, misjudgments, and delays. In addition, these methods often require customized development to meet specific monitoring needs, demanding highly skilled technical personnel and lacking flexibility. Summary of the Invention

[0003] In view of the above problems, this disclosure provides a monitoring data processing method, apparatus, electronic device, medium and program product.

[0004] According to one aspect of this disclosure, a monitoring data processing method is provided, the method comprising:

[0005] In response to monitoring requests for the target device, determine the data acquisition rules based on the request processing type contained in the monitoring request;

[0006] According to the data acquisition rules, target monitoring data for the target device is acquired, wherein the target monitoring data includes at least one type of indicator data;

[0007] For each type of metric data in the target monitoring data, the target model is invoked based on the data type and request processing type of the metric data; and

[0008] The target model is used to process the indicator data to obtain the processing results corresponding to the data type of the indicator data.

[0009] According to embodiments of this disclosure, request processing types include anomaly detection and anomaly prediction.

[0010] According to embodiments of this disclosure, when the request processing type is anomaly detection, the data acquisition rules include acquiring data within a first time period with a first collection cycle as the time interval; the monitoring request includes a data type identifier;

[0011] According to the data acquisition rules, the target monitoring data obtained for the target device includes:

[0012] The target monitoring data is obtained by acquiring the data generated by the target device within the first time period, which corresponds to the data type identifier, with the first acquisition cycle as the time interval.

[0013] According to embodiments of this disclosure, the above method further includes displaying processing results corresponding to the data type of the indicator data, including:

[0014] The target model is used to perform anomaly detection processing on the indicator data and output alarm indicators corresponding to the indicator data.

[0015] Based on the alarm indicator, if it is determined that an alarm is needed, the indicator data and alarm message will be displayed on the display interface.

[0016] Record the alarm status as "on" and the alarm start time.

[0017] According to embodiments of this disclosure, after recording the alarm status as "on" and the alarm start time, the method further includes:

[0018] Anomaly detection processing is performed on the indicator data collected after the alarm start time, and the output alarm flag indicates that no alarm information will be sent if an alarm is required.

[0019] According to embodiments of this disclosure, the above method further includes:

[0020] Anomaly detection processing is performed on the indicator data collected after the alarm start time, and the output alarm flag indicates that the alarm is not needed, and a message to turn off the alarm is sent.

[0021] The alarm status is recorded as off.

[0022] According to embodiments of this disclosure, the indicator data includes real data at m time points, where m ≥ 1;

[0023] The target model is used to perform anomaly detection processing on the indicator data, and the alarm indicators corresponding to the indicator data are output, including:

[0024] Generate predicted data for time m based on real data from m time points;

[0025] Based on the actual data at time m, the predicted data at time m, and the preset rules, an alarm flag is output.

[0026] According to embodiments of this disclosure, the alarm identifier is output based on the actual data at time m, the predicted data at time m, and preset rules, including:

[0027] If the actual data and the predicted data at time m meet the preset rules, output the alarm flag that needs to be alarmed.

[0028] If the actual data at time m and the predicted data at time m do not meet the preset rules, an alarm flag that does not require an alarm will be output.

[0029] According to embodiments of this disclosure, when the request processing type is anomaly prediction, the data acquisition rule includes acquiring data within a second time period with a second collection cycle as the time interval; the monitoring request includes a data type identifier;

[0030] According to the data acquisition rules, the target monitoring data obtained for the target device includes:

[0031] The target monitoring data is obtained by acquiring data generated by the target device within the second time period, with the second acquisition cycle as the time interval;

[0032] The target model is used to process the indicator data, and the processing results corresponding to the data type of the indicator data include:

[0033] The target model is used to perform anomaly prediction processing on the indicator data to obtain the predicted data corresponding to the indicator data.

[0034] According to embodiments of this disclosure, the above method further includes:

[0035] If the target monitoring data does not meet the model input format, the target monitoring data is processed according to the preset format to obtain the model input data.

[0036] According to embodiments of this disclosure, the above method further includes:

[0037] In response to a query request for historical alarm data, the target time period is determined based on the query start time and query end time contained in the query request;

[0038] The display interface shows the alarm data generated during the target time period.

[0039] According to another aspect of this disclosure, a monitoring data processing apparatus is provided, the apparatus comprising:

[0040] The determination module is used to respond to monitoring requests for target devices and determine data acquisition rules based on the request processing type contained in the monitoring request.

[0041] The acquisition module is used to acquire target monitoring data for the target device according to data acquisition rules, wherein the target monitoring data includes at least one type of indicator data;

[0042] The calling module is used to call the target model for each type of metric data in the target monitoring data, based on the data type and request processing type of the metric data; and

[0043] The module is used to process the indicator data using the target model and obtain the processing results corresponding to the data type of the indicator data.

[0044] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0045] One or more processors;

[0046] Memory, used to store one or more computer programs.

[0047] In this process, one or more processors execute one or more computer programs to implement the steps of the above method.

[0048] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program or instructions thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0049] According to another aspect of this disclosure, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0050] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0051] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0052] Figure 1 is a flowchart of a monitoring data processing method according to an embodiment of the present disclosure;

[0053] Figure 2 is a schematic diagram of Zabbix's data collection and monitoring process.

[0054] Figure 3A is a flowchart of an anomaly detection method according to an embodiment of the present disclosure;

[0055] Figure 3B is a flowchart of an anomaly detection method according to another embodiment of the present disclosure;

[0056] Figure 4 is a schematic diagram of anomaly detection alarm logic according to an embodiment of the present disclosure;

[0057] Figure 5 is a schematic diagram of the prediction results according to an embodiment of the present disclosure;

[0058] Figure 6 is a schematic diagram of the architecture of a data monitoring platform according to an embodiment of the present disclosure;

[0059] Figure 7 is a flowchart of an anomaly detection method according to another embodiment of the present disclosure;

[0060] Figure 8 is a flowchart of an anomaly prediction method according to an embodiment of the present disclosure;

[0061] Figure 9 is a structural block diagram of a monitoring data processing apparatus according to an embodiment of the present disclosure; and

[0062] Figure 10 is a block diagram of an electronic device suitable for implementing a monitoring data processing method according to an embodiment of the present disclosure. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the described embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. It should be noted that throughout the accompanying drawings, the same elements are represented by the same or similar reference numerals. In the following description, some specific embodiments are used for descriptive purposes only and should not be construed as limiting this disclosure in any way, but are merely examples of embodiments of this disclosure. Conventional structures or configurations will be omitted where they may cause confusion in understanding this disclosure. It should be noted that the shapes and dimensions of the components in the figures do not reflect actual size and proportion, but are only schematic representations of the embodiments of this disclosure.

[0064] Unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure shall have the ordinary meaning as understood by those skilled in the art. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components.

[0065] Terminology Explanation:

[0066] Server management and maintenance includes server monitoring, configuration management, performance optimization, fault diagnosis and repair, etc. By monitoring and managing server resources in real time, the availability, stability and performance of the server can be improved.

[0067] Big data analytics and algorithms: used to process and analyze collected server resource data. This technology identifies abnormal patterns, trends, and predicts future server resource needs, thereby supporting decision-making and resource planning.

[0068] IoT device management: Used to monitor and manage the status and performance of IoT devices. It includes sensor technology, remote monitoring and control, device connectivity and communication, etc., to achieve effective management and optimization of IoT devices.

[0069] Log management and alarm system: Used to record and analyze server logs to quickly detect and diagnose problems. The alarm system technology can automatically trigger alarms according to set rules and thresholds, and promptly notify maintenance personnel to handle the issues.

[0070] Data visualization and presentation: This feature displays server resource data visually in the form of charts, dashboards, etc., helping operations and maintenance personnel to better understand and analyze the data and discover potential problems and trends.

[0071] Machine intelligence and decision support technologies: Machine intelligence technologies include methods such as machine learning, deep learning, and artificial intelligence, which are used for automated analysis and decision support. By utilizing these technologies, the system can automatically identify abnormal states, predict future trends, and provide accurate decision recommendations for operations and maintenance personnel.

[0072] The relevant technologies have some limitations in anomaly detection. For example, they require manual data analysis and rule formulation, which is time-consuming and labor-intensive, and prone to omissions and false positives. Furthermore, these technologies often require customized development, demanding highly skilled technical personnel and lacking flexibility.

[0073] To address the above-mentioned technical problems, this disclosure provides a monitoring data processing method. The method includes: responding to a monitoring request for a target device, determining data acquisition rules based on a request processing type included in the monitoring request; acquiring target monitoring data for the target device according to the data acquisition rules, wherein the target monitoring data includes at least one type of indicator data; for each type of indicator data in the target monitoring data, invoking a target model according to the data type and request processing type of the indicator data; and processing the indicator data using the target model to obtain a processing result corresponding to the data type of the indicator data. The method provided by this disclosure, by processing the target monitoring data using a target model, can automatically identify normal and abnormal patterns, thereby improving the accuracy and efficiency of anomaly detection. Furthermore, the technical solution of this disclosure does not require customized development, reducing the burden on technical personnel, providing a more flexible and easily deployable solution, and accelerating the implementation and application of anomaly detection.

[0074] It should be noted that the monitoring data processing method provided in this disclosure can be applied to server monitoring, production equipment monitoring, sensor monitoring, smart home device monitoring, intelligent transportation equipment monitoring, logistics equipment monitoring, etc. The embodiments of this disclosure take server monitoring as an example for detailed description.

[0075] Figure 1 is a flowchart of a monitoring data processing method according to an embodiment of the present disclosure.

[0076] According to embodiments of this disclosure, the data monitoring and processing method can be applied to a data monitoring platform.

[0077] As shown in Figure 1, the monitoring data processing method according to an embodiment of the present disclosure includes operations S110 to S140.

[0078] In operation S110, in response to a monitoring request for the target device, data acquisition rules are determined based on the request processing type contained in the monitoring request.

[0079] According to embodiments of this disclosure, the target device can be any device that needs to be monitored. For example, the target device can be a server, production equipment, sensor, smart home device, smart transportation equipment, logistics equipment, etc. Request processing types can include, for example, anomaly detection and anomaly prediction.

[0080] According to embodiments of this disclosure, data acquisition rules may include specific requirements for acquiring data. For example, data acquisition rules may include data collection cycle, data collection volume, data collection time, etc.

[0081] It's important to note that different request processing types require different monitoring data; therefore, each type has its own data acquisition rules. For example, for anomaly detection requests, historical data generated within the last 10 minutes prior to the current time is needed; while for anomaly prediction requests, historical data generated within the last 3 months prior to the current time is required. Furthermore, for anomaly detection requests, data needs to be acquired every 5 minutes for anomaly detection; while for anomaly prediction requests, data needs to be acquired every day for anomaly prediction.

[0082] In operation S120, target monitoring data for the target device is acquired according to the data acquisition rules, wherein the target monitoring data includes at least one type of indicator data.

[0083] According to embodiments of this disclosure, target monitoring data may include data related to the operating status of the target device. For example, target monitoring data may include resource utilization data, operating parameter data, response time data, etc., of the target device. For example, target monitoring data may include CPU data, GPU data, memory data, disk data, response time data, etc., of a server. Furthermore, target monitoring data may also include production parameter data, process parameter data, etc., of production equipment. Furthermore, target monitoring data may also include humidity data, temperature data, etc., from sensors.

[0084] According to embodiments of this disclosure, target monitoring data may include various types of metric data. For example, if target monitoring data includes server CPU data and GPU data, then CPU data is one type of metric data, and GPU data is another type of metric data. It should be noted that different data types correspond to different processing models. For example, metric data of type CPU data corresponds to a model that processes CPU data, and metric data of type GPU data corresponds to a model that processes GPU data.

[0085] In operation S130, for each type of indicator data in the target monitoring data, the target model is invoked according to the data type and request processing type of the indicator data.

[0086] According to embodiments of this disclosure, different types of request processing types correspond to different processing models. For example, the request processing type for anomaly detection corresponds to the processing model applied to anomaly detection, and the request processing type for anomaly prediction corresponds to the processing model applied to anomaly prediction.

[0087] According to embodiments of this disclosure, calling the target model for each type of indicator data in the target monitoring data, based on the data type and request processing type of the indicator data, may include: preliminarily determining the processing model based on the request processing type to obtain a first candidate model set; for example, if the request processing type is anomaly detection, then the first candidate model set includes all models in the data monitoring platform used for anomaly detection; or if the request processing model is anomaly prediction, then the first candidate model set includes all models in the data monitoring platform used for anomaly prediction; and then, based on the data type, determining the processing model corresponding to that data type from the first candidate model set to obtain the target model.

[0088] In one embodiment, if the request processing type is anomaly detection and the data type is CPU data, then the target processing model can be a model for anomaly detection on CPU data. In another embodiment, if the request processing type is anomaly prediction and the data type is GPU data, then the target processing model can be a model for anomaly prediction on GPU data.

[0089] It should be noted that when the target monitoring data includes indicator data of multiple data types, each indicator data has its own corresponding target model. For example, if the target monitoring data includes indicator data A of data type A and indicator data B of data type B, then when determining the target model, it is necessary to determine the target model corresponding to indicator data A, such as model A, and also to determine the target model corresponding to indicator data B, such as model B.

[0090] In operation S140, the target model is used to process the indicator data to obtain the processing result corresponding to the data type of the indicator data.

[0091] According to embodiments of this disclosure, different data types of indicator data are processed using different models to obtain processing results corresponding to that data type. For example, when the indicator data is CPU data, the processing result for CPU data is obtained; when the indicator data is GPU data, the processing result for GPU data is obtained.

[0092] According to embodiments of this disclosure, in response to a monitoring request for a target device, data acquisition rules are determined based on the request processing type contained in the monitoring request, and target monitoring data for the target device is acquired. Then, for each type of indicator data in the target monitoring data, a target model is invoked according to the data type and request processing type of the indicator data; and the target model is used to process the indicator data to obtain a processing result corresponding to the data type of the indicator data. This method acquires monitoring data for the target device and invokes advanced big data algorithms for anomaly detection or anomaly prediction, which can automatically identify normal and abnormal patterns, thereby improving the accuracy and efficiency of anomaly detection. This enables maintenance personnel to promptly discover and resolve abnormal issues, improving the performance and operating efficiency of the target device. Simultaneously, this solution does not require customized development, reducing the burden on technical personnel, providing a more flexible and easily deployable solution, and accelerating the implementation and application of anomaly detection.

[0093] According to embodiments of this disclosure, target monitoring data for a target device is obtained by acquiring data from the target device in real time using a monitoring device.

[0094] According to embodiments of this disclosure, monitoring devices may include, for example, Zabbix, Prometheus, Nagios, Grafana, and Graylog. Zabbix is ​​a monitoring software platform used to monitor IT infrastructure, including servers, networks, and applications. It provides real-time monitoring, alerting, and visualization capabilities to help administrators maintain system performance and availability. Prometheus is an open-source monitoring and alerting system. Nagios is an open-source network monitoring tool that effectively monitors host status, network devices such as switches and routers, and printers. Grafana is a cross-platform, open-source visualization and analysis tool. Graylog is an open-source log aggregation, analysis, auditing, display, and alerting tool. Embodiments of this disclosure use Zabbix as an example for data collection.

[0095] Figure 2 is a schematic diagram of the Zabbix data collection and monitoring process.

[0096] According to embodiments of this disclosure, before data collection using Zabbix, a Zabbix client is deployed on each target device (server), and relevant data collection parameters are configured on the Zabbix configuration page. For example, the Zabbix client is configured to report information such as the server's CPU, memory, disk, and GPU every second to the Zabbix server. The Zabbix server is responsible for storing the reported metric name, server IP address, reporting time, etc., and storing the collected data in a database. It also provides a callable HTTP RESTful interface for various scenarios of the data monitoring platform. An HTTP RESTful interface is an application programming interface (API) design style based on the HTTP protocol. It follows the REST (Representational State Transfer) principle, using standard HTTP methods (such as GET, POST, PUT, DELETE) to operate on resources. RESTful interfaces locate resources via URLs, perform operations using HTTP verbs, and use HTTP status codes to indicate the operation results.

[0097] According to embodiments of this disclosure, obtaining target monitoring data for a target device may include calling a Zabbix service via an HTTP RESTful interface to obtain target monitoring data for the target device, such as CPU, memory, GPU, network, and other data.

[0098] According to an embodiment of this disclosure, when the request processing type is anomaly detection, the data acquisition rule includes acquiring data within a first time period with a first acquisition cycle as the time interval; the monitoring request includes a data type identifier; according to the data acquisition rule, acquiring target monitoring data for the target device includes: acquiring data generated by the target device within the first time period corresponding to the data type identifier with a first acquisition cycle as the time interval, thereby obtaining target monitoring data.

[0099] According to embodiments of this disclosure, taking a first acquisition period of n minutes, data type A, and a first time period of m minutes as an example, acquiring data corresponding to the data type identifier generated by the target device within the first time period, with the first acquisition period as the time interval, to obtain target monitoring data may include: every n minutes, acquiring data of data type A generated by the target device within the previous m minutes from the Zabbix service. Specifically, for example, if n is 5, m is 10, and the data type is CPU data, then every 5 minutes, acquiring CPU data generated by the target device within the previous 10 minutes from the Zabbix service; this CPU data is the target monitoring data.

[0100] In another embodiment, taking a first collection period of n minutes, data types of type A and type B, and a first time period of m minutes as an example, the target monitoring data is obtained by acquiring data corresponding to the data type identifier generated by the target device within the first time period, with the first collection period as the time interval. This can include: every n minutes, acquiring data of type A and type B generated by the target device within the previous m minutes from the Zabbix service. Specifically, for example, if n is 5, m is 10, and the data types are CPU data and GPU data, then every 5 minutes, acquiring the CPU data and GPU data generated by the target device within the previous 10 minutes from the Zabbix service. This CPU data and GPU data constitute the target monitoring data, which includes two metrics: CPU data and GPU data.

[0101] According to embodiments of this disclosure, the method further includes: when it is determined that the target monitoring data does not meet the model input format, processing the target monitoring data according to a preset format to obtain model input data.

[0102] According to embodiments of this disclosure, the preset format may include a preset data structure. For example, the preset data structure may sequentially include a data ID, a data acquisition time, and a CPU usage percentage. Specifically, when the target monitoring data is CPU data, the data obtained using the preset format may include a CPU data ID, a CPU data acquisition time, and a CPU usage percentage.

[0103] According to embodiments of this disclosure, when the collected target monitoring data is not the aforementioned preset data structure, data processing of the target monitoring data according to a preset format to obtain model input data may include: extracting attribute information from the preset format, namely data ID, data collection time, and occupancy percentage; extracting corresponding data from the target monitoring data based on the attribute information; and generating model input data according to the preset format using the extracted attribute information and data.

[0104] It should be noted that, in some embodiments, the preset format can be pre-configured during the data acquisition phase, ensuring that the acquired data conforms to the preset format. Data conforming to the preset format does not require format conversion after being acquired by the data monitoring platform. In other embodiments, the preset format may not be pre-configured during the data acquisition phase; instead, data processing is performed after the target monitoring data is acquired by the data monitoring platform to conform to the preset format.

[0105] According to embodiments of this disclosure, the target model for anomaly detection can use an autoencoder. The autoencoder is trained using historical indicator data within a preset time period to obtain an anomaly detection model corresponding to the indicator data. For example, if the indicator data is CPU data, historical CPU data within the past three months can be collected, and the autoencoder can be trained using this historical CPU data to obtain an anomaly detection model corresponding to the CPU data.

[0106] An autoencoder is an unsupervised learning algorithm used to learn effective data representations or feature extraction. It can be applied to multiple fields, including but not limited to dimensionality reduction, data repair, anomaly detection, and generative models. Its function is to learn useful features from input data and map them to a low-dimensional encoding space for analysis, processing, or generation of new data in subsequent tasks. In this embodiment, the autoencoder identifies outliers by learning a high-level representation of the data and detecting inconsistencies with the training data through a reconstruction phase.

[0107] According to embodiments of this disclosure, the above method further includes displaying the processing results corresponding to the data type of the indicator data, including: performing anomaly detection processing on the indicator data using a target model and outputting an alarm identifier corresponding to the indicator data; displaying the indicator data and alarm message on the display interface when it is determined that an alarm is needed based on the alarm identifier; recording the alarm status as "on" and recording the alarm start time.

[0108] According to embodiments of this disclosure, alarm identifiers include alarm identifiers that require an alarm and alarm identifiers that do not require an alarm. Alarm identifiers can be any identifier capable of distinguishing whether an alarm is required. Alarm identifiers can include numbers, letters, etc. Specifically, for example, the number 1 can be used to represent an alarm identifier that requires an alarm, and the number 2 can be used to represent an alarm identifier that does not require an alarm. Based on the alarm identifier, when it is determined that an alarm is required, displaying indicator data and an alarm message on the display interface can include: if the alarm identifier is 1, indicating that an alarm is required, then the indicator data and an alarm message are displayed on the display interface.

[0109] Figure 3A is a flowchart of an anomaly detection method according to an embodiment of the present disclosure.

[0110] As shown in Figure 3A, the anomaly detection method of this embodiment includes operations S301 to S307.

[0111] In operation S301, in response to a monitoring request for the target device, a data acquisition rule is determined based on the request processing type contained in the monitoring request, wherein the request processing type is anomaly detection; the data acquisition rule includes acquiring data within a first time period with a first collection cycle as the time interval.

[0112] In operation S302, data corresponding to the data type identifier generated by the target device within a first time period is acquired at intervals of the first acquisition cycle to obtain target monitoring data. The target monitoring data includes at least one type of indicator data.

[0113] In operation S303, for each type of indicator data in the target monitoring data, determine the anomaly detection model corresponding to the indicator data type to obtain the target model.

[0114] When operating S304, the target model is used to perform anomaly detection processing on the indicator data, and alarm indicators corresponding to the indicator data are output.

[0115] In operation S305, determine whether an alarm is needed based on the alarm indicator. If an alarm is needed, proceed to operation S306; if an alarm is not needed, proceed to operation S307.

[0116] When operating S306, an alarm message is sent, and the indicator data and alarm message are displayed on the display interface.

[0117] When operating S307, record monitoring requests and response logs.

[0118] According to embodiments of this disclosure, after recording the alarm status as "on" and the alarm start time, the method further includes: performing anomaly detection processing on the indicator data collected after the alarm start time, and outputting an alarm identifier indicating that no alarm information is sent when an alarm is required.

[0119] According to embodiments of this disclosure, the method further includes: performing anomaly detection processing on the indicator data collected after the alarm start time, outputting an alarm identifier indicating that the alarm is not required, and sending information to disable the alarm; recording the alarm status as disabled.

[0120] According to an embodiment of this disclosure, when the alarm status is on, if an alarm is needed again within multiple detection cycles after the alarm starts, no alarm information will be sent until the alarm is no longer needed, at which point a message to disable the alarm will be sent and the alarm will be disabled.

[0121] According to embodiments of this disclosure, based on the alarm identifier returned by the target model, a decision is made as to whether to send the information to the unified log management platform. If the alarm identifier indicates that an alarm is needed, the current alarm status is recorded as "on," and the original value of the data and the alarm message content are sent to the unified management platform for alarm reporting. If an alarm is still needed within the next n minutes, it is not necessary to report to the unified management platform again until the a-th (a>=2) n-th consecutive minute ends, at which point the intelligent algorithm layer returns that no alarm is needed. At this time, the data forwarding layer sends an alarm-off status to the unified management platform to indicate the end of the alarm process.

[0122] If the alarm flag indicates that no alarm is needed, only request and response logs are recorded for problem tracking and troubleshooting, and no logical processing is performed.

[0123] Figure 3B is a flowchart of an anomaly detection method according to another embodiment of the present disclosure.

[0124] As shown in Figure 3B, in addition to operations S301 to S307, the anomaly detection method of this embodiment may also include operations S308 to S312.

[0125] When operating S308, record the alarm status as "on" and the alarm start time.

[0126] When operating S309, perform anomaly detection processing on the indicator data collected after the alarm start time and output the alarm flag.

[0127] In operation S310, determine whether an alarm is needed based on the alarm indicator. If an alarm is needed, execute operation S312; if an alarm is not needed, execute operation S311.

[0128] When operating S311, send a message to turn off the alarm and record the alarm status as off.

[0129] When operating S312, no alarm information is sent.

[0130] Figure 4 is a schematic diagram of the anomaly detection alarm logic according to an embodiment of the present disclosure.

[0131] According to an embodiment of this disclosure, a CPU anomaly detection is performed every 10 minutes from 8:50 to 10:00, and the specific detection alarm logic is shown in Figure 3.

[0132] If the CPU is detected as normal at 8:50, no alarm is needed. If the CPU is detected as abnormal at 9:00 in the next detection cycle, an alarm is needed, and the alarm status is enabled. If the CPU is detected as abnormal for the next four consecutive detection cycles (9:10, 9:20, 9:30, and 9:40), no alarm is needed. If the CPU is detected as normal at 9:50, an alarm is needed, and the alarm status is disabled.

[0133] According to an embodiment of this disclosure, the indicator data includes real data at m time points, where m ≥ 1; the indicator data is processed by an anomaly detection method using a target model, and an alarm identifier corresponding to the indicator data is output, including: generating predicted data at the m-th time point based on the real data at the m-th time points; and outputting an alarm identifier based on the real data at the m-th time point, the predicted data at the m-th time point, and a preset rule.

[0134] According to embodiments of this disclosure, the output of an alarm identifier based on the actual data at time m, the predicted data at time m, and a preset rule includes: outputting an alarm identifier that requires an alarm when it is determined that the actual data at time m and the predicted data at time m meet the preset rule; and outputting an alarm identifier that does not require an alarm when it is determined that the actual data at time m and the predicted data at time m do not meet the preset rule.

[0135] According to embodiments of this disclosure, the preset rules can be rules pre-configured based on the type of indicator data. Preset rules can include whether the percentage fluctuation of predicted and actual data is less than a preset value; preset rules can also be the relationship between actual data and thresholds and mean values. Taking CPU as an example, each server pool can contain a CPU model (Model_cpu), a threshold (Threshold_cpu), and a mean value (Mean_cpu). Based on the actual and predicted CPU values, as well as the threshold and mean values, it can be determined whether CPU resource usage is abnormal. Specifically, the relationship between the actual value of server resources and the threshold and mean values ​​can be analyzed using the loaded model and threshold data. When the actual value exceeds the threshold, it can be determined as an abnormal situation; when the actual value is within the mean range, it can be determined as a normal situation.

[0136] In one embodiment, the preset rule may include whether the fluctuation percentage of the predicted data and the actual data is less than a first value. Specifically, the predicted data at time m is output based on the actual data at m times. Then, the alarm flag is output based on the actual data at time m, the predicted data at time m, and the preset rule. This may include: determining the fluctuation percentage based on the actual data at time m and the predicted data at time m; if the fluctuation percentage is less than the first value, it is judged as normal, and an alarm flag is output; if the fluctuation percentage is greater than the first value, it is judged as abnormal, and an alarm flag is output.

[0137] According to embodiments of this disclosure, when the request processing type is anomaly prediction, the data acquisition rules include acquiring data within a second time period with a second acquisition cycle as the time interval; the monitoring request includes a data type identifier; according to the data acquisition rules, acquiring target monitoring data for the target device includes: acquiring data generated by the target device within the second time period corresponding to the data type identifier with a second acquisition cycle as the time interval, obtaining target monitoring data, and processing the indicator data using a target model to obtain a processing result corresponding to the data type of the indicator data includes: performing anomaly prediction processing on the indicator data using the target model to obtain predicted data corresponding to the indicator data.

[0138] According to embodiments of this disclosure, the anomaly prediction process is similar to the anomaly detection described above, except that anomaly prediction uses a second collection cycle as the time interval to acquire data within a second time period. Specifically, when performing anomaly prediction, indicator data for the seven days prior to the current time can be acquired hourly, and the prediction for the next day can be made based on the indicator data from those seven days.

[0139] According to embodiments of this disclosure, the data structure for anomaly prediction is the same as that for anomaly detection, the difference being that the amount of target data acquired by anomaly prediction is greater than the amount of data acquired by anomaly detection. For example, anomaly detection acquires data from the past 10 minutes, while anomaly prediction can acquire data from the past 10 days.

[0140] According to embodiments of this disclosure, the output of anomaly prediction is the input data and the predicted data for the next day.

[0141] Figure 5 is a schematic diagram of the prediction results according to an embodiment of the present disclosure.

[0142] As shown in Figure 5, the actual and predicted CPU usage percentages of the monitoring server are displayed. The X-axis represents time, and the Y-axis represents the percentage of CPU usage. Peak values ​​indicate that the CPU usage percentage is relatively high at that time, meaning that CPU-intensive computing tasks are running during that time. Figure 5 shows that the results exhibit periodic peaks, which may occur at specific times each day when CPU-intensive scheduled tasks are executing. This period can be configured by operations or development personnel and can be different, such as daily, weekly, hourly, or minute-based periods.

[0143] According to embodiments of this disclosure, the predictive model used for anomaly prediction can employ ARIMA (Autoregressive Moving Average) and the pmdarima library. An ARIMA model is a time series analysis method that predicts future data changes by modeling patterns and trends in historical data. pmdarima is a library for automatically selecting ARIMA model parameters, providing a more convenient model selection and fitting process. Specifically, anomaly prediction may include: acquiring Zabbix metric data from the past three months, such as CPU utilization and memory usage, and using a predictive algorithm to train and fit the model to obtain the data trend for the next day.

[0144] According to embodiments of this disclosure, the server anomaly detection and prediction method provided by this disclosure enables operations and maintenance personnel to promptly identify and resolve server anomalies, improve server stability and performance, reduce system failure risks, and provide more reliable services to enterprises. Furthermore, this method has a wide range of applications, applicable to anomaly detection and prediction needs in various IoT devices and other fields, providing intelligent solutions for various industries.

[0145] According to embodiments of this disclosure, the method further includes: responding to a query request for historical alarm data, determining a target time period based on the query start time and query end time included in the query request; and displaying the alarm data generated within the target time period on a display interface.

[0146] According to embodiments of this disclosure, the unified log management platform module provides operation and maintenance personnel with a dashboard to view historical alarm data, helping them to handle system and server problems accurately and in real time, and improve problem-solving efficiency.

[0147] According to embodiments of this disclosure, the data monitoring and processing method can be executed by a data monitoring platform.

[0148] Figure 6 is a schematic diagram of the architecture of a data monitoring platform according to an embodiment of the present disclosure.

[0149] As shown in Figure 6, the data monitoring platform of this embodiment may include a chart display layer, a basic data layer, a data processing and forwarding layer, an intelligent algorithm layer, a persistence layer, and a unified log management platform.

[0150] According to the embodiments of this disclosure, the above-mentioned layers are defined at the technical level. Different "layers" perform different technical functions. Each layer can be a separate module. They work together through different data protocols and have calling relationships with each other.

[0151] The icon display layer can use different types of icons to display the processing results. For example, line charts, bar charts, pie charts, etc.

[0152] Basic Data Layer: This layer collects data from the server using monitoring devices. It communicates only with the data processing and forwarding layer using the HTTP protocol. The data processing and forwarding layer provides API interfaces for it to call. Monitoring devices can include ZABBIX, Prometheus, Nagios, Grafana, Graylog, etc.

[0153] Data Processing and Forwarding Layer: This layer receives indicator data collected from the server by the basic data layer and communicates with the persistence layer to complete data storage. Data transmission includes, but is not limited to, HTTP, RPC, message queues, WebSocket, and local call protocols. Simultaneously, the data processing and forwarding layer effectively extracts and transforms the indicator data, then passes the transformed data to the intelligent algorithm layer for processing. HTTP (Hypertext Transfer Protocol) is an application layer protocol used in distributed, collaborative, and hypermedia information systems. RPC (Remote Procedure Call Protocol) is a protocol that allows a program on a remote computer to request services over a network without needing to understand the underlying network technology. WebSocket is a network protocol based on TCP.

[0154] Intelligent Algorithm Layer: The intelligent algorithm layer includes anomaly detection models and anomaly prediction models. The anomaly detection models include detection models for processing different types of data, and the anomaly prediction models also include prediction models for processing different types of data. In this embodiment, this layer is limited to the data forwarding layer calling the corresponding model and responding with the result to the data processing forwarding layer. Data transmission includes, but is not limited to, protocols such as HTTP, RPC, message queues, and WebSocket. It should be noted that the data processing flow in the intelligent algorithm layer of Figure 6 is only one embodiment provided in this disclosure. In other embodiments, the data processing flow may include only some steps in this flow, or it may include other steps outside of this flow.

[0155] Persistence layer: Stores the algorithm data processed by the data forwarding layer.

[0156] Unified log management platform: This can be a third-party alarm system used to receive and display anomaly detection and prediction results, and provide them to operations and maintenance personnel for handling anomaly information. Data that needs to be alarmed can be transferred to this layer using the HTTP protocol, or it can use protocols such as RPC, message queues, and WebSocket.

[0157] It should be noted that the architecture of the data monitoring platform in this disclosure is only one embodiment. In some embodiments, the architecture of the data monitoring platform may not include any one of the icon display layer, persistence layer, and unified log management platform. In other embodiments, the architecture of the data monitoring platform may also include other layers.

[0158] By leveraging the collaborative efforts of the data processing and forwarding layer and intelligent algorithms, this method enables intelligent detection and prediction of server anomalies. Operations and maintenance personnel can intuitively understand the server's operational status and trends through the server graphical display layer, promptly identify anomalies, and take appropriate measures. This improves the efficiency and reliability of server management while reducing the workload of operations and maintenance personnel.

[0159] The intelligent algorithm layer, through the application of advanced machine learning and data mining algorithms, can automatically learn normal patterns in server data and perform anomaly detection and prediction based on historical and real-time data. Once an anomaly is detected, the system will promptly issue an alert, allowing operations and maintenance personnel to handle the situation accordingly. Simultaneously, this intelligent algorithm layer can also predict future server data, providing reference and decision support for operations and maintenance personnel.

[0160] According to embodiments of this disclosure, the data monitoring platform of this embodiment has the following advantages:

[0161] The adaptive anomaly detection algorithm uses an autoencoder to accurately detect anomalies in the data processing forwarding layer, improving accuracy and robustness through adaptability.

[0162] Real-time anomaly prediction algorithms use methods such as ARIMA to analyze historical data in real time and predict future trends, thereby identifying problems in advance and enhancing system stability.

[0163] Based on RESTful interfaces, it supports scenarios such as abnormal detection of process parameters and abnormal detection of IoT devices, flexibly adapting to different needs.

[0164] The data processing workflow covers the complete process of data collection, forwarding, detection, prediction, reporting, and display.

[0165] Figure 7 is a flowchart of an anomaly detection method according to another embodiment of the present disclosure.

[0166] As shown in Figure 7, the basic data layer uses monitoring devices to obtain indicator data 710 from the server. This indicator data may include CPU data, memory data, GPU data, network data, etc. The data processing and forwarding application of the data processing and forwarding layer receives the indicator data collected by the basic data layer and performs data processing 720, which may include parsing and transforming the indicator data. Then, the data processing and forwarding application sends the processed indicator data to the persistence layer for data storage 730. After that, the data processing and forwarding application calls the anomaly detection model of the intelligent algorithm layer to perform anomaly detection on the indicator data 740 and outputs an alarm identifier. Then, based on the alarm identifier, alarm detection is performed 750 to determine whether an alarm is needed. If an alarm is needed, an anomaly alarm is triggered 760, and an alarm message and indicator data are sent to the unified log management platform for display on the alarm dashboard, i.e., the display interface of the unified log management platform.

[0167] Figure 8 is a flowchart of an anomaly prediction method according to an embodiment of the present invention.

[0168] As shown in Figure 8, the basic data layer uses monitoring devices to obtain indicator data 810 from the server. This indicator data may include CPU data, memory data, GPU data, network data, etc. The data processing and forwarding application of the data processing and forwarding layer receives the indicator data obtained by the basic data layer and performs data processing 820, which may include parsing and transforming the indicator data. Then, the data processing and forwarding application sends the processed indicator data to the persistence layer for data storage 830. After that, the data processing and forwarding application calls the anomaly prediction model of the intelligent algorithm layer to perform anomaly prediction on the indicator data 840 and outputs the prediction result. Then, the data processing and forwarding application performs data processing on the prediction result 850, which may include parsing and transforming the prediction result. Finally, the data processing and forwarding application provides a data service interface 860 for the unified log management platform to call, and displays the prediction result on the prediction dashboard, i.e., the display interface of the unified log management platform.

[0169] According to embodiments of this disclosure, the method collects resource usage information data such as CPU, memory, GPU, and disk of the server, and invokes big data algorithms for anomaly detection and prediction. Operations and maintenance personnel can quickly monitor server resource usage, thereby enabling more efficient resource allocation, timely handling of anomalies, and improved server performance and operational efficiency. For data requiring alarms for anomaly detection, automatic alarms can be implemented by calling the log management platform interface, effectively reducing the burden on operations and maintenance personnel.

[0170] Based on the above-described monitoring data processing method, this disclosure also provides a monitoring data processing device. The device will be described in detail below with reference to Figure 9.

[0171] Figure 9 is a structural block diagram of a monitoring data processing apparatus according to an embodiment of the present disclosure.

[0172] As shown in Figure 9, the monitoring data processing device 900 of this embodiment includes a determination module 910, an acquisition module 920, a calling module 930, and a obtaining module 940.

[0173] The determination module 910 is used to determine data acquisition rules based on the request processing type contained in the monitoring request in response to a monitoring request for the target device. In one embodiment, the determination module 910 can be used to perform the operation S110 described above, which will not be repeated here.

[0174] The acquisition module 920 is used to acquire target monitoring data for the target device according to data acquisition rules, wherein the target monitoring data includes at least one type of indicator data. In one embodiment, the acquisition module 920 can be used to perform the operation S120 described above, which will not be repeated here.

[0175] The calling module 930 is used to call the target model for each type of indicator data in the target monitoring data, based on the data type of the indicator data and the request processing type. In one embodiment, the calling module 930 can be used to perform the operation S130 described above, which will not be repeated here.

[0176] The obtaining module 940 is used to process the indicator data using the target model to obtain a processing result corresponding to the data type of the indicator data. In one embodiment, the obtaining module 940 can be used to perform the operation S140 described above, which will not be repeated here.

[0177] According to embodiments of this disclosure, request processing types include anomaly detection and anomaly prediction.

[0178] According to embodiments of this disclosure, when the request processing type is anomaly detection, the data acquisition rules include acquiring data within a first time period with a first acquisition cycle as the time interval; the monitoring request includes a data type identifier.

[0179] According to embodiments of this disclosure, the acquisition module includes a first acquisition submodule.

[0180] The first acquisition submodule is used to acquire data generated by the target device within a first time period, corresponding to the data type identifier, at a time interval of the first acquisition cycle, to obtain the target monitoring data.

[0181] According to embodiments of this disclosure, the above-described monitoring data processing method further includes a first display module.

[0182] The first display module is used to show the processing results corresponding to the data type of the indicator data.

[0183] According to embodiments of this disclosure, the first display module includes an anomaly detection submodule, a display submodule, and a recording submodule.

[0184] The anomaly detection submodule is used to perform anomaly detection processing on the indicator data using the target model and output alarm indicators corresponding to the indicator data.

[0185] The display submodule is used to display indicator data and alarm messages on the display interface when an alarm is required based on the alarm identifier.

[0186] The recording submodule is used to record that the alarm status is on and to record the alarm start time.

[0187] According to embodiments of this disclosure, the anomaly detection submodule is further configured to perform anomaly detection processing on indicator data collected after the alarm start time, and output an alarm identifier indicating that no alarm information is sent when an alarm is required.

[0188] According to embodiments of this disclosure, the above-mentioned anomaly detection submodule is further used to perform anomaly detection processing on indicator data collected after the alarm start time, and the output alarm identifier indicates that if no alarm is needed, information to disable the alarm is sent.

[0189] The recording submodule is also used to record alarm status as off.

[0190] According to embodiments of this disclosure, the index data includes real data at m time points, where m ≥ 1.

[0191] According to embodiments of this disclosure, the anomaly detection submodule includes a generation unit and an output unit.

[0192] The generation unit is used to generate predicted data for time m based on real data at m time points.

[0193] The output unit is used to output an alarm flag based on the actual data at time m, the predicted data at time m, and preset rules.

[0194] According to embodiments of this disclosure, the output unit includes: a first output subunit and a second output subunit.

[0195] The first output subunit is used to output an alarm flag that requires an alarm when the actual data at time m and the predicted data at time m meet the preset rules.

[0196] The second output subunit is used to output an alarm flag that does not require alarming when it is determined that the actual data at time m and the predicted data at time m do not meet the preset rules.

[0197] According to embodiments of this disclosure, when the request processing type is anomaly prediction, the data acquisition rules include acquiring data within a second time period with a second collection cycle as the time interval; the monitoring request includes a data type identifier.

[0198] According to embodiments of this disclosure, the acquisition module further includes a second acquisition submodule.

[0199] The second acquisition submodule is used to acquire data generated by the target device within the second time period, corresponding to the data type identifier, at a time interval of the second acquisition cycle, to obtain the target monitoring data.

[0200] According to embodiments of this disclosure, the resulting module includes an anomaly prediction submodule.

[0201] The anomaly prediction submodule is used to perform anomaly prediction processing on the indicator data using the target model, and obtain the predicted data corresponding to the indicator data.

[0202] According to embodiments of this disclosure, the above-mentioned monitoring data processing device further includes a data processing module.

[0203] The data processing module is used to process the target monitoring data according to a preset format to obtain the model input data when it is determined that the target monitoring data does not meet the model input format.

[0204] According to embodiments of this disclosure, the above-mentioned monitoring data processing further includes: a query module and a second display module.

[0205] The query module is used to respond to query requests for historical alarm data and determine the target time period based on the query start time and query end time contained in the query request.

[0206] The second display module is used to display alarm data generated within the target time period on the display interface.

[0207] According to embodiments of this disclosure, any plurality of modules among the determining module 910, obtaining module 920, calling module 930, and obtaining module 940 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the determining module 910, obtaining module 920, calling module 930, and obtaining module 940 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the determining module 910, obtaining module 920, calling module 930, and obtaining module 940 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0208] Figure 10 is a block diagram of an electronic device suitable for implementing a monitoring data processing method according to an embodiment of the present disclosure.

[0209] As shown in FIG10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage portion 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0210] RAM 1003 stores various programs and data required for the operation of electronic device 1000. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Processor 1001 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1002 and / or RAM 1003. It should be noted that the programs may also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0211] According to embodiments of this disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to a bus 1004. The electronic device 1000 may also include one or more of the following components connected to the I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0212] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0213] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1002 and / or RAM 1003 and / or one or more memories other than ROM 1002 and RAM 1003 described above.

[0214] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the monitoring data processing method provided in the embodiments of this disclosure.

[0215] When the computer program is executed by the processor 1001, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0216] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1009, and / or installed from a removable medium 1011. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0217] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by processor 1001, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0218] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0219] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0220] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0221] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for processing monitoring data, the method comprising: In response to a monitoring request for a target device, data acquisition rules are determined based on the request processing type contained in the monitoring request; According to the data acquisition rules, target monitoring data for the target device is acquired, wherein the target monitoring data includes at least one type of indicator data; For each type of metric data in the target monitoring data, the target model is invoked according to the data type of the metric data and the request processing type; and The target model is used to process the indicator data to obtain a processing result corresponding to the data type of the indicator data.

2. The method according to claim 1, wherein, The request processing types include anomaly detection and anomaly prediction.

3. The method according to claim 2, wherein, When the request processing type is the anomaly detection, the data acquisition rule includes acquiring data within a first time period with a first acquisition cycle as the time interval; The monitoring request includes a data type identifier; The step of obtaining target monitoring data for the target device according to the data acquisition rules includes: The target monitoring data is obtained by acquiring data generated by the target device within the first time period corresponding to the data type identifier, with the first acquisition cycle as the time interval.

4. The method according to claim 3, further comprising displaying a processing result corresponding to the data type of the indicator data, including: The target model is used to perform anomaly detection processing on the indicator data, and an alarm identifier corresponding to the indicator data is output. Based on the alarm indicator, if it is determined that an alarm is needed, the indicator data and alarm message will be displayed on the display interface. Record that the alarm status is "on" and record the alarm start time.

5. The method according to claim 4, wherein, After recording that the alarm status is enabled and the alarm start time is recorded, the following is also included: The indicator data collected after the alarm start time is subjected to anomaly detection processing, and the output alarm identifier indicates that no alarm information is sent when an alarm is required.

6. The method according to claim 5, further comprising: Anomaly detection processing is performed on the indicator data collected after the alarm start time, and the output alarm identifier indicates that if the alarm is not needed, an alarm shutdown message is sent. The alarm status is recorded as off.

7. The method according to claim 4, wherein, The indicator data includes real data at m time points, where m ≥ 1; The target model is used to perform anomaly detection processing on the indicator data, and alarm identifiers corresponding to the indicator data are output, including: Generate the predicted data for the m-th time based on the actual data at the m-th time points; Based on the actual data at time m, the predicted data at time m, and the preset rules, the alarm identifier is output.

8. The method according to claim 7, wherein, The step of outputting the alarm identifier based on the actual data at time m, the predicted data at time m, and preset rules includes: If the actual data at time m and the predicted data at time m satisfy the preset rule, an alarm flag that requires an alarm will be output. If the actual data at time m and the predicted data at time m do not meet the preset rules, an alarm flag indicating that no alarm is required will be output.

9. The method according to claim 2, wherein, When the request processing type is anomaly prediction, the data acquisition rule includes acquiring data within a second time period with a second collection cycle as the time interval; The monitoring request includes a data type identifier; The step of obtaining target monitoring data for the target device according to the data acquisition rules includes: The target monitoring data is obtained by acquiring data generated by the target device within the second time period, corresponding to the data type identifier, with the second acquisition cycle as the time interval; The target model is used to process the indicator data to obtain processing results corresponding to the data type of the indicator data, including: The target model is used to perform anomaly prediction processing on the indicator data to obtain the predicted data corresponding to the indicator data.

10. The method according to claim 1, further comprising: If it is determined that the target monitoring data does not meet the model input format, the target monitoring data is processed according to a preset format to obtain the model input data.

11. The method according to claim 3, further comprising: In response to a query request for historical alarm data, the target time period is determined based on the query start time and query end time contained in the query request; The display interface shows the alarm data generated during the target time period.

12. A monitoring data processing device, the device comprising: The determination module is used to determine data acquisition rules based on the request processing type contained in the monitoring request in response to a monitoring request for a target device. The acquisition module is used to acquire target monitoring data for the target device according to the data acquisition rules, wherein the target monitoring data includes at least one type of indicator data; The calling module is used to call the target model for each type of indicator data in the target monitoring data, based on the data type of the indicator data and the request processing type; and The module is used to process the indicator data using the target model to obtain a processing result corresponding to the data type of the indicator data.

13. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program or instructions stored thereon, wherein, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 11.

Images