Multi-dimensional operation and maintenance time series data anomaly detection method and system

Through the multi-dimensional operation and maintenance timing data abnormal detection method, combined with graph attention and LSTM algorithm, the normal timing is reconstructed, and the problems of false alarms and missed alarms in the existing technology are solved, thus achieving high-accurate abnormal detection effect.

CN119760319APending Publication Date: 2025-04-04ECCOM NETWORK SYST CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411891684.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-20
Publication Date
2025-04-04

AI Technical Summary

Technical Problem

The prior art is difficult to consider the historical regularity of single-dimensional timing data and the mutual influence relationship between multi-dimensional timing data, resulting in the possibility of false alarms when abnormal detection is prone to occur.

Method used

The multi-dimensional operation and maintenance timing data abnormal detection method is adopted, and through preprocessing, grouping, convolution processing, graph attention processing, LSTM algorithm and diffusion algorithm, the correlation relationship and historical change laws between timing data are captured, the normal timing is reconstructed and abnormal detection is performed.

Benefits of technology

Effectively eliminate false alarms and missed alarms, improve the accuracy and efficiency of abnormal detection, ensure that the data in each independent group has mutual influence, and is suitable for abnormal detection of multi-dimensional time series data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119760319A_ABST
    Figure CN119760319A_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for detecting anomalies in multi-dimensional operation and maintenance time series data, including: step S1: preprocessing the data to be detected; step S2: grouping the data to be detected; step S3: convolution processing the grouped data to obtain multi-dimensional time series data; step S4: graph attention processing the multi-dimensional time series data; step S5: using the LSTM algorithm to obtain the historical change law of the multi-dimensional time series data; step S6: compressing the multi-dimensional time series data, eliminating anomalies, and reconstructing the normal time series through the diffusion algorithm to obtain the reconstruction result; step S7: comparing and judging the data to be detected and the reconstruction result, and outputting the anomaly determination result. The present invention groups the massive time series data generated in the operation and maintenance object, and groups the time series that are likely to affect each other into a group, and performs anomaly detection on the data in the same group together, so as to ensure the independence between groups and assist in obtaining the correlation between the time series within the group.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of IT operation and maintenance technology, and specifically, relates to a multi-dimensional operation and maintenance time series data anomaly detection method and system, and more specifically, a multi-dimensional operation and maintenance time series data anomaly detection method that focuses on the mutual influence relationship between dimensions. Background Art

[0002] Multi-dimensional time series data is a common indicator data in operation and maintenance scenarios, such as network flow data and server performance data in data center operation and maintenance scenarios, and sensor data in IoT scenarios.

[0003] Anomaly detection on this type of data can timely discover abnormal conditions of the entire system and take timely intervention measures. In actual situations, the changes in this type of data are quite complex; on the one hand, from a single-dimensional perspective, the value of the time series will change over time and show a certain historical regularity; on the other hand, from a cross-dimensional perspective, there is a mutual influence relationship between the dimensions, and their changes show a consistent trend.

[0004] At present, there are many methods for detecting anomalies in time series data with a single dimension, but only detecting each time series data as an isolated object is prone to false positives. Considering time series data across multiple dimensions requires the detection algorithm to capture the mutual influence relationship between dimensions.

[0005] Therefore, when performing anomaly detection on multidimensional time series, the above two aspects of the law cannot be taken into account at the same time, and this problem needs to be solved urgently. Summary of the invention

[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a multi-dimensional operation and maintenance time series data anomaly detection method and system.

[0007] A multi-dimensional operation and maintenance time series data anomaly detection method provided by the present invention includes:

[0008] Step S1: preprocessing the data to be detected;

[0009] Step S2: grouping the data to be detected to obtain grouped data;

[0010] Step S3: performing convolution processing on the grouped data to obtain multi-dimensional time series data;

[0011] Step S4: Processing the multi-dimensional time series data with graph attention;

[0012] Step S5: compressing the multi-dimensional time series data, removing anomalies, and reconstructing the normal time series through a diffusion algorithm to obtain a reconstruction result;

[0013] Step S6: Compare and judge the data to be detected and the reconstruction result, and output an abnormality determination result.

[0014] Preferably, in the step S1, the missing value of each time series data of the data to be detected is supplemented and normalized;

[0015] In step S2, the data to be detected are grouped according to the business system; based on the features of the data to be detected, the features of each piece of the data to be detected are reduced to 2 dimensions by the t-SNE algorithm to obtain a two-dimensional plane graph as a result of the dimension reduction, and grouping is performed according to the two-dimensional plane graph to obtain grouped data;

[0016] The data to be detected are represented by coordinate points in the two-dimensional plane diagram;

[0017] The business system includes: mail system and order management system;

[0018] The characteristics of the data to be detected include: device type and performance level.

[0019] Preferably, in step S4, the expression of the multidimensional time series data after convolution processing is:

[0020] V={v1,v2,v3,…,v n}

[0021] Among them, V is multidimensional time series data, n is the dimension, and v i is the vector corresponding to each time series, i is 1 to n;

[0022] In step S4:

[0023] Step S4.1: representing the multidimensional time series of the multidimensional time series data as a multidimensional time series graph;

[0024] Step S4.2: Calculate the graph attention based on the multi-dimensional time series graph;

[0025] Step S4.3: Aggregating the multi-dimensional time series data and graph attention;

[0026] In step S4.2, the mathematical expression of graph attention calculation is:

[0027]

[0028] Among them, h i is the node vector v i After the graph attention calculation, it is transformed into the representation form, σ is the activation function, α ij is the node vector v i and its neighboring node vector v jThe attention value between them, that is, the relationship value; L is the node vector v i The number of all adjacent nodes, L = n;

[0029] The mathematical expression of the attention value is:

[0030]

[0031] Among them, α ij is the attention value, is a connection symbol, which connects two nodes, w is a vector of learnable parameters, is the activation function; exp is the natural exponential function; the symbol · is the dot product;

[0032] In step S5, the historical change law of the multidimensional time series data is obtained through the LSTM algorithm; based on the historical change law and the graph attention, the normal time series is reconstructed through the diffusion algorithm to obtain the reconstruction result.

[0033] Preferably, in step S5, the multidimensional time series data is compressed by a self-encoding algorithm and its abnormal points are preliminarily removed to obtain compressed data;

[0034] The steps of the diffusion algorithm include:

[0035] Noise adding step: adding Gaussian noise to the compressed data to obtain noisy compressed data;

[0036] De-noising step: iteratively perform denoising and reconstruction processing on the noisy compressed data for the same number of times, and output a reconstruction result.

[0037] Preferably, in step S6, the difference between each time series of the reconstruction result and the corresponding time series in the data to be detected is calculated to obtain a difference, that is, a time series difference; the deviation threshold is set to T; and whether the time series difference is greater than T is determined one by one. If the result is yes, the time points of the two time series corresponding to the time series difference are abnormal points; if the result is no, no processing is performed.

[0038] A multi-dimensional operation and maintenance time series data anomaly detection system provided by the present invention includes:

[0039] Module M1: preprocessing the data to be detected;

[0040] Module M2: grouping the data to be detected to obtain grouped data;

[0041] Module M3: convolution processing the grouped data to obtain multi-dimensional time series data;

[0042] Module M4: graph attention processing of the multi-dimensional time series data;

[0043] Module M5: compress the multi-dimensional time series data, remove anomalies, and reconstruct the normal time series through a diffusion algorithm to obtain a reconstruction result;

[0044] Module M6: compare and judge the data to be detected and the reconstruction result, and output an abnormality determination result.

[0045] Preferably, in the module M1, the missing value of each time series data of the data to be detected is supplemented and normalized;

[0046] In the module M2, the data to be detected are grouped according to the business system; based on the features of the data to be detected, the features of each piece of the data to be detected are reduced to 2 dimensions by the t-SNE algorithm to obtain a two-dimensional plane graph as a result of the dimension reduction, and grouping is performed according to the two-dimensional plane graph to obtain grouped data;

[0047] The data to be detected are represented by coordinate points in the two-dimensional plane diagram;

[0048] The business system includes: mail system and order management system;

[0049] The characteristics of the data to be detected include: device type and performance level.

[0050] Preferably, in the module M4, the expression of the multidimensional time series data after convolution processing is:

[0051] V={v1,v2,v3,…,v n}

[0052] Among them, V is multidimensional time series data, n is the dimension, and v i is the vector corresponding to each time series, i is 1 to n;

[0053] In the module M4:

[0054] Module M4.1: representing the multidimensional time series of the multidimensional time series data as a multidimensional time series graph;

[0055] Module M4.2: Calculate graph attention based on multi-dimensional time series graph;

[0056] Module M4.3: Aggregate the multi-dimensional time series data and graph attention;

[0057] In the module M4.2, the mathematical expression of graph attention calculation is:

[0058]

[0059] Among them, h i is the node vector v iAfter the graph attention calculation, it is transformed into the representation form, σ is the activation function, α ij is the node vector v i and its neighboring node vector v j The attention value between them, that is, the relationship value; L is the node vector v i The number of all adjacent nodes, L = n;

[0060] The mathematical expression of the attention value is:

[0061]

[0062] Among them, α ij is the attention value, is a connection symbol, which connects two nodes, w is a vector of learnable parameters, is the activation function; exp is the natural exponential function; the symbol · is the dot product;

[0063] In the module M5, the historical change law of the multi-dimensional time series data is obtained through the LSTM algorithm; according to the historical change law and the graph attention, the normal time series is reconstructed through the diffusion algorithm to obtain the reconstruction result.

[0064] Preferably, in the module M5, the multi-dimensional time series data is compressed by a self-encoding algorithm and its abnormal points are preliminarily removed to obtain compressed data;

[0065] The module of the diffusion algorithm includes:

[0066] Noise adding module: adding Gaussian noise to the compressed data to obtain noisy compressed data;

[0067] Denoising module: iteratively performs denoising and reconstruction processing on the noisy compressed data for the same number of times, and outputs the reconstruction result.

[0068] Preferably, in the module M6, the difference between each time series of the reconstruction result and the corresponding time series in the data to be detected is calculated to obtain a difference, that is, a time series difference; the deviation threshold is set to T; and whether the time series difference is greater than T is determined one by one. If the result is yes, the time points of the two time series corresponding to the time series difference are abnormal points; if the result is no, no processing is performed.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] 1. The present invention groups the massive time series data generated in the operation and maintenance objects, and groups the time series that are likely to affect each other into one group. The data in the same group are jointly detected for anomalies, thereby ensuring the independence between groups and helping to obtain the correlation between the time series within the group.

[0071] 2. The present invention adopts the graph attention method and LSTM algorithm to capture the correlation between time series and the historical change rules of time series, which helps to comprehensively and accurately judge abnormal changes and eliminate false alarms.

[0072] 3. The present invention combines the use of an autoencoding algorithm and a diffusion algorithm in the data reconstruction process. Compared with only using an autoencoding algorithm or only using a diffusion algorithm, this combined use method can effectively eliminate abnormal components in the time series and obtain a smoother reconstruction result, thereby eliminating underreporting; in other words, obtaining high accuracy in anomaly detection is the greatest benefit of the present invention. In addition, since multi-dimensional time series data of a data group can perform anomaly detection at the same time, the detection rate is improved.

[0073] 4. The present invention adopts artificial intelligence algorithm to realize accurate anomaly detection of multi-dimensional time series data in operation and maintenance scenarios, and groups the massive and mixed time series data in IT systems to ensure that the data in each independent group has mutual influence and is suitable for participating in detection as a group of multi-dimensional data.

[0074] 5. In the present invention, the reconstruction process of the normal state of the time series data combines the use of the autoencoding algorithm and the diffusion algorithm to obtain a better reconstruction effect, which can reduce the situation of missed detection and thus improve the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0076] Figure 1 A schematic diagram of the data processing process provided by the present invention;

[0077] Figure 2 A schematic diagram of the attention calculation process provided by the present invention. DETAILED DESCRIPTION

[0078] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0079] The anomaly detection algorithm provided by the present invention, that is, the LSTM algorithm of this solution, not only considers the historical change law of a single target time series, but also takes into account the changes of other dimensional time series related to the time series, and improves the detection accuracy through the influence of the graph processing algorithm on the target time series.

[0080] For a group of multidimensional time series data, which includes multiple time series data, the algorithm of the present invention adopts a one-time detection mode, which can directly obtain the abnormal judgment results of each time series data in the group through one detection without performing detection one by one, thereby improving the detection efficiency.

[0081] A basic and important data type in operation and maintenance monitoring data is indicator-type time series data, which is characterized by a large amount of data and variable duration. Specifically, it may be a long sequence or a short sequence, which will change with actual needs, and there is mutual influence between indicators of each dimension, and there is a certain correlation. If anomalies of this type of data can be accurately judged in an automated manner, it can effectively improve the efficiency of operation and maintenance, reduce the workload of operation and maintenance personnel, and guide corresponding personnel to take timely intervention measures. In other words, with the advancement of machine learning technology, a feasible solution is provided for achieving the above goals. The present invention uses machine learning methods to meet this demand, which can be migrated and applied to other scenarios outside the field of IT operation and maintenance, and expand its application in similar scenarios in the industrial, market, and even financial fields.

[0082] A multi-dimensional operation and maintenance time series data anomaly detection method provided by the present invention includes:

[0083] Step 1: Preprocess the data to be tested;

[0084] Step 2: Group the data to be detected to obtain grouped data;

[0085] Step 3: Convolutionally process the grouped data to obtain multi-dimensional time series data;

[0086] Step 4: Process the multi-dimensional time series data with graph attention to obtain a graph attention value of the multi-dimensional time series data;

[0087] Step 5: Using the LSTM algorithm, the historical change pattern of the multi-dimensional time series data is obtained;

[0088] Step 6: compress the multi-dimensional time series data, remove anomalies, and reconstruct the normal time series through a diffusion algorithm based on the historical change law and the graph attention value to obtain a reconstruction result;

[0089] Step 7: Compare and evaluate the data to be detected and the reconstruction result, and output an abnormality determination result.

[0090] Specifically, for multidimensional time series data, before reconstructing it, it is necessary to capture the closeness of the relationship between the dimensions, which is obtained in step 4, and also to capture the change law of each dimension itself over time, that is, the "historical change law"; this can also be understood as the causal dependency between the data points in a time series. These laws are obtained in step 5. The above two types of information can help the reconstruction process obtain better reconstruction results.

[0091] Specifically, the present invention includes the following technical links, and each link is as follows: Figure 1 Combine the processes shown:

[0092] 1. Data aggregation and preprocessing: Aggregate all the data to be tested in the entire IT system, supplement missing values ​​for each time series data, and perform normalization.

[0093] 2. Data grouping: First, group the devices according to the actual business system, then collect the characteristics of each indicator to be tested for the devices in the group, and use the t-SNE algorithm to implement indicator grouping detection to obtain grouped data. Specifically, the t-SNE method can obtain high-quality dimensionality reduction clustering results on the one hand, and its good visualization effect on the other hand helps humans understand and intervene in data grouping results.

[0094] Specifically, according to the aforementioned "first group the devices according to the actual business system, and then collect the characteristics of each indicator to be detected of the devices in the group", several preliminary grouped data sets can be obtained;

[0095] Assume that one of the data sets is G, which contains several time series data to be detected and some features corresponding to each data; the features include: what type of device the data comes from, the device brand, the IP segment and the value range; assume that there are F features in total. For G, the time series data in it is further subdivided according to its corresponding features; assume that after further subdivision, several groups will be generated, one of which is g; then, all the time series data in g are sent as a whole to the subsequent multi-dimensional anomaly detection process. This can improve the accuracy of detection.

[0096] The t-SNE method is used in the segmentation process from G to g.

[0097] Specifically, with the help of t-SNE, the aforementioned F-dimensional features corresponding to each time series data are reduced to 2 dimensions, and it is ensured that the dimension reduction process can still retain the similarity relationship of the data. After the dimension reduction, each time series can be represented by a point in the two-dimensional plane coordinate system, that is, the dimension reduction result; specifically, one point corresponds to a time series data, and those points clustered together in the coordinate system are suitable as points for a subdivision group g. After manually observing the clustering situation, group selection is performed, and the time series data corresponding to these clustered points are extracted, merged, and sent to the subsequent multi-dimensional anomaly detection process.

[0098] Since the resulting image drawn by t-SNE is a two-dimensional plane graph with good visualization effect, it is very helpful for manual understanding and processing of data grouping steps.

[0099] In other words, the data to be detected are grouped according to the business system; based on the features of the data to be detected, the features of each piece of the data to be detected are reduced to 2 dimensions by the t-SNE algorithm to obtain a two-dimensional plane graph as a result of the dimension reduction, and grouping is performed according to the two-dimensional plane graph to obtain grouped data;

[0100] The data to be detected are represented by coordinate points in the two-dimensional plane diagram;

[0101] The business system includes: mail system and order management system;

[0102] The characteristics of the indicator to be detected include: device type and performance level.

[0103] 3. Convolution processing: For each set of data, which contains N-dimensional time series data, first perform a one-dimensional convolution operation on it, and through convolution, the characteristics of each time series can be obtained.

[0104] 4. Graph attention processing: Graph convolution processing can express the closeness of the relationship between nodes in the graph network, such as Figure 2 As shown, suppose the expression of multidimensional time series data after convolution processing is:

[0105] V={v1,v2,v3,…,v n}

[0106] Among them, V is multidimensional time series data, n is the dimension, and v i is the vector corresponding to each time series, and i ranges from 1 to n.

[0107] After the multidimensional time series data is represented as a graph, each node in the graph represents a time series, and the lines between the nodes represent the relationship value between the time series, which is obtained by graph attention calculation. For each dimension vector v i , the output h after graph attention calculation ifor:

[0108]

[0109] Among them, h i is the node vector v i After the graph attention calculation, it is converted into a representation, which is a vector and the node vector v i Similar; σ is the activation function; α ij is the node vector v i and one of its adjacent node vectors v j The attention value between them, that is, the relationship value between them. In particular, when j = i, α ij Transformed into alpha ii , which represents the node vector v i The relationship value with itself. L is the value containing the node v i The number of all adjacent nodes including itself, so L = n.

[0110] The mathematical expression of attention value is:

[0111]

[0112] Among them, α ij is the attention value, is a connection symbol, which connects two nodes, w is a vector of learnable parameters, is the activation function; exp is the natural exponential function; the symbol · is the dot product;

[0113] Each h vector obtained after the graph attention calculation will be aggregated with the vector V obtained after the previous convolution step and then sent to the subsequent steps.

[0114] 5. Long Short-Term Memory Network (LSTM) regression processing: In order to capture the historical change rules of each time series data, the present invention adopts the LSTM algorithm.

[0115] As a special recurrent neural network structure, the LSTM includes three gating mechanisms: input gate, forget gate and output gate, as well as a memory unit for storing long-term information. This structure can capture long-term dependencies in time series regression problems and reveal the historical change patterns of data.

[0116] 6. Auto-encoding reconstruction processing; the purpose of reconstruction is to remove the abnormal part of the time series and only retain the normal time series form.

[0117] When implementing, it should be noted that when these two reconstruction methods are used together, they should be performed in the order of self-encoding reconstruction first and then diffusion reconstruction. The order of these two steps cannot be reversed.

[0118] The reconstruction process of the present invention first adopts an autoencoding algorithm, in which the encoder compresses the time series data into a latent space, which can capture and express the key features and dynamic characteristics of the time series data and eliminate the influence of outliers, while the decoder is responsible for reconstructing the normal time series from the latent space.

[0119] 7. Diffusion reconstruction processing: The result of reconstruction using only the autoencoding algorithm will have many abnormal residues. In order to further eliminate these residues, the present invention uses a diffusion algorithm to further reconstruct the data.

[0120] Specifically, the diffusion algorithm is divided into two steps: denoising and denoising. Let the input before diffusion be X and the output after diffusion reconstruction be X′. The entire diffusion process can be expressed as:

[0121]

[0122] First, Gaussian noise is added to X one by one, and then the denoising and reconstruction process is iterated the same number of times to obtain a smoother time series output after removing the abnormal residue.

[0123] Specifically, diffusion reconstruction includes a forward process and a backward process, specifically:

[0124] 1. Forward process: gradual noise addition

[0125] In each step of the forward process, Gaussian noise is added to the input data. After M iterations, the original data is gradually transformed into a noise data close to Gaussian distribution.

[0126] 2. Backward process: gradual denoising

[0127] Starting from the final noisy data of the forward process, an existing, trained deep learning model that learns the inverse process of adding noise in the forward process is used to predict and remove the noise, and reconstruct the data after M denoising steps.

[0128] The above M is a preset parameter, which can be determined by observing how many times the original input data will be converted into uniform noise data after adding noise.

[0129] 8. Anomaly judgment: The result finally generated by the reconstruction process is set as R, and the original data after preprocessing is set as O, and they enter the final anomaly judgment link together. The judgment process is actually to find the difference between each time series in O and the corresponding time series in R, that is, to calculate the distance between each time point of the two time series. Let the deviation threshold be T, then the point whose distance exceeds T is judged as an abnormal point, and the abnormal judgment result is identified and output accordingly. In other words, it is judged whether the distance between the time points of the two time series exceeds the deviation threshold. If the result is yes, the time point is an abnormal point; if the result is no, it is not processed.

[0130] In other words, the difference between each time series of the reconstructed result and the corresponding time series in the data to be detected is calculated to obtain the difference, that is, the time series difference; the deviation threshold is set to T; whether the time series difference is greater than T is determined one by one. If the result is yes, the time points of the two time series corresponding to the time series difference are abnormal points; if the result is no, no processing is performed.

[0131] The present invention also provides a multi-dimensional operation and maintenance time series data anomaly detection system, which can be implemented by executing the process steps of the multi-dimensional operation and maintenance time series data anomaly detection method, that is, technical personnel in this field can understand the multi-dimensional operation and maintenance time series data anomaly detection method as a preferred implementation of the multi-dimensional operation and maintenance time series data anomaly detection system.

[0132] A multi-dimensional operation and maintenance time series data anomaly detection system provided by the present invention includes:

[0133] Module M1: preprocessing the data to be detected;

[0134] Module M2: grouping the data to be detected to obtain grouped data;

[0135] Module M3: convolution processing the grouped data to obtain multi-dimensional time series data;

[0136] Module M4: graph attention processing of the multi-dimensional time series data;

[0137] Module M5: compress the multi-dimensional time series data, remove anomalies, and reconstruct the normal time series through a diffusion algorithm to obtain a reconstruction result;

[0138] Module M6: compare and judge the data to be detected and the reconstruction result, and output an abnormality determination result.

[0139] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0140] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A multi-dimensional operation and maintenance time series data anomaly detection method, characterized in that: include: Step S1: preprocessing the data to be detected; Step S2: grouping the data to be detected to obtain grouped data; Step S3: performing convolution processing on the grouped data to obtain multi-dimensional time series data; Step S4: Processing the multi-dimensional time series data with graph attention; Step S5: compressing the multi-dimensional time series data, removing anomalies, and reconstructing the normal time series through a diffusion algorithm to obtain a reconstruction result; Step S6: Compare and judge the data to be detected and the reconstruction result, and output an abnormality determination result.

2. The multi-dimensional operation and maintenance time series data anomaly detection method according to claim 1 is characterized in that: In the step S1, the missing values ​​of each time series data of the data to be detected are supplemented and normalized; In step S2, the data to be detected are grouped according to the business system; based on the features of the data to be detected, the features of each piece of the data to be detected are reduced to 2 dimensions by the t-SNE algorithm to obtain a two-dimensional plane graph as a result of the dimension reduction, and grouping is performed according to the two-dimensional plane graph to obtain grouped data; The data to be detected are represented by coordinate points in the two-dimensional plane diagram; The business system includes: mail system and order management system; The characteristics of the data to be detected include: device type and performance level.

3. The multi-dimensional operation and maintenance time series data anomaly detection method according to claim 2 is characterized in that: In step S4, the expression of the multidimensional time series data after convolution processing is: V={v1,v2,v3,…,v n } Among them, V is multidimensional time series data, n is the dimension, and v i is the vector corresponding to each time series, i is 1 to n; In step S4: Step S4.1: representing the multidimensional time series of the multidimensional time series data as a multidimensional time series graph; Step S4.2: Calculate the graph attention based on the multi-dimensional time series graph; Step S4.3: Aggregating the multi-dimensional time series data and graph attention; In step S4.2, the mathematical expression of graph attention calculation is: Among them, h i is the node vector v i After the graph attention calculation, it is transformed into the representation form, σ is the activation function, α ij is the node vector v i and its neighboring node vector v j The attention value between them, that is, the relationship value; L is the node vector v i The number of all adjacent nodes, L = n; The mathematical expression of the attention value is: Among them, α ij is the attention value, is a connection symbol, which connects two nodes, w is a vector of learnable parameters, is the activation function; exp is the natural exponential function; the symbol · is the dot product; In step S5, the historical change law of the multidimensional time series data is obtained through the LSTM algorithm; based on the historical change law and the graph attention, the normal time series is reconstructed through the diffusion algorithm to obtain the reconstruction result.

4. The multi-dimensional operation and maintenance time series data anomaly detection method according to claim 3 is characterized in that: In step S5, the multidimensional time series data is compressed by a self-encoding algorithm and its abnormal points are preliminarily removed to obtain compressed data; The steps of the diffusion algorithm include: Noise adding step: adding Gaussian noise to the compressed data to obtain noisy compressed data; De-noising step: iteratively perform denoising and reconstruction processing on the noisy compressed data for the same number of times, and output a reconstruction result.

5. The multi-dimensional operation and maintenance time series data anomaly detection method according to claim 4 is characterized in that: In step S6, the difference between each time series of the reconstruction result and the corresponding time series in the data to be detected is calculated to obtain a difference value, that is, a time series difference value; Assume that the deviation threshold is T; determine whether the time series difference is greater than T one by one, and if the result is yes, then the time points of the two time series corresponding to the time series difference are abnormal points; If the result is no, no processing is performed.

6. A multi-dimensional operation and maintenance time series data anomaly detection system, characterized in that: include: Module M1: preprocessing the data to be detected; Module M2: grouping the data to be detected to obtain grouped data; Module M3: convolution processing the grouped data to obtain multi-dimensional time series data; Module M4: graph attention processing of the multi-dimensional time series data; Module M5: compress the multi-dimensional time series data, remove anomalies, and reconstruct the normal time series through a diffusion algorithm to obtain a reconstruction result; Module M6: compare and judge the data to be detected and the reconstruction result, and output an abnormality determination result.

7. The multi-dimensional operation and maintenance time series data anomaly detection system according to claim 6, characterized in that: In the module M1, the missing value of each time series data of the data to be detected is supplemented and normalized; In the module M2, the data to be detected are grouped according to the business system; based on the features of the data to be detected, the features of each piece of the data to be detected are reduced to 2 dimensions by the t-SNE algorithm to obtain a two-dimensional plane graph as a result of the dimension reduction, and grouping is performed according to the two-dimensional plane graph to obtain grouped data; The data to be detected are represented by coordinate points in the two-dimensional plane diagram; The business system includes: mail system and order management system; The characteristics of the data to be detected include: device type and performance level.

8. The multi-dimensional operation and maintenance time series data anomaly detection system according to claim 7, characterized in that: In the module M4, the expression of the multi-dimensional time series data after convolution processing is: V={v1,v2,v3,…,v n } Among them, V is multidimensional time series data, n is the dimension, and v i is the vector corresponding to each time series, i is 1 to n; In the module M4: Module M4.1: representing the multidimensional time series of the multidimensional time series data as a multidimensional time series graph; Module M4.2: Calculate graph attention based on multi-dimensional time series graph; Module M4.3: Aggregate the multi-dimensional time series data and graph attention; In the module M4.2, the mathematical expression of graph attention calculation is: Among them, h i is the node vector v i After the graph attention calculation, it is transformed into the representation form, σ is the activation function, α ij is the node vector v i and its neighboring node vector v j The attention value between them, that is, the relationship value; L is the node vector v i The number of all adjacent nodes, L = n; The mathematical expression of the attention value is: Among them, α ij is the attention value, is a connection symbol, which connects two nodes, w is a vector of learnable parameters, is the activation function; exp is the natural exponential function; the symbol · is the dot product; In the module M5, the historical change law of the multi-dimensional time series data is obtained through the LSTM algorithm; according to the historical change law and the graph attention, the normal time series is reconstructed through the diffusion algorithm to obtain the reconstruction result.

9. The multi-dimensional operation and maintenance time series data anomaly detection system according to claim 8, characterized in that: In the module M5, the multi-dimensional time series data is compressed by a self-encoding algorithm and its abnormal points are preliminarily removed to obtain compressed data; The module of the diffusion algorithm includes: Noise adding module: adding Gaussian noise to the compressed data to obtain noisy compressed data; Denoising module: iteratively performs denoising and reconstruction processing on the noisy compressed data for the same number of times, and outputs the reconstruction result.

10. The multi-dimensional operation and maintenance time series data anomaly detection system according to claim 9, characterized in that: In the module M6, the difference between each time series of the reconstruction result and the corresponding time series in the data to be detected is calculated to obtain a difference value, that is, a time series difference value; Assume that the deviation threshold is T; determine whether the time series difference is greater than T one by one, and if the result is yes, then the time points of the two time series corresponding to the time series difference are abnormal points; If the result is no, no processing is performed.

Citation Information

Cited By

  • Multi-dimensional time series data anomaly discrimination and restoration method based on convolutional neural network

    CN119961847A