Multivariable time series data anomaly detection method based on space-time diagram structure learning

Abstract

The invention discloses a multivariable time series data anomaly detection method based on space-time diagram structure learning, and relates to the technical field of machine learning. Comprising the steps of obtaining a multivariable time sequence anomaly detection data set; preprocessing the obtained multivariable time sequence anomaly detection data set; based on the multivariable time sequence coupling network, constructing an ST-GSLN multivariable time sequence anomaly detection model; carrying out the training of the ST-GSLN multivariable time sequence anomaly detection model, and obtaining a trained ST-GSLN multivariable time sequence anomaly detection model; and judging whether the multivariable time sequence data is abnormal or not through the trained ST-GSLN multivariable time sequence anomaly detection model, and giving a detection result. According to the method, the space-time depth features and the structural features of the multivariable time series coupling network are fused, the abnormal phenomenon caused by time delay causality can be effectively overcome, and the robustness and adaptability of the multivariable time series data anomaly detection model are further improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning. Background Art

[0002] Information systems usually use physical devices or virtual devices (for example, temperature sensors, barcodes, buried points, etc.) to collect data from different components or functional modules of the information system, which are usually presented in multidimensional data (or multivariate data, multivariate time series data) to reflect the operating status of the information system. Efficient multivariate time series data anomaly detection is of great significance for early warning of anomalies in information systems and ensuring stable operation. In recent years, multivariate time series data anomaly detection has been widely used in industrial manufacturing, financial transactions, network security and other fields. By identifying and marking abnormal information in various application scenarios, it plays an important role in promoting the development of intelligent operation and maintenance in various fields. However, with the continuous improvement of Internet hardware facilities and the deepening of system applications, the scale of multivariate time series data continues to expand, making multivariate time series data anomaly detection increasingly challenging. In addition, the lack of sensitivity of data sampling technology in application scenarios, the poor timeliness of system state change response speed, and the difficulty in estimating the strength of sampling data time series dependence have led to a high missed detection rate. In the context of large-scale multivariate time series data and complex coupling relationships, it is a crucial basic work to carry out the description of the coupling relationship between multivariate time series, the research on multivariate time series data time dependency misalignment and multivariate time series data anomaly detection under the lag causal effect, and improve the level of intelligent operation and maintenance.

[0003] Due to the limitations of information systems in identifying and processing initial subtle anomalies, the system continues to operate in an abnormal state, which in turn causes more serious anomalies, and the anomaly problem caused by the time-lag causal effect of multivariate time series data. In response to this problem, considering the strong dependence characteristics of time-lag causality in the time series dimension and the structural differences of micro-anomalies, a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning is proposed to solve the difficulties existing in the existing technology, which is a problem that technicians in this field urgently need to solve. Summary of the invention

[0004] In view of this, the present invention provides a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning, which integrates the spatiotemporal depth characteristics and structural characteristics of the multivariate time series coupling network, can effectively overcome the anomaly phenomena caused by time lag causality, and further improve the robustness and adaptability of the multivariate time series data anomaly detection model.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A multivariate time series data anomaly detection method based on spatiotemporal graph structure learning includes the following steps:

[0007] S1. Obtain data: Obtain a multivariate time series anomaly detection dataset;

[0008] S2. Data preprocessing: preprocess the acquired multivariate time series anomaly detection dataset;

[0009] S3. Model construction: Based on the multivariate time series coupling network, the ST-GSLN multivariate time series anomaly detection model is constructed;

[0010] S4. Model training: The ST-GSLN multivariate time series anomaly detection model is trained using the preprocessed multivariate time series anomaly detection dataset to obtain a trained ST-GSLN multivariate time series anomaly detection model;

[0011] S5. Anomaly detection: The trained ST-GSLN multivariate time series anomaly detection model is used to determine whether the multivariate time series data is abnormal and give the detection results.

[0012] Optionally, the anomaly detection datasets in S1 include server monitoring datasets SMD and ASD, SMAP and MSL of the Mars exploration mission, and monitoring data SWaT and WADI of the water treatment plant.

[0013] Optionally, the ST-GSLN multivariate time series anomaly detection model constructed in S4 includes a multivariate time series data feature embedding module, a graph structure learning module, a feature extraction module, and a reconstruction module connected in sequence;

[0014] The multivariate time series data feature embedding module samples the data within a specified time period through a sliding window sampling strategy as the embedding features of the graph structure learning module of the multivariate time series coupling network;

[0015] The graph structure learning module constructs and learns the topological structure of the multivariate time series coupling network by embedding features of multivariate time series data through similarity matrix construction and heuristic graph structure learning algorithms;

[0016] The feature extraction module includes a spatiotemporal depth extraction unit and a graph structure feature extraction unit. The spatiotemporal depth extraction unit adaptively captures the correlation between different timestamps to obtain spatiotemporal depth features by combining convolution with LSTM. The graph structure feature extraction unit extracts the features of the spatial structure of the multivariate temporal coupling network based on the multi-channel graph encoding layer to obtain graph structure features.

[0017] Reconstruction module,The reconstruction module based on VAE consists of an encoder and a decoder. The features after the spatiotemporal depth features and graph structure features are cascaded as the input of the encoder, and the cascade features are mapped to distribution parameters in the latent space. The decoder reconstructs the output data by sampling the latent space, and determines whether the multivariate time series data is abnormal through the reconstruction loss, and gives the detection results.

[0018] Optionally, the graph structure learning module represents the multivariate time series data as a graph structure G(V,E,X), where V is a node set, a node represents a variable, E is an edge set, an edge represents a coupling relationship between nodes, and X is a weight set, each weight represents the coupling strength on each edge;

[0019] The graph structure learning module includes the following contents:

[0020] The similarity matrix is ​​constructed by integrating similarity and distance metrics to enhance the representation ability of the coupling relationship between variables. The specific formula is as follows:

[0021]

[0022] S(X t )=SoftPlus(S(X t ))

[0023]

[0024] in, N is the number of multivariate variables, W is the sampling window length, W1 and W2 are weight matrices, S(X t ) is a nonlinear activation function, S ij For node v i and v j The correlation between ij For node v i and v j The distance between

[0025] The heuristic graph structure constructs a multivariable temporal coupling network topology. The construction process is as follows:

[0026] First, the initial discrete nodes do not have coupling relationships between variables, where the nodes are independent clusters. Where O is an all-zero matrix;

[0027] Secondly, according to the distance matrix, the two clusters G1 and G2 closest to cluster G0 are identified and marked in red, and the average dissimilarity of clusters G1 and G2 relative to cluster G0 is calculated. and

[0028] Finally, compare the distance between each node in G1 and G0 with the distance threshold If the distance between the two nodes is less than or equal to the threshold, the corresponding two nodes are connected, and the above steps are repeated until all variable nodes belong to the same cluster G0, thereby constructing a sparse and connected graph structure.

[0029] Optionally, the spatiotemporal depth extraction unit consists of an input layer, a convolutional long short-term memory unit, a multi-channel graph encoding layer, and a fully connected layer. The spatiotemporal depth extraction unit learns the multivariate temporal coupling network topology and the multivariate temporal data X t is the input, and the output is the current data X t The spatiotemporal depth features of the convolutional long short-term memory unit are as follows:

[0030] i t =σ(w xi *X t +w hi *h t-1 +w ci ⊙c t-1 +b i )

[0031] f t =σ(w xf *X t +w hf *h t-1 +w cf ⊙c t-1 +b f )

[0032] c t =f t ⊙c t-1 +i t ⊙τ(w xc *X t +w hc *h t-1 +b c )

[0033] o t =σ(w xo *X t +w ho *h t-1 +w co ⊙c t +b o )

[0034] h t =o t ⊙τ(c t )

[0035] in, N is the number of multivariate variables, W is the sampling window length, w xi ,w hi,w ci is the weight matrix, b i is the bias term, σ(·) is the Sigmoid activation function, and h t-1 is the output of the hidden layer at the previous moment, c t-1 is the memory unit output of the previous moment, ⊙ is the HadamardProduct calculation, * is the convolution operation, w xf ,w hf ,w cf is the weight matrix, b f is the bias term, f t is the output of the forget gate, w xc ,w hc is the weight matrix, i t is the output of the input gate, b c is the bias term, τ(·) is the Tanh activation function, c t-1 is the memory unit output at the previous moment, w xo ,w ho ,w co is the weight matrix, b o For paranoid items, o t is the result of the current output gate, c t is the memory unit output at the current moment;

[0036] Multi-channel image encoding layer, the specific formula is as follows:

[0037]

[0038] att i,j =LeaklyReLU(e T (W G x i,t ||W G x j,t ))

[0039]

[0040] Among them, || is a cascade operation, and is the k-th layer node v i and v j The characteristic variable, α is the attention function, H is the number of multi-head attention channels, e T and W G is a learnable parameter, x i,t is the node v at time t i The characteristic vector, x j,t is the node v at time t j The characteristic vector of node v i The number of neighbors;

[0041] The graph structure feature extraction unit takes the current input data X t The multivariate time series coupling network is used as input to extract the structural information of the coupling between multivariate time series. The structural features include degree centrality, closeness centrality, betweenness centrality, PageRank centrality and eigenvector centrality. The specific formula is as follows:

[0042]

[0043] Among them, CNM(G) is the representation process of structural features, It is the number of intersections of communication paths in a multivariate time series coupling network, the degree of information flow in the network, high-influence nodes, node interaction relationships, and node connection strength. It is used to quantify the structural characteristics of multivariate time series data. The specific definitions are as follows:

[0044] The mean degree centrality measures how many directly connected neighbor nodes a node has. The specific formula is as follows:

[0045]

[0046] Where N is the number of nodes, k i is the node degree, and nodes with high degree centrality have more multivariate temporal associations;

[0047] The mean betweenness centrality measures the frequency of a node appearing on all shortest paths in the network. The specific formula is as follows:

[0048]

[0049] in, is the number of shortest paths passing through node i, g st is the number of shortest paths connecting s and t. A high betweenness centrality of a node indicates that it plays an important role in controlling the flow of information or resource allocation in the network.

[0050] Close to the mean centrality, it measures the average distance from a node to all other nodes in the network. The specific formula is as follows:

[0051]

[0052] Among them, l i The average distance between node i and other nodes. Closeness centrality not only considers the degree of the node, but also the position of the node in the network, reflecting the global structural characteristics of the network;

[0053] The PageRank centrality mean measures the importance of a node and determines the importance of each variable by analyzing the connection relationship between multivariate time series. The specific formula is as follows:

[0054]

[0055] Where D(v) is the node’s degree, L(v) is the node’s out-degree, and B u is the in-degree of the node;

[0056] The mean eigenvector centrality takes into account the centrality of the node's neighbor nodes and the strength of the connection between the node and its neighbor nodes. The specific formula is as follows:

[0057]

[0058] Where N is the number of nodes, A is the graph adjacency matrix, λ is the largest eigenvalue of the adjacency matrix, and x j is the eigenvector corresponding to the eigenvalue of the adjacency matrix. The eigenvector centrality is used to characterize the position of the node in the network.

[0059] Optionally, the VAE-based reconstruction module consists of an encoder and a decoder. Based on probability distribution sampling, the features after the spatiotemporal depth features and the graph structure features are concatenated as the input of the encoder, and the cascaded features are mapped to distribution parameters in the latent space. The decoder reconstructs the output data by sampling the latent space and obtains the data distribution characteristics of the multivariate time series data under normal conditions. The specific formula is as follows:

[0060] argmin f,g {[δ(X f ,D q(φ) (E p(θ) (X f ; W e );W d ))]}

[0061] Among them, E p(θ) (X f ; W e ) is the encoder, X f is the data to be reconstructed input by the encoder, and is the feature extracted by the feature extraction module in the previous step. q(f) (E p(θ) ; W d ) is the decoder, E p(θ) is the encoding vector, W e and W d are the learnable parameters of the multilayer perceptron.

[0062] Optionally, the loss function consists of two parts: prediction loss and reconstruction loss. The specific formula is as follows:

[0063] Loss rec =KL(X f ,D(X f))

[0064] Loss for =||X t -X p ||2

[0065] Loss=Loss for +bLoss rec

[0066] Loss rec is the reconstruction loss, which is the difference between the input data reconstructed after VAE processing and the difference between the extraction modules using KL divergence to calculate the cascade features; Loss for is the prediction loss, which is the difference between the observed data and the model predicted data, calculated using the narrow Minkowski distance; Loss is the total loss function;

[0067] Among them, X f is the data to be reconstructed input by the encoder, D(X f ) is the reconstruction result, X p is the model output data, X t is the observed data and β is the smoothing factor.

[0068] Optionally, the ST-GSLN multivariate time series anomaly detection model trained in S5 determines whether the multivariate time series data is abnormal based on the anomaly score and gives the detection result;

[0069] The anomaly score uses the robust normalization method of the loss function. The specific formula is as follows:

[0070]

[0071] in, is the median of the model loss result Loss, It is the difference between the upper quartile and the lower quartile of the model loss result Loss.

[0072] It can be seen from the above technical solution that compared with the prior art, the present invention provides a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning, which has the following beneficial effects: the present invention integrates the spatiotemporal depth characteristics and structural characteristics of the multivariate time series coupling network, which can effectively overcome the abnormal phenomena caused by time lag causality, and further improve the robustness and adaptability of the multivariate time series data anomaly detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0074] Figure 1 A flow chart of a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning provided by the present invention;

[0075] Figure 2 A schematic diagram of the structure of the ST-GSLN multivariate time series anomaly detection model provided by the present invention;

[0076] Figure 3 A schematic diagram of the graph structure construction based on the heuristic multivariable temporal coupling network provided by the present invention;

[0077] Figure 4 A schematic diagram of the spatiotemporal depth feature extraction performed by the spatiotemporal depth extraction unit provided by the present invention;

[0078] Figure 5 A schematic diagram of a graph structure feature extraction unit provided by the present invention performing graph structure feature extraction;

[0079] Figure 6 A schematic diagram of the reconstruction module structure provided by the present invention;

[0080] Figure 7 This is a schematic diagram of comprehensive analysis of the comparative experiments provided by the present invention. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] Reference Figure 1 As shown, the present invention discloses a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning, comprising the following steps:

[0083] S1. Obtain data: Obtain a multivariate time series anomaly detection dataset;

[0084] S2. Data preprocessing: preprocess the acquired multivariate time series anomaly detection dataset;

[0085] S3. Model construction: Based on the multivariate time series coupling network, the ST-GSLN multivariate time series anomaly detection model is constructed;

[0086] S4. Model training: The ST-GSLN multivariate time series anomaly detection model is trained using the preprocessed multivariate time series anomaly detection dataset to obtain a trained ST-GSLN multivariate time series anomaly detection model;

[0087] S5. Anomaly detection: The trained ST-GSLN multivariate time series anomaly detection model is used to determine whether the multivariate time series data is abnormal and give the detection results.

[0088] Furthermore, the anomaly detection datasets in S1 include server monitoring datasets SMD and ASD, SMAP and MSL of the Mars exploration mission, and monitoring data SWaT and WADI of the water treatment plant.

[0089] Further, such as Figure 2 As shown, the ST-GSLN multivariate time series anomaly detection model constructed in S4 includes a multivariate time series data feature embedding module, a graph structure learning module, a feature extraction module, and a reconstruction module connected in sequence;

[0090] The multivariate time series data feature embedding module samples the data within a specified time period through a sliding window sampling strategy as the embedding features of the graph structure learning module of the multivariate time series coupling network;

[0091] The graph structure learning module constructs and learns the topological structure of the multivariate time series coupling network by embedding features of multivariate time series data through similarity matrix construction and heuristic graph structure learning algorithms;

[0092] The feature extraction module includes a spatiotemporal depth extraction unit and a graph structure feature extraction unit. The spatiotemporal depth extraction unit adaptively captures the correlation between different timestamps to obtain spatiotemporal depth features by combining convolution with LSTM. The graph structure feature extraction unit extracts the features of the spatial structure of the multivariate temporal coupling network based on the multi-channel graph encoding layer to obtain graph structure features.

[0093] Reconstruction module,The reconstruction module based on VAE consists of an encoder and a decoder. The features after the spatiotemporal depth features and graph structure features are cascaded as the input of the encoder, and the cascade features are mapped to distribution parameters in the latent space. The decoder reconstructs the output data by sampling the latent space, and determines whether the multivariate time series data is abnormal through the reconstruction loss, and gives the detection results.

[0094] Further, such as Figure 3As shown in the figure, the graph structure learning module represents multivariate time series data as a graph structure G(V,E,X), where V is a set of nodes, nodes represent variables, E is an edge set, edges represent the coupling relationship between nodes, and X is a set of weights, each weight represents the coupling strength on each edge.

[0095] The graph structure learning module includes the following contents:

[0096] The similarity matrix is ​​constructed by integrating similarity and distance metrics to enhance the representation ability of the coupling relationship between variables. The specific formula is as follows:

[0097]

[0098] S(X t )=SoftPlus(S(X t ))

[0099]

[0100] in, N is the number of multivariate variables, W is the sampling window length, W1 and W2 are weight matrices, S(X t ) is a nonlinear activation function, S ij For node v i and v j The correlation between ij For node v i and v j The distance between

[0101] The heuristic graph structure constructs a multivariable temporal coupling network topology. The construction process is as follows:

[0102] First, the initial discrete nodes do not have coupling relationships between variables, where the nodes are independent clusters. Where O is an all-zero matrix;

[0103] Secondly, according to the distance matrix, the two clusters G1 and G2 closest to cluster G0 are identified and marked in red, and the average dissimilarity of clusters G1 and G2 relative to cluster G0 is calculated. and

[0104] Finally, compare the distance between each node in G1 and G0 with the distance threshold If the distance between the two nodes is less than or equal to the threshold, the corresponding two nodes are connected, and the above steps are repeated until all variable nodes belong to the same cluster G0, thereby constructing a sparse and connected graph structure.

[0105] Further, such as Figure 4 and Figure 5As shown in Figure 2, the spatiotemporal depth extraction unit consists of an input layer, a convolutional long short-term memory unit, a multi-channel graph encoding layer, and a fully connected layer. The spatiotemporal depth extraction unit learns the multivariate temporal coupling network topology and the multivariate temporal data X. t is the input, and the output is the current data X t The spatiotemporal depth features of the convolutional long short-term memory unit are as follows:

[0106] i t =σ(w xi *X t +w hi *h t-1 +w ci ⊙c t-1 +b i )

[0107] f t =σ(w xf *X t +w hf *h t-1 +w cf ⊙c t-1 +b f )

[0108] c t =f t ⊙c t-1 +i t ⊙τ(w xc *X t +w hc *h t-1 +b c )

[0109] o t =σ(w xo *X t +w ho *h t-1 +w co ⊙c t +b o )

[0110] h t =o t ⊙τ(c t )

[0111] in, N is the number of multivariate variables, W is the sampling window length, w xi ,w hi ,w ci is the weight matrix, b i is the bias term, σ(·) is the Sigmoid activation function, and h t-1 is the output of the hidden layer at the previous moment, c t-1is the memory unit output of the previous moment, ⊙ is the HadamardProduct calculation, * is the convolution operation, w xf ,w hf ,w cf is the weight matrix, b f is the bias term, f t is the output of the forget gate, w xc ,w hc is the weight matrix, i t is the output of the input gate, b c is the bias term, τ(·) is the Tanh activation function, c t-1 is the memory unit output at the previous moment, w xo ,w ho ,w co is the weight matrix, b o For paranoid items, o t is the result of the current output gate, c t is the memory unit output at the current moment;

[0112] Multi-channel image encoding layer, the specific formula is as follows:

[0113]

[0114] att i,j =LeaklyReLU(e T (W G x i,t ||W G x j,t ))

[0115]

[0116] Among them, || is a cascade operation, and is the k-th layer node v i and v j The characteristic variable, α is the attention function, H is the number of multi-head attention channels, e T and W G is a learnable parameter, x i,t is the node v at time t i The characteristic vector, x j,t is the node v at time t j The characteristic vector of node v i The number of neighbors;

[0117] The graph structure feature extraction unit takes the current input data X tThe multivariate time series coupling network is used as input to extract the structural information of the coupling between multivariate time series. The structural features include degree centrality, closeness centrality, betweenness centrality, PageRank centrality and eigenvector centrality. The specific formula is as follows:

[0118]

[0119] Among them, CNM(G) is the representation process of structural features, It is the number of intersections of communication paths in a multivariate time series coupling network, the degree of information flow in the network, high-influence nodes, node interaction relationships, and node connection strength. It is used to quantify the structural characteristics of multivariate time series data. The specific definitions are as follows:

[0120] The mean degree centrality measures how many directly connected neighbor nodes a node has. The specific formula is as follows:

[0121]

[0122] Where N is the number of nodes, k i is the node degree, and nodes with high degree centrality have more multivariate temporal associations;

[0123] The mean betweenness centrality measures the frequency of a node appearing on all shortest paths in the network. The specific formula is as follows:

[0124]

[0125] in, is the number of shortest paths passing through node i, g st is the number of shortest paths connecting s and t. A high betweenness centrality of a node indicates that it plays an important role in controlling the flow of information or resource allocation in the network.

[0126] Close to the mean centrality, it measures the average distance from a node to all other nodes in the network. The specific formula is as follows:

[0127]

[0128] Among them, l i The average distance between node i and other nodes. Closeness centrality not only considers the degree of the node, but also the position of the node in the network, reflecting the global structural characteristics of the network;

[0129] The PageRank centrality mean measures the importance of a node and determines the importance of each variable by analyzing the connection relationship between multivariate time series. The specific formula is as follows:

[0130]

[0131] Where D(v) is the node’s degree, L(v) is the node’s out-degree, and B u is the in-degree of the node;

[0132] The mean eigenvector centrality takes into account the centrality of the node's neighbor nodes and the strength of the connection between the node and its neighbor nodes. The specific formula is as follows:

[0133]

[0134] Where N is the number of nodes, A is the graph adjacency matrix, λ is the largest eigenvalue of the adjacency matrix, and x j is the eigenvector corresponding to the eigenvalue of the adjacency matrix. The eigenvector centrality is used to characterize the position of the node in the network.

[0135] Further, such as Figure 6 As shown in the figure, the VAE-based reconstruction module consists of an encoder and a decoder. Based on probability distribution sampling, the features of the spatiotemporal depth features and the graph structure features are concatenated as the input of the encoder, and the cascaded features are mapped to distribution parameters in the latent space. The decoder reconstructs the output data by sampling the latent space and obtains the data distribution characteristics of the multivariate time series data in the normal state. The specific formula is as follows:

[0136] argmin f,g {[δ(X f ,D q(φ) (E p(θ) (X f ; W e );W d ))]}

[0137] Among them, E p(θ) (X f ; W e ) is the encoder, X f is the data to be reconstructed input by the encoder, and is the feature extracted by the feature extraction module in the previous step. q(f) (E p(θ) ; W d ) is the decoder, E p(θ) is the encoding vector, W e and W d are the learnable parameters of the multilayer perceptron.

[0138] Furthermore, the loss function consists of two parts: prediction loss and reconstruction loss. The specific formula is as follows:

[0139] Loss rec =KL(X f ,D(X f ))

[0140] Loss for =||X t -X p ||2

[0141] Loss=Loss for +bLoss rec

[0142] Loss rec is the reconstruction loss, which is the difference between the input data reconstructed after VAE processing and the difference between the extraction modules using KL divergence to calculate the cascade features; Loss for is the prediction loss, which is the difference between the observed data and the model predicted data, calculated using the narrow Minkowski distance; Loss is the total loss function;

[0143] Among them, X f is the data to be reconstructed input by the encoder, D(X f ) is the reconstruction result, X p is the model output data, X t is the observed data and β is the smoothing factor.

[0144] Furthermore, the ST-GSLN multivariate time series anomaly detection model trained in S5 determines whether the multivariate time series data is abnormal based on the anomaly score and gives the detection result;

[0145] The anomaly score uses the robust normalization method of the loss function. The specific formula is as follows:

[0146]

[0147] in, is the median of the model loss result Loss, It is the difference between the upper quartile and the lower quartile of the model loss result Loss.

[0148] In a specific embodiment, in order to verify the performance of ST-GSLN in the anomaly detection task, a comparative experiment was conducted, and the relevant parameters of the ST-GSLN multivariate time series anomaly detection model were configured as follows:

[0149] Table 1 Basic parameters of ST-GSLN multivariate time series anomaly detection model

[0150] name parameter operating system Linux CPU Intel(R)Xeon(R)Platinum8352VCPU@2.10GHz GPU RTX4090(24GB) Model Optimizer AdamW Learning Rate 0.0001 Learning rate decay function StepLR CUDA 11.7 Pytorch 2.0.0 BatchSize 1 SampleWindow 5 Data normalization method <![CDATA[x * =x-x min / x max -x min ]]> b 0.01

[0151] Select the following comparison models for comparative experiments:

[0152] (1) Anomaly detection model based on graph structure (the present invention): GDN uses graph neural network to compare the differences between graph structures to detect anomalies, MTAD-GAT combines the characteristics of graph attention network and multivariate time series data to detect anomalies, GAT-GRU combines graph attention network and gated recurrent unit GRU to learn spatiotemporal features in graphs and detect anomalies, and GAE uses graph attention mechanism to learn graph representation for anomaly detection.

[0153] (2) Anomaly detection models based on time series prediction: Conv_LSTM uses convolution and LSTM network structure to extract features and detect anomalies in time series data; LSTM-NDT uses LSTM network for time series analysis and combines it with non-deterministic time series analysis for anomaly detection; GRU-AD uses GRU network to learn time series features and perform anomaly detection; OmniAnomaly combines autoencoder and graph neural network to process anomaly detection of multivariate time series data; MAD-GAN uses GAN for multivariate time series anomaly detection.

[0154] (3) Anomaly detection model based on attention mechanism: TranAD uses the Transformer attention mechanism to learn the complex dependencies of data and perform anomaly detection.

[0155] Table 2 shows the comparative experimental results of SMD and SMAP datasets, Table 3 shows the comparative experimental results of MSL and ASD datasets, and Table 4 shows the comparative experimental results of SWaT and WADI datasets. The evaluation indicators are Rec, Pre, AUC and F1. Black bold indicates the best result. If there are multiple identical values ​​for the evaluation indicator, it means that there is no optimal value for the indicator and no bold processing is performed. The specific analysis is as follows:

[0156] Table 2 Comparison experiment between SMD and SMAP

[0157]

[0158] As shown in Table 2, the AUC and F1 of the proposed model on the SMD and SMAP datasets are better than those of the comparison models, indicating that the introduction of spatiotemporal depth features and structural features can further improve the level of anomaly detection. The MAD-GAN and TranAD models have better AUC performance on the SMD dataset. This shows that the GAN-based reconstruction strategy and the Transformer-based attention strategy can improve the anomaly detection performance.

[0159] Table 3 Comparative experiments between MSL and ASD

[0160]

[0161] As shown in Table 3, the proposed model performs well in terms of AUC and F1 improvement on the MSL dataset, and has significant advantages over GSL-AD, indicating that the introduction of time series and causal features can significantly improve the accuracy of anomaly detection, and its performance is close to that of the OmniAnomaly model. GAT-GRU is a model for anomaly detection of time series data, but the recognition accuracy of anomalies caused by time-lag causal effects is limited.

[0162] Table 4 SWaT and WADI comparison experiment

[0163]

[0164] As shown in Table 4, GAT-GRU and TranAD both have the best and second best performance in AUC and F1 on the SWaT dataset, while the difference between the model of the present invention and GSL-AD is small. However, on the WADI dataset, the AUC and F1 of the model of the present invention are both the best, indicating that the model of the present invention has an improvement effect on the performance improvement in terms of high-dimensional data feature construction by introducing structural features and spatiotemporal depth features.

[0165] In order to evaluate the comprehensive performance of the model of the present invention, the average values ​​of the evaluation indicators of the model of the present invention and the comparative model on different experimental data sets are calculated, such as Figure 7 shown. Figure 7 The results show that ST-GSLN has the best comprehensive performance in AUC and F1 on different data sets, which is significantly improved compared with GSL-AD, and its generalization ability on different data sets has also been significantly improved, indicating that in the anomaly detection model, the structural characteristics of the multivariate temporal coupling network and the embedding of spatiotemporal depth features are helpful to improve the model representation ability and further enhance the robustness and generalization ability of the model.

[0166] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0167] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multivariate time series data anomaly detection method based on spatiotemporal graph structure learning, characterized in that: The following steps are involved: S1. Obtain data: Obtain a multivariate time series anomaly detection dataset; S2. Data preprocessing: preprocess the acquired multivariate time series anomaly detection dataset; S3. Model construction: Based on the multivariate time series coupling network, the ST-GSLN multivariate time series anomaly detection model is constructed; S4. Model training: The ST-GSLN multivariate time series anomaly detection model is trained using the preprocessed multivariate time series anomaly detection dataset to obtain a trained ST-GSLN multivariate time series anomaly detection model; S5. Anomaly detection: The trained ST-GSLN multivariate time series anomaly detection model is used to determine whether the multivariate time series data is abnormal and give the detection results.

2. According to the method of claim 1, the method is characterized in that: The anomaly detection datasets in S1 include server monitoring datasets SMD and ASD, SMAP and MSL of the Mars exploration mission, and monitoring data SWaT and WADI of the water treatment plant.

3. According to the method of claim 1, the method is characterized in that: The ST-GSLN multivariate time series anomaly detection model constructed in S4 includes a multivariate time series data feature embedding module, a graph structure learning module, a feature extraction module, and a reconstruction module connected in sequence; The multivariate time series data feature embedding module samples the data within a specified time period through a sliding window sampling strategy as the embedding features of the graph structure learning module of the multivariate time series coupling network; The graph structure learning module constructs and learns the topological structure of the multivariate time series coupling network by embedding features of multivariate time series data through similarity matrix construction and heuristic graph structure learning algorithms; The feature extraction module includes a spatiotemporal depth extraction unit and a graph structure feature extraction unit. The spatiotemporal depth extraction unit adaptively captures the correlation between different timestamps to obtain spatiotemporal depth features by combining convolution with LSTM. The graph structure feature extraction unit extracts the features of the spatial structure of the multivariate temporal coupling network based on the multi-channel graph encoding layer to obtain graph structure features. Reconstruction module,The reconstruction module based on VAE consists of an encoder and a decoder. The features after the spatiotemporal depth features and graph structure features are cascaded as the input of the encoder, and the cascade features are mapped to distribution parameters in the latent space. The decoder reconstructs the output data by sampling the latent space, and determines whether the multivariate time series data is abnormal through the reconstruction loss, and gives the detection results.

4. According to the method of multivariate time series data anomaly detection based on spatiotemporal graph structure learning according to claim 3, it is characterized in that: The graph structure learning module represents multivariate time series data as a graph structure G(V,E,X), where V is a node set, nodes represent variables, E is an edge set, edges represent the coupling relationship between nodes, and X is a weight set, each weight represents the coupling strength on each edge. The graph structure learning module includes the following contents: The similarity matrix is ​​constructed by integrating similarity and distance metrics to enhance the representation ability of the coupling relationship between variables. The specific formula is as follows: S(X t )=SoftPlus(S(X t )) in, N is the number of multivariate variables, W is the sampling window length, W1 and W2 are weight matrices, S(X t ) is a nonlinear activation function, S ij For node v i and v j The correlation between ij For node v i and v j The distance between The heuristic graph structure constructs a multivariable temporal coupling network topology. The construction process is as follows: First, the initial discrete nodes do not have coupling relationships between variables, where the nodes are independent clusters. Where O is an all-zero matrix; Secondly, according to the distance matrix, the two clusters G1 and G2 closest to cluster G0 are identified and marked in red, and the average dissimilarity of clusters G1 and G2 relative to cluster G0 is calculated. and Finally, compare the distance between each node in G1 and G0 with the distance threshold If the distance between the two nodes is less than or equal to the threshold, the corresponding two nodes are connected, and the above steps are repeated until all variable nodes belong to the same cluster G0, thereby constructing a sparse and connected graph structure.

5. According to claim 3, a multivariate time series data anomaly detection method based on spatiotemporal graph structure learning is characterized in that: The spatiotemporal depth extraction unit consists of an input layer, a convolutional long short-term memory unit, a multi-channel graph encoding layer, and a fully connected layer. The spatiotemporal depth extraction unit learns the multivariate temporal coupling network topology and multivariate temporal data X t is the input, and the output is the current data X t The spatiotemporal depth features of the convolutional long short-term memory unit are as follows: i t =σ(w xi *X t +w hi *h t-1 +w ci ⊙c t-1 +b i ) f t =σ(w xf *X t +w hf *h t-1 +w cf ⊙c t-1 +b f ) c t =f t ⊙c t-1 +i t ⊙τ(w xc *X t +w hc *h t-1 +b c ) o t =σ(w xo *X t +w ho *h t-1 +w co ⊙c t +b o ) h t =o t ⊙τ(c t ) in, N is the number of multivariate variables, W is the sampling window length, w xi ,w hi ,w ci is the weight matrix, b i is the bias term, σ(·) is the Sigmoid activation function, and h t-1 is the output of the hidden layer at the previous moment, c t-1 is the memory unit output of the previous moment, ⊙ is the HadamardProduct calculation, * is the convolution operation, w xf ,w hf ,w cf is the weight matrix, b f is the bias term, f t is the output of the forget gate, w xc ,w hc is the weight matrix, i t is the output of the input gate, b c is the bias term, τ(·) is the Tanh activation function, c t-1 is the memory unit output at the previous moment, w xo ,w ho ,w co is the weight matrix, b o For paranoid items, o t is the result of the current output gate, c t is the memory unit output at the current moment; Multi-channel image encoding layer, the specific formula is as follows: to i,j =LeaklyReLU(e T (W G x i,t ||W G x j,t )) Among them, || is a cascade operation, and is the k-th layer node v i and v j The characteristic variable, α is the attention function, H is the number of multi-head attention channels, e T and W G is a learnable parameter, x i,t is the node v at time t i The characteristic vector, x j,t is the node v at time t j The characteristic vector of node v i The number of neighbors; The graph structure feature extraction unit takes the current input data X t The multivariate time series coupling network is used as input to extract the structural information of the coupling between multivariate time series. The structural features include degree centrality, closeness centrality, betweenness centrality, PageRank centrality and eigenvector centrality. The specific formula is as follows: Among them, CNM(G) is the representation process of structural features, It is the number of intersections of communication paths in a multivariate time series coupling network, the degree of information flow in the network, high-influence nodes, node interaction relationships, and node connection strength. It is used to quantify the structural characteristics of multivariate time series data. The specific definitions are as follows: The mean degree centrality measures how many directly connected neighbor nodes a node has. The specific formula is as follows: Where N is the number of nodes, k i is the node degree, and nodes with high degree centrality have more multivariate temporal associations; The mean betweenness centrality measures the frequency of a node appearing on all shortest paths in the network. The specific formula is as follows: in, is the number of shortest paths passing through node i, g st is the number of shortest paths connecting s and t. A high betweenness centrality of a node indicates that it plays an important role in controlling the flow of information or resource allocation in the network. Close to the mean centrality, it measures the average distance from a node to all other nodes in the network. The specific formula is as follows: Among them, l i The average distance between node i and other nodes. Closeness centrality not only considers the degree of the node, but also the position of the node in the network, reflecting the global structural characteristics of the network; The PageRank centrality mean measures the importance of a node and determines the importance of each variable by analyzing the connection relationship between multivariate time series. The specific formula is as follows: Where D(v) is the node’s degree, L(v) is the node’s out-degree, and B u is the in-degree of the node; The mean eigenvector centrality takes into account the centrality of the node's neighbor nodes and the strength of the connection between the node and its neighbor nodes. The specific formula is as follows: Where N is the number of nodes, A is the graph adjacency matrix, λ is the largest eigenvalue of the adjacency matrix, and x j is the eigenvector corresponding to the eigenvalue of the adjacency matrix. The eigenvector centrality is used to characterize the position of the node in the network.

6. According to the method of claim 3, the method is characterized in that: The VAE-based reconstruction module consists of an encoder and a decoder. Based on probability distribution sampling, the features of the spatiotemporal depth features and the graph structure features are concatenated as the input of the encoder, and the cascaded features are mapped to distribution parameters in the latent space. The decoder reconstructs the output data by sampling the latent space and obtains the data distribution characteristics of multivariate time series data in a normal state. The specific formula is as follows: argmin f,g {[δ(X f ,D q(φ) (E p(θ) (X f ;IN e );IN d ))]} Among them, E p(θ) (X f ; W e ) is the encoder, X f is the data to be reconstructed input by the encoder, and is the feature extracted by the feature extraction module in the previous step. q(f) (E p(θ) ; W d ) is the decoder, E p(θ) is the encoding vector, W e and W d are the learnable parameters of the multilayer perceptron.

7. The method for detecting anomalies in multivariate time series data based on spatiotemporal graph structure learning according to claim 1, characterized in that: The loss function consists of two parts: prediction loss and reconstruction loss. The specific formula is as follows: Loss rec =KL(X f ,D(X f )) Loss for =||X t -X p ||2 Loss=Loss for +bLoss rec Loss rec is the reconstruction loss, which is the difference between the input data reconstructed after VAE processing and the difference between the extraction modules using KL divergence to calculate the cascade features; Loss for is the prediction loss, which is the difference between the observed data and the model predicted data, calculated using the narrow Minkowski distance; Loss is the total loss function; Among them, X f is the data to be reconstructed input by the encoder, D(X f ) is the reconstruction result, X p is the model output data, X t is the observed data and β is the smoothing factor.

8. The method for detecting anomalies in multivariate time series data based on spatiotemporal graph structure learning according to claim 1 or 7, characterized in that: The ST-GSLN multivariate time series anomaly detection model trained in S5 determines whether the multivariate time series data is abnormal based on the anomaly score and gives the detection result; The anomaly score uses the robust normalization method of the loss function. The specific formula is as follows: in, is the median of the model loss result Loss, It is the difference between the upper quartile and the lower quartile of the model loss result Loss.

Images