Multivariate time series anomaly detection method and system based on multi-subgraph fusion

Through a multi-subgraph fusion method, combined with graph neural network and Transformer encoder, dynamic changes and noise problems in multivariate time series anomaly detection are solved, and more efficient and accurate anomaly detection is achieved.

CN118691931BActive Publication Date: 2024-11-22EAST CHINA JIAOTONG UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411157169.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-22
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

Existing multivariate time series anomaly detection methods are difficult to effectively capture dynamic changes and local patterns in time series, and traditional graph structure learning is costly to process dynamically changing multivariate time series and cannot completely eliminate noise, resulting in low detection accuracy.

Method used

Using a method based on multi-subgraph fusion, the molecular sequences are divided by sliding window strategy, feature extraction, graph structure construction and graph neural network weighted fusion, combined with the Transformer encoder and the Kolmogolov-Arnold network, to achieve Adaptive anomaly detection.

Benefits of technology

It improves the accuracy and efficiency of anomaly detection, can capture the complex interaction and dynamic characteristics of multiple time series more accurately, reduces computational overhead and enhances the adaptability and generalization capabilities of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118691931B_ABST
    Figure CN118691931B_ABST
Patent Text Reader

Abstract

The present invention discloses a multivariate time series anomaly detection method and system based on multi-subgraph fusion, which uses a sliding window strategy to divide the denoised multivariate time series into multiple subsequences, inputs the multiple subsequences into a feature extraction module for feature extraction, and splices the features extracted by the feature extraction module with the denoised multivariate time series to obtain a fused feature matrix; for the fused feature matrix, two subgraph construction methods are used to obtain multiple subgraphs respectively; a graph neural network and a Transformer encoder are used to weighted fuse the obtained subgraphs to obtain a fused graph structure and feature representation; the fused graph structure and feature representation are input into an anomaly detection model to perform time series anomaly detection. The present invention can well capture the dynamic changes and local patterns in the time series, well fuse multiple subgraph structures, and improve the ability of anomaly detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of data anomaly detection, and in particular relates to a multivariate time series anomaly detection method and system based on multi-subgraph fusion. Background Art

[0002] With the rapid development of the Internet of Things and sensor technology, various real-time systems and devices (such as industrial control systems, smart grids, transportation networks, and environmental monitoring systems) are generating a large amount of multivariate time series data every day. These data usually contain readings from multiple sensors, which can provide important information about the operating status of the system. However, the large amount of data and the complexity of the data bring great challenges to anomaly detection. Anomaly detection refers to the identification of points or sequences from data that do not conform to the expected pattern, which is crucial to ensure system security, reliability, and efficiency. Due to the high dimensionality, time dependence, and spatial dependence of multivariate time series, these characteristics make multivariate time series anomaly detection very complex. Traditional univariate time series anomaly detection methods cannot effectively capture the interactive relationship between variables, resulting in low detection accuracy. In addition, current anomaly detection methods cannot make good use of the complex interactive relationship between sensors, but in many applications, the readings between different sensors are not independent, and they may affect each other through physical location or functional connections. This has prompted researchers to explore anomaly detection methods based on graph structure learning.

[0003] Graph neural networks (GNNs) can effectively learn the representation of graph structure data and update the features of nodes, thereby performing tasks such as node classification, edge prediction, and graph classification. This method has been successfully applied to recommendation systems, predicting traffic flow and congestion, etc. However, graph neural networks usually use the initialized graph structure as input. Obviously, this way of initializing the graph structure is not applicable to dynamically changing multivariate time series, and it is impossible to learn the complex interactive relationships between multiple variables.

[0004] Therefore, some researchers have applied graph structure learning to multivariate time series anomaly detection to fully consider the time dependency and spatial dependency of multivariate time series and effectively capture the interaction between multivariate variables. For example, a multivariate time series anomaly detection model is proposed in the invention patent document with patent publication number CN117454266A. A robust multivariate time series anomaly detection model in the invention patent document with patent publication number CN 117421666 A. However, learning the prior graph structure in the graph structure learning stage to eliminate the influence of noise in the time series will not only introduce a large computational cost, but also fail to completely eliminate the influence of noise and may ignore global information. In addition, for time series with time dependency and dynamic changes, it is impossible to adaptively and flexibly model, and the time dependency and spatial dependency of the time series cannot be well captured, which will reduce the accuracy of time series anomaly detection. Summary of the invention

[0005] The purpose of the present invention is to provide a multivariate time series anomaly detection method and system based on multi-subgraph fusion, which can not only capture the dynamic changes and local patterns in the time series well, but also realize adaptive and flexible modeling capabilities, and integrate multiple subgraph structures well to improve the ability of anomaly detection.

[0006] The present invention is achieved through the following technical solutions.

[0007] The multivariate time series anomaly detection method based on multi-subgraph fusion has the following steps:

[0008] Step S1: denoising the multivariate time series;

[0009] Step S2: Using a sliding window strategy, the denoised multivariate time series is divided into multiple subsequences, and the multiple subsequences are input into a feature extraction module for feature extraction, and the features extracted by the feature extraction module are spliced ​​with the denoised multivariate time series to obtain a fusion feature matrix;

[0010] Step S3: For the fusion feature matrix, each sensor is a node, and the edge represents the dependency relationship between sensors at the current time point. The first method of constructing a subgraph is: calculating the cosine similarity of the feature vector of each node, dynamically selecting the largest K edge weight values, and retaining the corresponding edges to obtain multiple subgraphs; the second method of constructing a subgraph is: using the weighted sum of KL divergence and bulldozer (Wasserstein) distance to measure the correlation between two nodes to determine whether there is an edge, and obtaining multiple subgraphs;

[0011] Step S4: Use graph neural network and Transformer encoder to perform weighted fusion on the obtained subgraphs to obtain the fused graph structure and feature representation;

[0012] Step S5: input the fused graph structure and feature representation into the anomaly detection model to perform time series anomaly detection;

[0013] Step S6: Calculate the anomaly score through the prediction error of the anomaly detection model, set a threshold, compare the anomaly score with the threshold, and determine whether the multivariate time series is abnormal to complete the anomaly detection task.

[0014] Further preferably, step S1 uses multivariate singular spectrum decomposition to perform denoising on the multivariate time series.

[0015] Further preferably, in step S2, the feature extraction module consists of three parts: a convolutional neural network (CNN), a temporal convolutional network (TCN) and a densely connected gated recurrent unit (DenseGRU); one branch of the feature extraction module connects the convolutional neural network (CNN) and the temporal convolutional network (TCN) in parallel, and fuses the outputs of the convolutional neural network (CNN) and the temporal convolutional network (TCN) as the output of this branch; another branch connects the convolutional neural network (CNN) and the densely connected gated recurrent unit (DenseGRU) in parallel, and fuses the outputs of the convolutional neural network (CNN) and the densely connected gated recurrent unit (DenseGRU) as the output of this branch; and the outputs of the two branches are weightedly fused as the final output of the feature extraction module.

[0016] Further preferably, the densely connected gated recurrent unit uses dense connections between multiple gated recurrent units, and the hidden state of the previous gated recurrent unit is not only input to the next gated recurrent unit, but also serves as the input of all subsequent gated recurrent units.

[0017] Further preferably, in step S3, the process of the first method for constructing a subgraph is as follows:

[0018] The relevant weight is calculated by fusing the cosine similarity of the vectors in the feature matrix to determine the weight of the edge between two nodes, expressed as:

[0019] , ;

[0020] represents the correlation value between the i-th node and the z-th node, represents the feature vector of the i-th node at time t, represents the feature vector of the zth node at time t, express The transpose of all Constructing the adjacency matrix ;

[0021] Symmetrically normalize the adjacency matrix ,in, represents the normalized adjacency matrix, Represents the degree matrix of the node; introduces a dynamic K value to retain the largest weight edge and calculate the degree of each node , Represents the normalized adjacency matrix between the ith node and the zth node; according to the degree of the ith node Dynamically adjust the number of edges K of the i-th node i ; Set a base edge number K0 and an adjustment coefficient , the adjustment formula is ;

[0022] For each node, select the K nodes with the largest weights among their connections to other nodes. i edges, forming a new adjacency matrix , the specific formula is as follows:

[0023] ;

[0024] in, Indicates the new adjacency matrix between the i-th node and the z-th node at time t The corresponding value in Represents the adjacency matrix The set of weight values ​​from the i-th node to the other s associated nodes.

[0025] Further preferably, in step S3, the second method of constructing the subgraph is:

[0026] KL divergence is used to measure the difference between node probability distributions. The formula is as follows:

[0027] ;

[0028] in, and Respectively represent the probabilities in the probability distributions P and Q, and KL represents the KL divergence;

[0029] The Wasserstein distance measures the minimum transmission cost between two probability distributions P and Q, and the formula is as follows:

[0030] k ( P,Q ) = <m> inf < / m> P ( u , ς ) ~c [ ‖ u − ς ‖ ] ;

[0031] in, represents the set of all possible joint distributions of P and Q; for each possible joint distribution , sampled from Get a pair of samples and , and calculate the distance between the two samples , calculate the joint distribution Under this condition, the expected value of the sample distance P u , ς ~c [ ‖ u − ς ‖ ] ; The lower bound of this expected value among all possible joint distributions is the bulldozer distance;

[0032] The relationship between two nodes is measured using the weighted sum of the two:

[0033] ;

[0034] in, represents the weighted sum of KL divergence and Wasserstein distance to measure the correlation between two nodes. is the weight parameter, represents the probability distribution of the i-th node; represents the probability distribution of the zth node, || represents the concatenation operation, and all Forming the adjacency matrix , by setting a threshold To retain the relevant edges and form a new adjacency matrix :

[0035] ;

[0036] in, The weighted sum of KL divergence and bulldozer distance is used to measure the distance between the i-th node and the z-th node in the new adjacency matrix at time t. The corresponding value in .

[0037] Further preferably, in step S4, a graph neural network is first used to extract the overall features of the subgraph of each time series; then the overall features of the subgraph of the time series extracted by the graph neural network are input into the Transformer encoder to encode the overall features of the subgraph of the time series and generate the weight of the subgraph.

[0038] Further preferably, in step S4, the Transformer encoder obtains the weight of the b-th subgraph at time t of the first subgraph construction method ; Transformer encoder obtains the weight of the b-th subgraph at time t of the second subgraph construction method ; and use softmax to ensure the normalization of weights; use the final generated weights to perform weighted fusion on multiple subgraphs to obtain the final graph structure ,in represents the graph structure after fusion, Respectively represent the b-th adjacency matrix at time t; use the generated weights to fuse the graph feature representation, in, represents the fused feature representation, They respectively represent the features of graph nodes extracted by graph neural network.

[0039] Further preferably, the anomaly detection model described in step S5 consists of three parts: a convolutional neural network (CNN), a Transformer encoder, and a Kolmogorov-Arnold network (KANs); first, the fused graph structure and feature representation are subjected to a CNN convolution operation to obtain local features of the time series, and then the local features are input into the Transformer encoder to capture the global features of the time series, and the final global features are input into the Kolmogorov-Arnold network for prediction.

[0040] Further preferably, in step S5, the process of inputting the final global features into the Kolmogorov-Arnold network (KANs) for prediction is:

[0041] Global Features Input into the Kolmogorov-Arnold network to get the predicted value , using a two-layer Kolmogorov-Arnold network, described as follows:

[0042] ∂= F 1 ( s ) = [ ∑ g = 1 one in f 1 , g ( s g ) , ∑ g = 1 one in f 2 , g ( s g ) , … , ∑ g = 1 one in f one out , g ( s g ) ] i ^ = F 2 ( ∂ ) = [ ∑ ι = 1 one out f ι , 1 ( ∂ ι ) , ∑ ι = 1 one out f ι , 2 ( ∂ ι ) , … , ∑ ι = 1 one out f ι , two out ( ∂ ι ) ] ;

[0043] in These are the operations of the first layer Kolmogorov-Arnold network and the second layer Kolmogorov-Arnold network, is the first global feature of the input elements, represents the hidden output after the first layer of Kolmogorov-Arnold network operation, is the output of the first layer of the Kolmogorov-Arnold network. elements, They represent the trainable spline functions in the first layer Kolmogorov-Arnold network operation and the second layer Kolmogorov-Arnold network operation respectively; They represent the dimension of the first-layer Kolmogorov-Arnold network input, the dimension of the first-layer Kolmogorov-Arnold network output, and the dimension of the second-layer Kolmogorov-Arnold network output, respectively.

[0044] The present invention also provides a system for implementing a multivariate time series anomaly detection method based on multi-subgraph fusion, comprising:

[0045] The data acquisition module is used to collect data from various sensors to form a multivariate time series;

[0046] Denoising module, used to denoise multivariate time series;

[0047] A feature extraction module is used to extract features from multiple subsequences divided from a multivariate time series;

[0048] The feature fusion module is used to combine the features extracted by the feature extraction module with the denoised multivariate time series to obtain a fused feature matrix;

[0049] A graph construction module, used to construct the fused feature matrix into multiple subgraphs;

[0050] The graph structure fusion module uses graph neural network and Transformer encoder to perform weighted fusion on the obtained subgraphs to obtain the fused graph structure and feature representation;

[0051] Anomaly detection module, used to input the fused graph structure and feature representation into the anomaly detection model to perform time series anomaly detection;

[0052] The diagnosis module calculates the anomaly score through the prediction error of the anomaly detection model, sets the threshold, compares the anomaly score with the threshold, and determines whether the multivariate time series is abnormal to complete the anomaly detection task.

[0053] The present invention has the following advantages:

[0054] (1) Introducing graph structure learning into the multivariate time series anomaly detection task, graph structure learning can provide more accurate, comprehensive and dynamically adaptive modeling capabilities for the multivariate time series anomaly detection task. The denoised time series data is input into the feature extraction module composed of CNN-TCN-DenseGRU for feature extraction, and then the graph structure is learned, thereby further improving the effect and application scope of anomaly detection. Adding a denoising module before graph structure learning reduces the additional computational overhead of generating the structured prior graph and eliminates the global and local structural information that is not considered in the denoising process of the structured prior graph.

[0055] (2) By training multiple subsequences, multiple subgraph structures are obtained, and then the graph neural network and Transformer encoder method are used to fuse multiple learned graph structures, so as to make full use of the advantages of different methods and make up for their respective limitations, and achieve more efficient and accurate anomaly detection results in time series anomaly detection tasks.

[0056] (3) In parallel graph structure learning, the output of the feature extraction module is concatenated with the denoised data to learn the adjacency matrix. The first method of constructing a subgraph is to normalize the learned adjacency matrix first, and then select and retain the K edges with the largest weights through a dynamic K value. The change of K value depends on the size of the node degree. The second method of constructing a subgraph introduces the KL divergence and the weighted sum of the bulldozer distance to measure the correlation between two nodes to determine whether there is an edge. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flow chart of the multivariate time series anomaly detection method based on multi-subgraph fusion of the present invention.

[0058] Figure 2 This is a structural diagram of the feature extraction module. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below with reference to the accompanying drawings.

[0060] like Figure 1 As shown in the figure, the multivariate time series anomaly detection method based on multi-subgraph fusion has the following steps:

[0061] Step S1: denoising the multivariate time series;

[0062] Step S2: Using a sliding window strategy, the denoised multivariate time series is divided into multiple subsequences, and the multiple subsequences are input into a feature extraction module for feature extraction, and the features extracted by the feature extraction module are spliced ​​with the denoised multivariate time series to obtain a fusion feature matrix;

[0063] Step S3: Two parallel methods are used to construct multiple subgraphs. For the fusion feature matrix, each sensor is a node, and the edge represents the dependency relationship between sensors at the current time point. The first method of constructing a subgraph is: calculate the cosine similarity of the feature vector of each node, dynamically select the largest K edge weight values, and retain the corresponding edges to obtain multiple subgraphs; the second method of constructing a subgraph is: use the weighted sum of KL divergence and bulldozer (Wasserstein) distance to measure the correlation between two nodes to determine whether there is an edge, and obtain multiple subgraphs;

[0064] Step S4: Use graph neural network and Transformer encoder to perform weighted fusion on the obtained subgraphs to obtain the fused graph structure and feature representation; assign different weights to the learned multiple subgraphs through graph neural network and Transformer encoder, and then perform weighted fusion on the subgraphs obtained by the two methods, which can better capture the complex relationship and dynamic characteristics of the graph structure, make the graph structure have better adaptability and generalization ability, and improve the accuracy of anomaly detection;

[0065] Step S5: input the fused graph structure and feature representation into the anomaly detection model to perform time series anomaly detection;

[0066] Step S6: Calculate the anomaly score through the prediction error of the anomaly detection model, set a threshold, compare the anomaly score with the threshold, and determine whether the multivariate time series is abnormal to complete the anomaly detection task.

[0067] The generation, collection and transmission of multivariate time series are inevitably affected by noise. In order to improve the accuracy of anomaly detection, multivariate time series need to be denoised. Multivariate Singular Spectrum Analysis is a powerful method for denoising and feature extraction of multivariate time series. In step S1 of the present invention, multivariate singular spectrum analysis is used to denoise the multivariate time series.

[0068] Constructing trajectory matrix: The training data comes from the time series data generated by M sensors in N time periods. M sensors can be regarded as M features. Therefore, multivariate time series ,in , Respectively represent the 1st, 2nd, ..., Mth sensor data at time t; by selecting the window length , convert the original time series into a trajectory matrix, which is a matrix containing several windows of length The matrix of subsequences of ; Construct the corresponding trajectory matrix for each time series, expressed as follows:

[0069] (1);

[0070] in, represents the trajectory matrix corresponding to the time series of the i-th sensor, , , represents the i-th sensor data at time L, represents the i-th sensor data at time L+1, express The sensor data at time i, express The sensor data at time i.

[0071] Concatenate the trajectory matrices of multiple time series into a trajectory matrix of a multivariate time series :

[0072] (2);

[0073] in, is the trajectory matrix corresponding to the time series of the first sensor, Represents the trajectory matrix corresponding to the time series of the second sensor, Represents the trajectory matrix corresponding to the time series of the Mth sensor;

[0074] Singular value decomposition: for the trajectory matrix Perform singular value decomposition ,in yes The eigenvector matrix of yes The transpose of is a diagonal matrix of singular values, yes The eigenvector matrix of yes The transpose of .

[0075] Select the main components: Select the first r main components according to the size of the singular value. Keep the singular values ​​and eigenvectors corresponding to these main components, and treat the unimportant components as noise, thereby removing the noise. The trajectory matrix after noise reduction is obtained: ,in and They are and The first r columns of yes The diagonal matrix consisting of the first r singular values ​​of .

[0076] Reconstructing time series: The trajectory matrix after noise reduction Reconstruct into time series, initialize and reconstruct multivariate time series and weight array ,in , all initial values ​​are set to 0:

[0077] X ˜ ( t ) = [ 0 , 0 , … , 0 ] T , (3);

[0078] The trajectory matrix after denoising Traverse each column of , add the data of each column to the corresponding position, and record the weight. Indicates Column data, where ; Add this column of data to the first arrive Positions:

[0079] (4);

[0080] in, is used to index the elements in the time series, , Represents the reconstructed time series at The value of the position, represents the jth column in the trajectory matrix arrive The submatrix of rows;

[0081] And update the corresponding weight array:

[0082] (5);

[0083] in, Represents the weight used for reconstruction.

[0084] Since there will be overlapping parts in the embedding process, the overlapping parts need to be averaged:

[0085] (6);

[0086] in represents the multivariate time series after denoising, , They respectively represent the 1st, 2nd, ..., Mth sensor data at time t after noise reduction.

[0087] In step S2, the feature extraction module consists of three parts: convolutional neural network (CNN), temporal convolutional network (TCN) and densely connected gated recurrent unit (DenseGRU). When processing time series, CNN can extract local temporal features through convolution kernels. TCN uses dilated causal convolution operations to extract local patterns and long-term dependencies in time series, which can effectively capture features of long time spans. GRU is a variant of RNN and is good at processing sequence data, especially in capturing dynamic changes in time series. GRU can remember and forget information better through the gating mechanism, thereby effectively modeling the temporal dependencies of time series. The problem of gradient vanishing can be alleviated by adding dense connections between multiple GRUs. At the same time, through dense connections, each layer of GRU can directly access the features of the previous layer, making feature reuse more efficient, thereby improving the performance of the model. Figure 2As shown in the figure, one branch of the feature extraction module connects the convolutional neural network (CNN) and the temporal convolutional network (TCN) in parallel (a series combination of a G-layer convolutional neural network and a temporal convolutional network), and fuses the outputs of the two as the output of this branch; the other branch connects the convolutional neural network (CNN) and the densely connected gated recurrent unit (DenseGRU) in parallel (a series combination of a G-layer convolutional neural network and a densely connected gated recurrent unit), and fuses the outputs of the two as the output of this branch; the outputs of the two branches are then weighted and fused as the final output of the feature extraction module. The features extracted by the feature extraction module are spliced ​​with the denoised multivariate time series to obtain a fused feature matrix, which can form a higher-dimensional feature representation. This diversified feature representation helps to better learn the complex relationships in the graph structure.

[0088] The convolutional neural network (CNN) extracts local features of the time series through convolution operations. The convolutional neural network (CNN) includes a one-dimensional convolution layer and a maximum pooling layer.

[0089] The core of the Temporal Convolutional Network (TCN) consists of multiple residual blocks, each of which contains two dilated causal convolutions and a residual connection. The dilated causal convolution operation is used to extract local patterns and long-term dependency patterns in time series. The dilated causal convolution inserts holes in the convolution kernel so that the convolution kernel can cover a longer time range. The residual connection adds the original input to the final output, and the features extracted by the temporal convolution network are , represents the input of the temporal convolutional network, Represents the result after dilated causal convolution, weight normalization layer, ReLU layer, and Dropout layer. In order to ensure residual connection, an additional 1×1 convolution is needed to ensure tensor matching when elements are added. Represents the activation function.

[0090] The Gated Recurrent Unit (GRU) can remember and forget information well through the gating mechanism, thereby effectively modeling the temporal dependencies of time series. The core part of the GRU consists of a reset gate and an update gate, where the reset gate controls how much historical information of the previous state is written to the current hidden state. The smaller the reset gate, the less information of the previous state is written; the update gate is used to control the degree to which the state information of the previous moment is brought into the current state. The larger the value of the update gate, the more state information of the previous moment is brought in. In this embodiment, dense connections are used between multiple gated recurrent units to optimize the GRU model. In addition to being input to the next gated recurrent unit, the hidden state of the previous gated recurrent unit will also be used as the input of all subsequent gated recurrent units. Assume that there is layer gated recurrent unit, then The input of the layer (starting from the second layer) is represented as ,in Respectively represent from the 1st layer to the The hidden state of the layer at time t, is the input of the densely connected gated recurrent unit.

[0091] The features extracted by the convolutional neural network (CNN) in a branch Features extracted by the temporal convolutional network (TCN) The weighted fusion is used as the output of this branch; the features extracted by the convolutional neural network (CNN) in the other branch are then Features extracted from the densely connected gated recurrent unit (DenseGRU) The weighted fusion is performed as the output of this branch; and the outputs of the two branches are finally weighted fused as the features extracted by the feature extraction module:

[0092] (7);

[0093] in, are three hyperparameters, which are used to control the weights of feature fusion.

[0094] In this embodiment, the features extracted by the feature extraction module are spliced ​​with the denoised multivariate time series to obtain a fusion feature matrix, which is expressed as: ,|| represents the splicing operation, Represents the fused feature matrix at time t.

[0095] In step S3 of this embodiment, the first method of constructing a subgraph is: calculating the cosine similarity of the feature vector of each node, dynamically selecting the largest K edge weight values, and retaining the corresponding edges to obtain multiple subgraphs. The process is as follows:

[0096] The relevant weight is calculated by fusing the cosine similarity of the vectors in the feature matrix to determine the weight of the edge between two nodes, expressed as:

[0097] , (8);

[0098] represents the correlation value between the i-th node and the z-th node, represents the feature vector of the i-th node at time t, represents the feature vector of the zth node at time t, express The transpose of all Constructing the adjacency matrix . In order to prevent learning the wrong topological structure, the learned adjacency matrix needs to be further refined. And convert it into a directed graph, retaining only the key edges. Select the largest K edge weights in the weighted adjacency matrix to retain the corresponding edges, but the importance and connection characteristics of different nodes in the graph may be different, and the fixed K value may not be able to adapt to these differences. Therefore, a dynamic K value is introduced for edge selection. The dynamic K value can select the most appropriate number of neighbors according to the characteristics of the node and the local structure changes, thereby improving the adaptability of the graph structure. Before selecting the key edges, the adjacency matrix is ​​first symmetric normalized. ,in, represents the normalized adjacency matrix, Represents the degree matrix of the node. The dynamic K value is introduced to retain the largest weight edge. The change of K value is related to the degree of the node. First, the degree of each node is calculated. , represents the normalized adjacency matrix between the ith node and the zth node. Dynamically adjust the number of edges K of the i-th node i First, set a base edge number K0 and an adjustment coefficient , so that for nodes with larger degrees, more edges will be retained, and the adjusted formula is .

[0099] For each node, select the K nodes with the largest weights among their connections to other nodes. i edges, forming a new adjacency matrix The specific formula is as follows:

[0100] (9);

[0101] in, Indicates the new adjacency matrix between the i-th node and the z-th node at time t The corresponding value in Represents the adjacency matrix The set of weight values ​​from the i-th node to the other s associated nodes.

[0102] The second method of constructing a subgraph is to use the weighted sum of KL divergence and Wasserstein distance to measure the correlation between two nodes to determine whether there is an edge. First, the feature vector of the node is converted into a probability distribution through the Softmax function, so that the KL divergence and Wasserstein distance can be used to measure the correlation between nodes. KL divergence is used to measure the difference between the probability distributions of nodes. The formula is as follows:

[0103] (10);

[0104] in, and They represent the probabilities in the probability distributions P and Q respectively, and KL represents the KL divergence.

[0105] The Wasserstein distance measures the minimum transmission cost between two probability distributions P and Q, and the formula is as follows:

[0106] k ( P,Q ) = <m> inf < / m> P ( u , ς ) ~c [ ‖ u − ς ‖ ] (11);

[0107] in, represents the set of all possible joint distributions of P and Q. For each possible joint distribution , from which we can sample Get a pair of samples and , and calculate the distance between the two samples , so we can calculate the joint distribution Under this condition, the expected value of the sample distance P u , ς ~c [ ‖ u − ς ‖ ] The lower bound on this expected value among all possible joint distributions is the bulldozer distance.

[0108] The relationship between two nodes is measured using the weighted sum of the two:

[0109] (12);

[0110] in, represents the weighted sum of KL divergence and Wasserstein distance to measure the correlation between two nodes. is a weight parameter that adjusts the contribution of the two metrics in the sum. represents the probability distribution of the i-th node; represents the probability distribution of the zth node, and || represents the concatenation operation. Forming the adjacency matrix , by setting a threshold To retain the relevant edges and form a new adjacency matrix :

[0111] (13);

[0112] in, The weighted sum of KL divergence and bulldozer distance is used to measure the distance between the i-th node and the z-th node in the new adjacency matrix at time t. The corresponding value in .

[0113] In step S4 of this embodiment, the obtained subgraphs are weightedly fused using a graph neural network and a Transformer encoder to obtain a fused graph structure and feature representation. The process is as follows:

[0114] First, a graph neural network is used to extract the overall features of each time series subgraph. The overall features of the time series subgraph extracted by the graph neural network are then input into the Transformer encoder to encode the overall features of the time series subgraph to generate the weight of the subgraph. Therefore, the weight of the b-th subgraph at time t of the first subgraph construction method is obtained. ; Get the weight of the b-th subgraph at time t of the second subgraph construction method ; and the normalization of weights is ensured by softmax.

[0115] Use the final generated weights to perform weighted fusion on multiple subgraphs to obtain the final graph structure ,in represents the graph structure after fusion, Respectively represent the b-th adjacency matrix at time t; use the generated weights to fuse the graph feature representation, in, represents the fused feature representation, They respectively represent the features of graph nodes extracted by graph neural network.

[0116] The anomaly detection model described in step S5 consists of three parts: convolutional neural network (CNN), Transformer encoder and Kolmogorov-Arnold network (KANs); first, the fused graph structure and feature representation are convolutionally represented by CNN to obtain the local features of the time series. CNN consists of a one-dimensional convolution layer and a maximum pooling layer. Then the local features are input into the Transformer encoder to capture the global features of the time series, and the final global features are input into KANs for prediction. KANs utilizes the Kolmogorov-Arnold representation theorem to better approximate any continuous function. Compared with traditional MLP, the spline function of KANs can capture complex nonlinear relationships more finely. This is particularly important for predicting time series with complex patterns and dynamic changes.

[0117] In step S5, the process of inputting the final global features into the Kolmogorov-Arnold network (KANs) for prediction is:

[0118] Global Features Input into the Kolmogorov-Arnold network to get the predicted value , using a two-layer Kolmogorov-Arnold network, described as follows:

[0119] ∂= F 1 ( s ) = [ ∑ g = 1 one in f 1 , g ( s g ) , ∑ g = 1 one in f 2 , g ( s g ) , … , ∑ g = 1 one in f one out , g ( s g ) ] i ^ = F 2 ( ∂ ) = [ ∑ ι = 1 one out f ι , 1 ( ∂ ι ) , ∑ ι = 1 one out f ι , 2 ( ∂ ι ) , … , ∑ ι = 1 one out f ι , two out ( ∂ ι ) ] (14);

[0120] in These are the operations of the first layer Kolmogorov-Arnold network and the second layer Kolmogorov-Arnold network, is the first global feature of the input elements, represents the hidden output after the first layer of Kolmogorov-Arnold network operation, is the output of the first layer of the Kolmogorov-Arnold network. elements, They represent the trainable spline functions in the first layer Kolmogorov-Arnold network operation and the second layer Kolmogorov-Arnold network operation respectively; They represent the dimension of the first-layer Kolmogorov-Arnold network input, the dimension of the first-layer Kolmogorov-Arnold network output, and the dimension of the second-layer Kolmogorov-Arnold network output, respectively.

[0121] This example uses mean square error as the loss function:

[0122] (15);

[0123] in, , represent the true value and the predicted value respectively. is the number of time series samples used for training.

[0124] The final inference score uses the residual between the true and predicted values ​​to calculate the anomaly score based on the residual and set the threshold for anomaly detection.

[0125] (16);

[0126] The threshold is automatically selected using the peak over-threshold method, and timestamps with inference scores greater than the threshold are considered anomalies.

[0127] In the current graph structure learning, most methods only model the overall time series, learn the weights between the embedding vectors of the source node and the target node, and then select the top K largest weights as neighbors in the learned adjacency matrix, that is, retain the relevant edges. This method is difficult to capture the local dependencies of the time series, and does not consider the dynamic changes of the time series, and it is difficult to model dynamically changing time series data. In addition, the current method does not make full use of the original feature information, which will reduce the expressive power of the model. Therefore, we divide the original time series into multiple subsequences using a sliding window strategy, and input the multiple subsequences into CNN (convolutional neural network)-TCN (temporal convolutional network)-DenseGRU for multi-scale feature extraction, where DenseGRU is a dense connection of multiple GRUs, and the output of the feature extraction module is spliced ​​with the time series data after denoising. Two graph structure learning methods are used, and a new weight metric is introduced to generate multiple subsequence graph structures. And the learned multiple subsequence graph structures are fused using the GNN and Transformer encoder combination structure proposed by the present invention. This graph fusion method has better adaptability and generalization ability to achieve adaptive and flexible modeling capabilities. Convolutional neural networks (CNNs) are used to capture local patterns and trends in multivariate time series, while Transformer encoders are used to capture global time dependencies and patterns. In addition, due to the convolution operation of CNN, it is less susceptible to gradient vanishing and gradient exploding than DCRNN, making the model training more stable. Transformer encoders have strong parallel computing capabilities, and CNN can quickly extract local features through convolution operations, so the combination of the two has high parallel efficiency, further improving the model calculation efficiency. In addition, the weight matrix of the self-attention mechanism in the Transformer encoder can provide a certain degree of interpretability, showing which time steps or which variables have a greater contribution to the current prediction, improving the interpretability of the model. Kolmogorov-Arnold networks (KANs) are used instead of traditional multi-layer perceptrons (MLPs) networks to generate the final prediction value. Compared with traditional multi-layer perceptrons (MLPs), the spline function of KANs can capture complex nonlinear relationships more finely. This is particularly important for time series prediction with complex patterns and dynamic changes.

[0128] Another embodiment of the present invention provides a system for implementing a multivariate time series anomaly detection method based on multi-subgraph fusion, comprising:

[0129] The data acquisition module is used to collect data from various sensors to form a multivariate time series;

[0130] Denoising module, used to denoise multivariate time series;

[0131] A feature extraction module is used to extract features from multiple subsequences divided from a multivariate time series;

[0132] The feature fusion module is used to combine the features extracted by the feature extraction module with the denoised multivariate time series to obtain a fused feature matrix;

[0133] A graph construction module, used to construct the fused feature matrix into multiple subgraphs;

[0134] The graph structure fusion module uses graph neural network and Transformer encoder to perform weighted fusion on the obtained subgraphs to obtain the fused graph structure and feature representation;

[0135] Anomaly detection module, used to input the fused graph structure and feature representation into the anomaly detection model to perform time series anomaly detection;

[0136] The diagnosis module calculates the anomaly score through the prediction error of the anomaly detection model, sets the threshold, compares the anomaly score with the threshold, and determines whether the multivariate time series is abnormal to complete the anomaly detection task.

[0137] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0138] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A multivariate time series anomaly detection method based on multi-subgraph fusion, characterized in that: Here are the steps: Step S1: denoising the multivariate time series; Step S2: Using a sliding window strategy, the denoised multivariate time series is divided into multiple subsequences, and the multiple subsequences are input into a feature extraction module for feature extraction, and the features extracted by the feature extraction module are spliced ​​with the denoised multivariate time series to obtain a fusion feature matrix; Step S3: For the fusion feature matrix, each sensor is a node, and the edge represents the dependency relationship between sensors at the current time point. The first method of constructing a subgraph is: calculating the cosine similarity of the feature vector of each node, dynamically selecting the largest K edge weight values, and retaining the corresponding edges to obtain multiple subgraphs; the second method of constructing a subgraph is: using the weighted sum of KL divergence and bulldozer distance to measure the correlation between two nodes to determine whether there is an edge, and obtaining multiple subgraphs; Step S4: Use graph neural network and Transformer encoder to perform weighted fusion on the obtained subgraphs to obtain the fused graph structure and feature representation; Step S5: input the fused graph structure and feature representation into the anomaly detection model to perform time series anomaly detection; Step S6: Calculate the anomaly score through the prediction error of the anomaly detection model, set a threshold, compare the anomaly score with the threshold, and determine whether the multivariate time series is abnormal to complete the anomaly detection task.

2. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 1 is characterized in that: Step S1 uses multivariate singular spectrum decomposition to denoise the multivariate time series.

3. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 1 is characterized in that: In step S2, the feature extraction module consists of three parts: a convolutional neural network, a temporal convolutional network and a densely connected gated recurrent unit; one branch of the feature extraction module connects the convolutional neural network and the temporal convolutional network in parallel, and fuses the outputs of the convolutional neural network and the temporal convolutional network as the output of this branch; another branch connects the convolutional neural network and the densely connected gated recurrent unit in parallel, and fuses the outputs of the convolutional neural network and the densely connected gated recurrent unit as the output of this branch; the outputs of the two branches are weightedly fused as the final output of the feature extraction module.

4. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 3 is characterized in that: The densely connected gated recurrent unit uses dense connections between multiple gated recurrent units, and the hidden state of the previous gated recurrent unit is input to the next gated recurrent unit and serves as the input of all subsequent gated recurrent units.

5. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 1 is characterized in that: In step S3, the process of the first method of constructing a subgraph is as follows: The relevant weight is calculated by fusing the cosine similarity of the vectors in the feature matrix to determine the weight of the edge between two nodes, expressed as: , ; represents the correlation value between the i-th node and the z-th node, represents the feature vector of the i-th node at time t, represents the feature vector of the zth node at time t, express The transpose of all Constructing the adjacency matrix ; Symmetrically normalize the adjacency matrix ,in, represents the normalized adjacency matrix, Represents the degree matrix of the node; introduces a dynamic K value to retain the largest weight edge and calculate the degree of each node , Represents the normalized adjacency matrix between the ith node and the zth node; according to the degree of the ith node Dynamically adjust the number of edges K of the i-th node i ; Set a base edge number K0 and an adjustment coefficient , the adjustment formula is ; For each node, select the K nodes with the largest weights among their connections to other nodes. i edges, forming a new adjacency matrix , the specific formula is as follows: ; in, Indicates the new adjacency matrix between the i-th node and the z-th node at time t The corresponding value in Represents the adjacency matrix The set of weight values ​​from the i-th node to the other s associated nodes.

6. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 1 is characterized in that: In step S3, the second method of constructing a subgraph is: KL divergence is used to measure the difference between node probability distributions. The formula is as follows: ; in, and Respectively represent the probabilities in the probability distributions P and Q, and KL represents the KL divergence; The bulldozer distance measures the minimum transmission cost between two probability distributions P and Q, and the formula is as follows: ; in, represents the set of all possible joint distributions of P and Q; for each possible joint distribution , sampled from Get a pair of samples and , and calculate the distance between the two samples , calculate the joint distribution Under this condition, the expected value of the sample distance ; The lower bound of this expected value among all possible joint distributions is the bulldozer distance; The relationship between two nodes is measured using the weighted sum of the two: ; in, represents the weighted sum of KL divergence and bulldozer distance to measure the correlation between two nodes, is the weight parameter, represents the probability distribution of the i-th node; P z represents the probability distribution of the zth node, || represents the concatenation operation, and all Forming the adjacency matrix , by setting a threshold To retain the relevant edges and form a new adjacency matrix : ; in, The weighted sum of KL divergence and bulldozer distance is used to measure the distance between the i-th node and the z-th node in the new adjacency matrix at time t. The corresponding value in .

7. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 1 is characterized in that: In step S4, the graph neural network is first used to extract the overall features of the subgraph of each time series; the overall features of the subgraph of the time series extracted by the graph neural network are then input into the Transformer encoder to encode the overall features of the subgraph of the time series to generate the weight of the subgraph; the Transformer encoder obtains the weight of the b-th subgraph at time t of the first subgraph construction method ; Transformer encoder obtains the weight of the b-th subgraph at time t of the second subgraph construction method ; And the normalization of weights is ensured through softmax; Use the final generated weights to perform weighted fusion on multiple subgraphs to obtain the final graph structure ,in represents the graph structure after fusion, Respectively represent the b-th adjacency matrix at time t; use the generated weights to fuse the graph feature representation, in, represents the fused feature representation, They respectively represent the features of graph nodes extracted by the graph neural network.

8. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 1 is characterized in that: The anomaly detection model described in step S5 consists of three parts: a convolutional neural network, a Transformer encoder, and a Kolmogorov-Arnold network; first, the fused graph structure and feature representation are convolved through a convolutional neural network to obtain local features of the time series, and then the local features are input into the Transformer encoder to capture the global features of the time series, and the final global features are input into the Kolmogorov-Arnold network for prediction.

9. The multivariate time series anomaly detection method based on multi-subgraph fusion according to claim 8 is characterized in that: In step S5, the process of inputting the final global features into the Kolmogorov-Arnold network for prediction is: Global Features Input into the Kolmogorov-Arnold network to get the predicted value , using a two-layer Kolmogorov-Arnold network, described as follows: ; in These are the operations of the first layer Kolmogorov-Arnold network and the second layer Kolmogorov-Arnold network, is the first global feature of the input elements, represents the hidden output after the first layer of Kolmogorov-Arnold network operation, is the output of the first layer of the Kolmogorov-Arnold network. elements, denote the trainable spline functions in the first layer Kolmogorov-Arnold network operation and the second layer Kolmogorov-Arnold network operation respectively; They represent the dimension of the first-layer Kolmogorov-Arnold network input, the dimension of the first-layer Kolmogorov-Arnold network output, and the dimension of the second-layer Kolmogorov-Arnold network output, respectively.

10. A system for implementing the multivariate time series anomaly detection method based on multi-subgraph fusion according to any one of claims 1 to 9, characterized in that: include: The data acquisition module is used to collect data from various sensors to form a multivariate time series; Denoising module, used to denoise multivariate time series; A feature extraction module is used to extract features from multiple subsequences divided from a multivariate time series; The feature fusion module is used to combine the features extracted by the feature extraction module with the denoised multivariate time series to obtain a fused feature matrix; A graph construction module, used to construct the fused feature matrix into multiple subgraphs; The graph structure fusion module uses graph neural network and Transformer encoder to perform weighted fusion on the obtained subgraphs to obtain the fused graph structure and feature representation; Anomaly detection module, used to input the fused graph structure and feature representation into the anomaly detection model to perform time series anomaly detection; The diagnosis module calculates the anomaly score through the prediction error of the anomaly detection model, sets the threshold, compares the anomaly score with the threshold, and determines whether the multivariate time series is abnormal to complete the anomaly detection task.