Multivariate time sequence anomaly detection method and system

Abstract

The invention provides a multivariate time sequence anomaly detection method and system, and relates to the technical field of data security. Generating a target multivariate time sequence by obtaining a multivariate time sequence and performing preprocessing; determining a sliding window sequence; extracting time features in the sliding window sequence by using a gating circulation unit, and extracting dimension features in the sliding window sequence by using a graph convolutional neural network; generating fusion features according to the time features and the dimension features; the channel attention module and the multi-head attention module are combined with the fusion features to realize fusion of the global features and the local features, and a fusion result is generated; reconstructing the target multivariate time sequence by using a preset training model to determine an abnormal score; and obtaining a preset threshold value and judging whether the multivariate time sequence is abnormal or not. And the detection precision of abnormal data is improved.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a multivariate time series anomaly detection method and system. Background Art

[0002] When time series data anomalies occur, traditional methods based on manual anomaly detection are highly dependent on the personal experience of the operation and maintenance personnel and the correctness of the troubleshooting direction. This anomaly detection method cannot meet the operation and maintenance needs of the industrial big data era. In order to break through the technical bottleneck of traditional anomaly detection, some more intelligent and automated tools are needed to assist. Therefore, time series anomaly detection based on artificial intelligence algorithms has been widely used in many fields such as network security, industrial production, and supply chain. By analyzing and processing time series data, the network environment or production process can be more finely monitored and controlled, effectively avoiding economic losses caused by network attacks or production failures.

[0003] At present, most of the anomaly detection methods are aimed at a single time series. In real life, the main form of time series is multivariate time series, and there are interactive relationships between multiple time series. In view of this, the anomaly detection method based on multivariate time series is more meaningful. The existing research on multivariate time series anomaly detection methods is still in its initial stage, and the systematic nature of the research work is not yet perfect. Time series anomaly detection faces many difficulties. For example, the imbalance between normal samples and abnormal samples will have a certain impact on the detection performance of the algorithm, the increase in the dimension of multivariate time series data will bring about the "dimensionality disaster", the complex correlation between data is easy to interfere with the real structure of the data and lead to erroneous analysis results, and some abnormal time series data are falsely reported or missed because they are similar to normal outputs.

[0004] Therefore, how to improve the detection accuracy of abnormal data has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to improve the detection accuracy of abnormal data, the present application provides a multivariate time series anomaly detection method and system.

[0006] In the first aspect, the present application provides a multivariate time series anomaly detection method using the following technical solution:

[0007] A multivariate time series anomaly detection method, comprising:

[0008] Acquire a multivariate time series and preprocess the multivariate time series to generate a target multivariate time series;

[0009] Determining a sliding window sequence according to the target multivariate time series in the form of a sliding window;

[0010] Using a gated recurrent unit to extract temporal features in the sliding window sequence, and using a graph convolutional neural network to extract dimensional features in the sliding window sequence;

[0011] Generate a fusion feature according to the time feature and the dimensional feature;

[0012] The channel attention module and the multi-head attention module are combined with the fusion features to achieve the fusion of the global features and local features of the multivariate time series and generate a fusion result;

[0013] Reconstructing the target multivariate time series using a preset training model to determine an anomaly score;

[0014] A preset threshold is obtained, and whether the multivariate time series is abnormal is determined according to the preset threshold and the abnormality score.

[0015] Optionally, the step of determining a sliding window sequence according to the target multivariate time series in the form of a sliding window includes:

[0016] Determine the data point x at timestamp t in the target multivariate time series t Dependencies;

[0017] Model the dependencies and set a sliding window of length k for cutting:

[0018]

[0019] Where W is the sliding window sequence, T is the target multivariate time series, and x′ t is the normalized time series, w n-k+1 is the time subsequence containing features cut into.

[0020] Optionally, the step of extracting time features from the sliding window sequence using a gated recurrent unit and extracting dimensional features from the sliding window sequence using a graph convolutional neural network includes:

[0021] A gated recurrent unit is used to extract temporal features in the sliding window sequence.

[0022]

[0023] Where x is the input vector consisting of multivariate time series data, F GRU is the GRU gated loop calculation function, W i is the parameter matrix of the fully connected layer, b i is the bias vector of the fully connected layer;

[0024] A graph convolutional neural network is used to extract dimensional features in the sliding window sequence, and randomly initialized embedding matrices I1, I2 are used to represent features F1, F2, F1, F2 and adjacency matrix E∈R M×M The calculation process is:

[0025] F1=tanh(I1q1),I1∈R M×k

[0026] F2=tanh(I2q2),I2∈R M×k

[0027] E=σ(tanh(F1F2 T -F2F1 T )),F1,F2∈R M×k

[0028] In the formula, N represents the number of graph nodes, i.e., the variable; e represents the embedding dimension of the graph nodes; q1 and q2 represent the fully connected network parameters; σ represents the Sigmoid function;

[0029] Perform graph convolution operation, the calculation process of the graph convolution layer is:

[0030]

[0031] Where G is the number of graph convolution layers; X g,i is the output of the g-th layer graph convolution; α is the hyperparameter of the graph convolution layer; E' is the normalized adjacency matrix;

[0032] The result of the graph convolution layer is calculated to determine the dimensional characteristics of the time series, and the calculation process is as follows:

[0033]

[0034] Optionally, the step of fusing the global features and local features of the multivariate time series and generating a fusion result by combining the fusion features with the channel attention module and the multi-head attention module includes:

[0035] The fusion feature F T,D ∈R T×C×W Mapping to feature F' T,D ∈R T'×C'×W' In the above example, the global average pooling layer AvgPool(·) is used to compress the fusion features to obtain the feature F (avg-c),i :

[0036] F (avg-c),i =AvgPool(F T,D );

[0037] The global maximum pooling layer MaxPool(·) is used to compress the fusion features to obtain F (max-c),i ∈R C :

[0038] F (max-c),i =MaxPool(F T,D );

[0039] F (avg-c),i and F (max-c),i ∈R C The sum is then taken and the interdependence between channels is extracted through convolution. The calculation process is:

[0040] F c,w =F (avg-c),i +F (max-c),i

[0041]

[0042] The characteristic values ​​of each channel are combined for weighted processing, and the calculation process is:

[0043] H=F ex (Z) = σ(W2δ(W1Z))

[0044] Where W1 and W2 represent weight matrices, δ represents Relu function, and σ represents Sigmoid function;

[0045] Apply the channel attention weights to the fused feature F T,D ∈R T×C×W The calculation process is:

[0046] S=F scale (H,X)=F T,D H.

[0047] Optionally, the step of fusing the global features and local features of the multivariate time series and generating a fusion result by combining the fusion features with the channel attention module and the multi-head attention module further includes:

[0048] The sequence S is processed using the position code PE, and the calculation process is:

[0049]

[0050] S i =Embedding(S)+PE(S)

[0051] Among them, pos represents the absolute position of the value at a certain moment in the long time series, and d represents the dimension;

[0052] The self-attention mechanism is used to filter out the key information in the time series and capture the dependency between feature vectors to highlight the key features. The calculation process is:

[0053] S i ·[W q ,W k ,W v ]=[Q,K,V]

[0054]

[0055] Where W q , W k , W v is a parameter matrix used to calculate Q, K, and V vectors; Q, K, and V correspond to the query vector, key vector, and value vector, respectively. k is the number of dimensions of K.

[0056] Optionally, after the step of implementing the global feature fusion of the target multivariate time series through the channel attention module and implementing the local feature fusion of the target multivariate time series through the multi-head attention module, the method further includes:

[0057] The probabilistic sparse attention calculation method is introduced. The Kullback-Leibler (KL) divergence is used to select u important queries to calculate self-attention. The sparsity calculation process of the i-th query is:

[0058]

[0059] Among them, q i , k i , v i represent the i-th row in Q, K, and V respectively, The first half of the term represents q i Log-Sum-Exp (LSE) on all keys, the second half of the term represents their arithmetic square roots;

[0060] For each q in Query i ∈R d and k in Key j ∈R d , there is the following relationship:

[0061]

[0062] Therefore, the maximum mean is used to calculate the sparsity of the i-th Query, and the calculation process is:

[0063]

[0064] Top-u is selected by prioritizing the above processes The size of u is determined by the number of samples, that is:

[0065]

[0066] Merge the average scores of Q and V. If q i If it belongs to Top-u, the score is calculated by A(Q,K,V); otherwise, the mean of V is used instead. The specific process is as follows:

[0067]

[0068] Where i(i∈{1,...,h}) is the number of the self-attention head, H i is the self-attention expression of the i-th self-attention, and h is the number of self-attention heads;

[0069] The multi-head attention mechanism allows the model to perform self-attention calculations in different subspaces, thereby capturing the correlation between different positions in the sequence. The calculation process is:

[0070]

[0071] Where MHA(I1,I1,I1) represents the multi-head attention operation of the input matrix I1, and is added to its own matrix to obtain the input of the Feed Forward Networks layer At the same time, the input matrix Using the convolution operation, that is

[0072] Optionally, the step of reconstructing the target multivariate time series using a preset training model to determine anomaly scores includes:

[0073] Adopting generative decoder Decoder1 and discriminative decoder Decoder2 to generate adversarial training module;

[0074] Obtain a preset training model, and use the adversarial training module to train the preset training model in an adversarial training manner;

[0075] The target multivariate time series is reconstructed using the trained preset training model to determine anomaly scores.

[0076] Optionally, the step of generating an adversarial training module by using a generating decoder Decoder1 and a discriminating decoder Decoder2 includes:

[0077] Generate reconstructed data that is similar to the input window W, where the generating decoder Decoder1 generates Y1 and the discriminating decoder Decoder2 generates Y2;

[0078] Calculate the reconstruction error The calculation method is:

[0079]

[0080] Where, ||·||2 represents the mean square error function;

[0081] According to the reconstruction error As the focus matrix Combined with the reconstructed sequence Y1, the output Y2 of the identification decoder Decoder2 is obtained through model reasoning to determine the adversarial error The calculation method is:

[0082]

[0083] The loss is calculated based on the reconstruction error and the adversarial error:

[0084]

[0085] Where n is the training epoch and ε is a training parameter close to 1.

[0086] Optionally, before the step of obtaining a preset threshold and judging whether the multivariate time series is abnormal according to the preset threshold and the abnormality score, the following steps are included:

[0087] The preset threshold is determined by using the POT automatic threshold selection method based on extreme value theory.

[0088] In a second aspect, the present application provides a multivariate time series anomaly detection system, comprising:

[0089] A preprocessing module, used for acquiring a multivariate time series and preprocessing the multivariate time series to generate a target multivariate time series;

[0090] A sliding window module, used to determine a sliding window sequence according to the target multivariate time series in the form of a sliding window;

[0091] A dimension extraction module, for extracting time features from the sliding window sequence using a gated recurrent unit, and for extracting dimensional features from the sliding window sequence using a graph convolutional neural network;

[0092] A fusion feature module, used to generate a fusion feature according to the time feature and the dimensional feature;

[0093] A fusion result module, used for fusing the global features and local features of the multivariate time series by combining the fusion features with the channel attention module and the multi-head attention module and generating a fusion result;

[0094] an anomaly score module, for reconstructing the target multivariate time series using a preset training model to determine an anomaly score;

[0095] The anomaly judgment module is used to obtain a preset threshold and judge whether the multivariate time series is abnormal according to the preset threshold and the anomaly score.

[0096] In summary, the present application includes the following beneficial technical effects:

[0097] This application generates a target multivariate time series by obtaining a multivariate time series and performing preprocessing; determines a sliding window sequence; uses a gated recurrent unit to extract time features in the sliding window sequence, and uses a graph convolutional neural network to extract dimensional features in the sliding window sequence; generates fusion features based on time features and dimensional features; combines fusion features through a channel attention module and a multi-head attention module to achieve fusion of global features and local features and generate fusion results; reconstructs the target multivariate time series using a preset training model to determine anomaly scores; obtains a preset threshold and determines whether the multivariate time series is abnormal. Improve the detection accuracy of abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 It is a flowchart of the first embodiment of the multivariate time series anomaly detection method of the present application;

[0099] Figure 2 This is a dual attention network structure diagram of the first embodiment of the multivariate time series anomaly detection method of the present application;

[0100] Figure 3 This is a diagram of the adversarial training process of the first embodiment of the multivariate time series anomaly detection method of the present application;

[0101] Figure 4 It is a schematic diagram of a multivariate time series anomaly detection method based on dual attention and feature fusion according to the first embodiment of the multivariate time series anomaly detection method of the present application;

[0102] Figure 5 It is a structural block diagram of the first embodiment of the multivariate time series anomaly detection system of the present application. DETAILED DESCRIPTION

[0103] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0104] The present application embodiment provides a multivariate time series anomaly detection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the multivariate time series anomaly detection method of the present application.

[0105] In this embodiment, the multivariate time series anomaly detection method includes the following steps:

[0106] Step S10: Acquire a multivariate time series and preprocess the multivariate time series to generate a target multivariate time series.

[0107] In a specific implementation, the method of obtaining a multivariate time series and preprocessing the multivariate time series to generate a target multivariate time series is to process each time step of the input time series T through linear normalization, adjust its value range to the range of [0,1], and obtain the normalized time series T'. The calculation formula of linear normalization is:

[0108]

[0109] Among them, T' represents the normalized time series, and its value range is [0,1]; x t Represents the value of the input time series T at time t; max(T) and min(T) represent the maximum and minimum values ​​in the input vector T respectively; ε' is a constant vector to prevent division by zero.

[0110] It can be understood that a time series (or dynamic series) refers to a series of values ​​of the same statistical indicator arranged in the order of their occurrence. The main purpose of time series analysis is to predict the future based on existing historical data.

[0111] Step S20: Determine a sliding window sequence in the form of a sliding window according to the target multivariate time series.

[0112] It should be noted that the step of determining the sliding window sequence according to the target multivariate time series in the form of a sliding window includes: determining the data point x at the timestamp t in the target multivariate time series t Dependencies; Model according to the dependencies and set a sliding window of length k for cutting:

[0113]

[0114] Where W is the sliding window sequence, T is the target multivariate time series, and x′ t is the normalized time series, w n-k+1 is the time subsequence containing features cut into.

[0115] Step S30: Use the gated recurrent unit to extract time features in the sliding window sequence, and use the graph convolutional neural network to extract dimensional features in the sliding window sequence.

[0116] It can be understood that the gated recurrent unit (GRU) is used to extract the temporal features of the preprocessed data, while the graph convolutional neural network (GCN) is used to extract the dimensional features of the data, and the temporal features and dimensional features are fused.

[0117] It should be noted that the steps of extracting time features in the sliding window sequence using a gated recurrent unit and extracting dimensional features in the sliding window sequence using a graph convolutional neural network include: extracting time features in the sliding window sequence using a gated recurrent unit

[0118]

[0119] Where x is the input vector consisting of multivariate time series data, F GRU is the GRU gated loop calculation function, W i is the parameter matrix of the fully connected layer, b i is the bias vector of the fully connected layer; a graph convolutional neural network is used to extract dimensional features in the sliding window sequence, and randomly initialized embedding matrices I1 and I2 are used to represent the features F1, F2, F1, F2 and the adjacency matrix E∈R M×M The calculation process is:

[0120] F1=tanh(I1q1),I1∈R M×k

[0121] F2=tanh(I2q2),I2∈R M×k

[0122] E=σ(tanh(F1F2 T -F2F1 T )),F1,F2∈R M×k

[0123] In the formula, N represents the number of graph nodes, i.e., the variable; e represents the embedding dimension of the graph nodes; q1 and q2 represent the fully connected network parameters; σ represents the Sigmoid function; the graph convolution operation is performed, and the calculation process of the graph convolution layer is:

[0124]

[0125] Where G is the number of graph convolution layers; X g,iis the output of the g-th layer of graph convolution; α is the hyperparameter of the graph convolution layer; E' is the normalized adjacency matrix; the result of the graph convolution layer is calculated to determine the dimensional characteristics of the time series, and the calculation process is:

[0126]

[0127] Step S40: Generate fusion features according to the time features and the dimensional features.

[0128] In the specific implementation, the features extracted by GRU and GCN are spliced ​​to obtain the fusion feature F of the multivariate time series. T,D , the calculation process is:

[0129]

[0130] Step S50: The channel attention module and the multi-head attention module are combined with the fusion features to achieve the fusion of the global features and local features of the multivariate time series and generate a fusion result.

[0131] It can be understood that the schematic diagram of the dual attention network in this embodiment is as follows Figure 2 shown.

[0132] It should be noted that the steps of combining the channel attention module and the multi-head attention module with the fusion feature to achieve the fusion of the global features and local features of the multivariate time series and generate the fusion result include: T,D ∈R T ×C×W Mapping to feature F' T,D ∈R T'×C'×W' In the above example, the global average pooling layer AvgPool(·) is used to compress the fusion features to obtain the feature F (avg-c),i :

[0133] F (avg-c),i =AvgPool(F T,D );

[0134] The global maximum pooling layer MaxPool(·) is used to compress the fusion features to obtain F (max-c),i ∈R C :

[0135] F (max-c),i =MaxPool(F T,D );

[0136] F (avg-c),i and F (max-c),i ∈R C The sum is then taken and the interdependence between channels is extracted through convolution. The calculation process is:

[0137] Fc,w =F (avg-c),i +F (max-c),i

[0138]

[0139] The characteristic values ​​of each channel are combined for weighted processing, and the calculation process is:

[0140] H=F ex (Z) = σ(W2δ(W1Z))

[0141] Where W1 and W2 represent weight matrices, δ represents Relu function, and σ represents Sigmoid function;

[0142] Apply channel attention weights to the fused features F T,D ∈R T×C×W The calculation process is:

[0143] S=F scale (H,X)=F T,D H.

[0144] It can be understood that the step of fusing the global features and local features of the multivariate time series and generating the fusion result by combining the channel attention module and the multi-head attention module with the fusion features also includes: processing the sequence S using the position encoding PE, and the calculation process is:

[0145]

[0146] S i =Embedding(S)+PE(S)

[0147] Among them, pos represents the absolute position of the value at a certain moment in the long time series, and d represents the dimension. The self-attention mechanism is used to filter out the key information in the time series and capture the dependency between feature vectors to highlight the key features. The calculation process is:

[0148] S i ·[W q ,W k ,W v ]=[Q,K,V]

[0149]

[0150] Where W q , W k , W v is a parameter matrix used to calculate Q, K, and V vectors; Q, K, and V correspond to the query vector, key vector, and value vector, respectively. k is the number of dimensions of K.

[0151] It can be understood that after the steps of realizing global feature fusion of the target multivariate time series through the channel attention module and realizing local feature fusion of the target multivariate time series through the multi-head attention module, it also includes: introducing a probabilistic sparse attention calculation method, selecting u important queries through Kullback-Leibler (KL) divergence to calculate self-attention, and the sparsity calculation process of the i-th Query is:

[0152]

[0153] Among them, q i , k i , v i represent the i-th row in Q, K, and V respectively, The first half of the term represents q i Log-Sum-Exp (LSE) on all keys, the second half of the term represents their arithmetic square roots;

[0154] For each q in Query i ∈R d and k in Key j ∈R d , there is the following relationship:

[0155]

[0156] Therefore, the maximum mean is used to calculate the sparsity of the i-th Query, and the calculation process is:

[0157]

[0158] Top-u is selected by prioritizing the above processes The size of u is determined by the number of samples, that is:

[0159]

[0160] Merge the average scores of Q and V. If q i If it belongs to Top-u, the score is calculated by A(Q,K,V); otherwise, the mean of V is used instead. The specific process is as follows:

[0161]

[0162] Where i(i∈{1,...,h}) is the number of the self-attention head, H i is the self-attention expression of the i-th self-attention, and h is the number of self-attention heads;

[0163] The multi-head attention mechanism allows the model to perform self-attention calculations in different subspaces, thereby capturing the correlation between different positions in the sequence. The calculation process is:

[0164] I1 1 =LN(I1+MHA(I1,I1,I1))

[0165] I1 2 =LN(I1 1 +FFN(Conv(I1 1 )))

[0166] Where MHA(I1,I1,I1) represents the multi-head attention operation of the input matrix I1, and is added to its own matrix to obtain the input I1 of the Feed Forward Networks layer 1 , and at the same time for the input matrix Using the convolution operation, namely Conv(I1 1 ).

[0167] In the specific implementation, position encoding is applied to the input window W to obtain I2, and the self-attention in the decoder is modified to mask the data of subsequent positions. The purpose is to prevent the decoder from viewing the data values ​​of future timestamps during training. This part uses the complete sequence As V and K, use the encoded input window as Q. The calculation process is:

[0168]

[0169] Finally, use Feed Forward Networks and Sigmoid compression function to Processing is performed to obtain the final output. The calculation process is:

[0170]

[0171] Where the Sigmoid activation function is used to generate outputs in the range [0,1] to match the normalized input window.

[0172] Step S60: reconstruct the target multivariate time series using a preset training model to determine anomaly scores.

[0173] It should be noted that if Figure 3 As shown, Figure 3The diagram of the adversarial training process is shown in FIG. The steps of reconstructing the target multivariate time series using the preset training model to determine the anomaly score include: generating an adversarial training module using a generation decoder Decoder1 and a discrimination decoder Decoder2; obtaining the preset training model, and training the preset training model using the adversarial training module; and reconstructing the target multivariate time series using the trained preset training model to determine the anomaly score.

[0174] It can be understood that the steps of generating the adversarial training module using the generator decoder Decoder1 and the discriminator decoder Decoder2 include: generating reconstructed data that is similar to the input window W, wherein the generator decoder Decoder1 generates Y1 and the discriminator decoder Decoder2 generates Y2; calculating the reconstruction error The calculation method is:

[0175]

[0176] In the formula, ||·‖2 represents the mean square error function; according to the reconstruction error As the focus matrix Combined with the reconstructed sequence Y1, the output Y2 of the discriminator decoder Decoder2 is obtained through model inference to determine the adversarial error The calculation method is:

[0177]

[0178] The loss is calculated based on the reconstruction error and the adversarial error:

[0179]

[0180] Where n is the training epoch and ε is a training parameter close to 1.

[0181] Use the trained model to test the set For abnormal judgment, the calculation method of the abnormal score is:

[0182]

[0183] Step S70: Obtain a preset threshold, and determine whether the multivariate time series is abnormal based on the preset threshold and the abnormality score.

[0184] It should be noted that before the step of obtaining the preset threshold and determining whether the multivariate time series is abnormal based on the preset threshold and the abnormality score, it includes: using the POT automatic threshold selection method based on extreme value theory to determine the preset threshold.

[0185] In the specific implementation, the POT automatic threshold selection method based on extreme value theory determines the abnormal threshold μ, thereby judging the abnormality of multivariate time series data. The calculation process is:

[0186]

[0187] Where μ is the initial threshold, which is the quantile set based on experience; ξ is the shape parameter in the generalized Pareto distribution; β is the scale parameter L = {l1,l2,...,l N}; L-μ represents the part above the threshold μ. The anomaly score is compared with the threshold, and the data exceeding the threshold is judged as abnormal data, otherwise it is normal data. The anomaly score is compared with the threshold, and the anomaly detection results of each dimension are defined as:

[0188] s i =1(f i ≥POT(f i ))

[0189] If any of the m dimensions of a multivariate time series is abnormal, the current timestamp is marked as abnormal, that is:

[0190]

[0191] like Figure 4 As shown, Figure 4 This is a schematic diagram of the dual-attention multivariate time series anomaly detection method based on feature fusion in this embodiment. The multivariate time series data is preprocessed to convert the time series data into an input format that conforms to the network structure; the normalized data is divided into a training set and a test set according to a certain ratio, and the time features of the preprocessed data are extracted by a gated recurrent unit (GRU). At the same time, a graph convolutional neural network (GCN) is used to extract the dimensional features of the data, and the time features and dimensional features are fused; the global and local features of the time series are fused through a channel attention module (SEAB) and a multi-head attention module (MHAB); an improved Transformer training model is used in an adversarial training manner, and then the multivariate time series T is reconstructed using the trained model, the reconstruction error is calculated, and the anomaly score is obtained; the POT automatic threshold selection method based on extreme value theory is used to determine the anomaly threshold, and data exceeding the threshold is judged as abnormal data, otherwise, it is normal data.

[0192] This embodiment generates a target multivariate time series by acquiring a multivariate time series and performing preprocessing; determines a sliding window sequence; uses a gated recurrent unit to extract time features in the sliding window sequence, and uses a graph convolutional neural network to extract dimension features in the sliding window sequence; generates fusion features based on time features and dimension features; combines fusion features through a channel attention module and a multi-head attention module to achieve fusion of global features and local features and generate fusion results; reconstructs the target multivariate time series using a preset training model to determine anomaly scores; obtains a preset threshold and determines whether the multivariate time series is abnormal. Improve the detection accuracy of abnormal data.

[0193] In addition, an embodiment of the present application further proposes a computer-readable storage medium, on which a program for low-light target detection is stored. When the program for low-light target detection is executed by a processor, the steps of the method for low-light target detection as described above are implemented.

[0194] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the multivariate time series anomaly detection system of the present application.

[0195] like Figure 5 As shown, the multivariate time series anomaly detection system proposed in the embodiment of the present application includes:

[0196] A preprocessing module 10, for acquiring a multivariate time series and preprocessing the multivariate time series to generate a target multivariate time series;

[0197] A sliding window module 20, used to determine a sliding window sequence according to a target multivariate time series in the form of a sliding window;

[0198] A dimension extraction module 30, for extracting temporal features in the sliding window sequence using a gated recurrent unit, and for extracting dimensional features in the sliding window sequence using a graph convolutional neural network;

[0199] A fusion feature module 40, used to generate fusion features according to time features and dimensional features;

[0200] A fusion result module 50 is used to fuse the global features and local features of the multivariate time series by combining the fusion features with the channel attention module and the multi-head attention module and generate a fusion result;

[0201] An anomaly score module 60, for reconstructing the target multivariate time series using a preset training model to determine an anomaly score;

[0202] The abnormality judgment module 70 is used to obtain a preset threshold and judge whether the multivariate time series is abnormal according to the preset threshold and the abnormality score.

[0203] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0204] This embodiment generates a target multivariate time series by acquiring a multivariate time series and performing preprocessing; determines a sliding window sequence; uses a gated recurrent unit to extract time features in the sliding window sequence, and uses a graph convolutional neural network to extract dimension features in the sliding window sequence; generates fusion features based on time features and dimension features; combines fusion features through a channel attention module and a multi-head attention module to achieve fusion of global features and local features and generate fusion results; reconstructs the target multivariate time series using a preset training model to determine anomaly scores; obtains a preset threshold and determines whether the multivariate time series is abnormal. Improve the detection accuracy of abnormal data.

[0205] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0206] In addition, for technical details not described in detail in this embodiment, please refer to the method for low-light target detection provided in any embodiment of the present application, and will not be repeated here.

[0207] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0208] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation modes, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk), including several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present application. The above is only a preferred embodiment of the present application, and the patent scope of the present application is not limited thereto. All equivalent structures or equivalent process changes made by using the contents of the present application specification and the accompanying drawings, or directly or indirectly applied in other related technical fields, are similarly included in the patent protection scope of the present application.

Claims

1. A multivariate time series anomaly detection method, characterized in that: include: Acquire a multivariate time series and preprocess the multivariate time series to generate a target multivariate time series; Determining a sliding window sequence according to the target multivariate time series in the form of a sliding window; Using a gated recurrent unit to extract temporal features in the sliding window sequence, and using a graph convolutional neural network to extract dimensional features in the sliding window sequence; Generate a fusion feature according to the time feature and the dimensional feature; The channel attention module and the multi-head attention module are combined with the fusion features to achieve the fusion of the global features and local features of the multivariate time series and generate a fusion result; Reconstructing the target multivariate time series using a preset training model to determine an anomaly score; A preset threshold is obtained, and whether the multivariate time series is abnormal is determined according to the preset threshold and the abnormality score.

2. The multivariate time series anomaly detection method according to claim 1, characterized in that: The step of determining a sliding window sequence according to the target multivariate time series in the form of a sliding window comprises: Determine the data point x at timestamp t in the target multivariate time series t Dependencies; Model the dependencies and set a sliding window of length k for cutting: Where W is the sliding window sequence, T is the target multivariate time series, and x′ t is the normalized time series, w n-k+1 is the time subsequence containing features cut into.

3. The multivariate time series anomaly detection method according to claim 1, characterized in that: The step of extracting time features from the sliding window sequence using a gated recurrent unit and extracting dimensional features from the sliding window sequence using a graph convolutional neural network includes: A gated recurrent unit is used to extract temporal features in the sliding window sequence. Where x is the input vector consisting of multivariate time series data, F GRU is the GRU gated loop calculation function, W i is the parameter matrix of the fully connected layer, b i is the bias vector of the fully connected layer; A graph convolutional neural network is used to extract dimensional features in the sliding window sequence, and randomly initialized embedding matrices I1, I2 are used to represent features F1, F2, F1, F2 and adjacency matrix E∈R M×M The calculation process is: F1=tanh(I1q1),I1∈R M×k <h2 style=";text-align:left;direction:ltr">F2 = tanh(I2q2), I2∈R<h2 style=";text-align:left;direction:ltr"> M×k In the formula, N represents the number of graph nodes, i.e., the variable; e represents the embedding dimension of the graph nodes; q1 and q2 represent the fully connected network parameters; σ represents the Sigmoid function; Perform graph convolution operation, the calculation process of the graph convolution layer is: Where G is the number of graph convolution layers; X g,i is the output of the g-th layer graph convolution; α is the hyperparameter of the graph convolution layer; E' is the normalized adjacency matrix; The result of the graph convolution layer is calculated to determine the dimensional characteristics of the time series, and the calculation process is as follows:

4. The multivariate time series anomaly detection method according to claim 1, characterized in that: The step of fusing the global features and local features of the multivariate time series and generating a fusion result by combining the fusion features with the channel attention module and the multi-head attention module includes: The fusion feature F T,D ∈R T×C×W Mapping to feature F' T,D ∈R T'×C'×W' In the above example, the global average pooling layer AvgPool(·) is used to compress the fusion features to obtain the feature F (avg-c),i : F (avg-c),i =AvgPool(F T,D ); The global maximum pooling layer MaxPool(·) is used to compress the fusion features to obtain F (max-c),i ∈R C : F (max-c),i =MaxPool(F T,D ); F (avg-c),i and F (max-c),i ∈R C The sum is then calculated and the interdependence between channels is extracted through convolution. The calculation process is: F c,w =F (avg-c),i +F (max-c),i The characteristic values ​​of each channel are combined for weighted processing, and the calculation process is: H=F ex (Z)=σ(W2δ(W1Z)) Where W1 and W2 represent weight matrices, δ represents Relu function, and σ represents Sigmoid function; Apply the channel attention weights to the fused feature F T,D ∈R T×C×W The calculation process is: S=F scale (H,X)=F T,D H。 5. The multivariate time series anomaly detection method according to claim 4, characterized in that: The step of fusing the global features and local features of the multivariate time series and generating a fusion result by combining the fusion features with the channel attention module and the multi-head attention module also includes: The sequence S is processed using the position code PE, and the calculation process is: S i =Embedding(S)+PE(S) Among them, pos represents the absolute position of the value at a certain moment in the long time series, and d represents the dimension; The self-attention mechanism is used to filter out the key information in the time series and capture the dependency between feature vectors to highlight the key features. The calculation process is: S i ·[W q ,W k ,W v ]=[Q,K,V] Where W q , W k , W v is a parameter matrix used to calculate Q, K, and V vectors; Q, K, and V correspond to the query vector, key vector, and value vector, respectively. k is the number of dimensions of K.

6. The multivariate time series anomaly detection method according to claim 5, characterized in that: After the steps of implementing the global feature fusion of the target multivariate time series through the channel attention module and implementing the local feature fusion of the target multivariate time series through the multi-head attention module, the method further includes: The probabilistic sparse attention calculation method is introduced. The Kullback-Leibler (KL) divergence is used to select u important queries to calculate self-attention. The sparsity calculation process of the i-th query is: Among them, q i , k i , v i represent the i-th row in Q, K, and V respectively, The first half of the term represents q i Log-Sum-Exp (LSE) on all keys, the second half of the term represents their arithmetic square roots; For each q in Query i ∈R d and k in Key j ∈R d , there is the following relationship: Therefore, the maximum mean is used to calculate the sparsity of the i-th Query, and the calculation process is: Top-u is selected by prioritizing the above processes The size of u is determined by the number of samples, that is: Merge the average scores of Q and V. If q i If it belongs to Top-u, the score is calculated by A(Q,K,V); otherwise, the mean of V is used instead. The specific process is as follows: Where i(i∈{1,...,h}) is the number of the self-attention head, H i is the self-attention expression of the i-th self-attention, and h is the number of self-attention heads; The multi-head attention mechanism allows the model to perform self-attention calculations in different subspaces, thereby capturing the correlation between different positions in the sequence. The calculation process is: Where MHA(I1,I1,I1) represents the multi-head attention operation of the input matrix I1, and is added to its own matrix to obtain the input of the Feed Forward Networks layer At the same time, the input matrix Using the convolution operation, that is 7. The multivariate time series anomaly detection method according to claim 1, characterized in that: The step of reconstructing the target multivariate time series using a preset training model to determine anomaly scores includes: Adopting generative decoder Decoder1 and discriminative decoder Decoder2 to generate adversarial training module; Obtain a preset training model, and use the adversarial training module to train the preset training model in an adversarial training manner; The target multivariate time series is reconstructed using the trained preset training model to determine anomaly scores.

8. The multivariate time series anomaly detection method according to claim 7, characterized in that: The step of using the generation decoder Decoder1 and the identification decoder Decoder2 to generate the adversarial training module includes: Generate reconstructed data that is similar to the input window W, where the generating decoder Decoder1 generates Y1 and the discriminating decoder Decoder2 generates Y2; Calculate the reconstruction error The calculation method is: Where, ||·||2 represents the mean square error function; According to the reconstruction error As the focus matrix Combined with the reconstructed sequence Y1, the output Y2 of the identification decoder Decoder2 is obtained through model reasoning to determine the adversarial error The calculation method is: The loss is calculated based on the reconstruction error and the adversarial error: Where n is the training epoch and ε is a training parameter close to 1.

9. The multivariate time series anomaly detection method according to claim 1, characterized in that: Before the step of obtaining a preset threshold and judging whether the multivariate time series is abnormal according to the preset threshold and the abnormality score, the method includes: The preset threshold is determined by using the POT automatic threshold selection method based on extreme value theory.

10. A multivariate time series anomaly detection system, characterized in that: include: A preprocessing module, used for acquiring a multivariate time series and preprocessing the multivariate time series to generate a target multivariate time series; A sliding window module, used to determine a sliding window sequence according to the target multivariate time series in the form of a sliding window; A dimension extraction module, for extracting time features from the sliding window sequence using a gated recurrent unit, and for extracting dimensional features from the sliding window sequence using a graph convolutional neural network; A fusion feature module, used to generate a fusion feature according to the time feature and the dimensional feature; A fusion result module, used for fusing the global features and local features of the multivariate time series by combining the fusion features with the channel attention module and the multi-head attention module and generating a fusion result; an anomaly score module, for reconstructing the target multivariate time series using a preset training model to determine an anomaly score; The anomaly judgment module is used to obtain a preset threshold and judge whether the multivariate time series is abnormal according to the preset threshold and the anomaly score.

Images