Neural network-based time sequence anomaly detection method

Abstract

The invention discloses a time series anomaly detection method based on a neural network, and the method comprises the steps: obtaining to-be-detected multivariable time series data, and inputting the multivariable time series data into a spatial-temporal feature fusion unit, performing time domain information and attribute information analysis on the multivariable time sequence data by a time domain convolutional network and a graph attention influence network in the spatial-temporal feature fusion unit, and obtaining spatial-temporal fusion features through feature fusion; decomposing the multivariable time sequence data by adopting a self-adaptive time sequence decomposition algorithm to obtain a trend sequence and a periodic sequence; inputting the space-time fusion feature, the trend sequence and the periodic sequence into a time sequence reconstruction unit to obtain a decoding feature; performing anomaly detection on the decoding features through an anomaly detection unit to obtain an abnormal state of the multivariable time series data; according to the method, the reconstruction deviation between the potential abnormal data and the reconstruction data can be expanded, and the potential abnormal data in the multi-dimensional time sequence can be rapidly and accurately detected.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series anomaly detection, and in particular relates to a time series anomaly detection method based on a neural network. Background Art

[0002] Multivariate time series anomaly detection is of great significance in many fields. In the financial field, it can identify abnormal transactions, market manipulation and fraud, and improve market transparency and security. In industrial manufacturing, by analyzing equipment sensors and production line data, equipment failures and production anomalies can be discovered in a timely manner, improving production efficiency and reducing maintenance costs. In the field of network security, it can monitor network traffic and log data, quickly identify network attacks, abnormal traffic and intrusion behaviors, and enhance network security protection capabilities. This technology has shown great potential in practical applications and provides efficient risk prevention and control measures for various industries.

[0003] Traditional anomaly detection methods mainly include statistical and rule-based methods. Traditional methods work well when processing simple time series data, but they have obvious limitations when dealing with complex multidimensional time series data: first, multivariate time series data contains multiple features, and there are complex relationships between the features, which are difficult for traditional methods to capture effectively; second, the abnormal patterns in multidimensional data are complex and may only appear in specific feature combinations, which are difficult for traditional methods to identify; finally, many scenarios require real-time detection of anomalies, and traditional methods are difficult to meet real-time requirements when processing large-scale data. Summary of the invention

[0004] The present invention provides a time series anomaly detection method based on a neural network, which simultaneously utilizes the time domain information and spatial domain information of a multidimensional sequence, and expands the reconstruction deviation between potential abnormal data and reconstructed data, and then dynamically updates the threshold to determine the anomaly, thereby realizing fast and accurate detection of potential abnormal data in a multidimensional time series.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A first aspect of the present invention provides a time series anomaly detection method based on a neural network, comprising:

[0007] Acquire multivariate time series data to be detected and input it into a preset time series anomaly detection model; the time series anomaly detection model includes a spatiotemporal feature fusion unit, a time series reconstruction unit and an anomaly detection unit;

[0008] The multivariate time series data is input into the spatiotemporal feature fusion unit, and the temporal convolutional network and graph attention influence network in the spatiotemporal feature fusion unit analyze the temporal information and attribute information of the multivariate time series data, and obtain the spatiotemporal fusion features through feature fusion. ;

[0009] Adopting adaptive time series decomposition algorithm to decompose multivariate time series data to obtain trend series G and period series S;

[0010] The time series reconstruction unit includes an encoder, a feature compression module and a decoder; the trend sequence G and the spatiotemporal fusion feature Input to the encoder, and obtain the encoded features through the cross attention mechanism and feedforward neural network in the encoder to enhance the feature representation ; Use feature compression module to encode features Perform redundant filtering to obtain coding compression features ; Encode the compressed features The periodic sequence S is input to the decoder, and the decoding features are obtained by feature reconstruction through the KL (Kullback-Leibler) sparse self-attention mechanism and cross-attention mechanism in the decoder. ;

[0011] Decoding features through anomaly detection unit Anomaly detection is performed to obtain the abnormal state of multivariate time series data.

[0012] Furthermore, the temporal convolutional network and the graph attention influence network in the spatiotemporal feature fusion unit analyze the temporal information and attribute information of the multivariate time series data, and obtain the spatiotemporal fusion feature through feature fusion. , including:

[0013] Input the multivariate time series data into the first time domain convolutional network to obtain the time series fusion feature, which is expressed as:

[0014]

[0015] In the formula, is the time series fusion feature at the tth moment, is the timestamp of the multivariate time series data, is the ReLU nonlinear activation function, is the number of data points involved in the convolution operation of the convolution kernel in the time domain convolution network; is the serial number of the convolution kernel in the time domain convolutional network; is the first time domain convolutional network The weight of the convolution kernel; is the expansion factor; is the bias term; is the multivariate time series data at time t; In accordance with Indexing multivariate time series data;

[0016] The attribute information of the time series fusion feature is regarded as the node of the graph structure; a directed edge connection is established between each node to obtain a directed graph; the association strength of the directed edge between two adjacent nodes is calculated, and the expression formula is:

[0017]

[0018] In the formula, is the i-th node, recorded as the target node; is the jth node, recorded as the adjacent node; is the association strength between the target node and the adjacent nodes; T represents the matrix transpose; is the candidate node set of the target node, , is the set of nodes of a directed graph; Expressed as The node set after removing the target node;

[0019] Sort the association strength of the directed edges between the target node and the adjacent nodes and select the top The directed edge of the target node is expressed as:

[0020]

[0021] In the formula, is the adjacency matrix of the target node; Represented as the node's serial number; It is represented as the set of association strengths of directed edges between the target node and adjacent nodes; is the set of association strengths Sort and select first Directed edges to target nodes;

[0022] Traverse each node in the directed graph to obtain a simplified directed graph; use the SoftMax function to calculate the weight of the directed edge from the adjacent node to the target node, and the expression formula is:

[0023]

[0024] In the formula, The weight of the directed edge from the adjacent node to the target node; For the directed graph nodes; The target node and the node The strength of the association between To simplify the node set of a directed graph;

[0025] According to the directed edges and weights in the simplified directed graph, the target node is updated to obtain the spatiotemporal features. The expression formula is:

[0026]

[0027] In the formula, is the updated feature representation of the target node; is the learnable weight matrix; For Node The weight of the directed edge pointing to the target node; ReLU nonlinear activation function; || represents vector concatenation operation;

[0028] Use the second time domain convolutional network to perform time domain feature analysis on the time domain features to obtain time domain fusion features .

[0029] Furthermore, an adaptive time series decomposition algorithm is used to decompose the multivariate time series data to obtain the trend sequence G and the period sequence S, which specifically includes:

[0030] Construct N seasonal trend decomposition units (Seasonal-Trend Decomposition using LOESS, referred to as STL units); decompose multivariate time series data through seasonal trend decomposition units to obtain trend sub-items and period sub-items , the expression formula is:

[0031]

[0032] In the formula, For the Hyperparameters of seasonal trend decomposition units; is a learnable matrix; is multivariate time series data; is the serial number of the seasonal trend decomposition unit; Decompose the units into seasonal trends;

[0033] Through the SoftMax function according to the periodic sub-item Generate data item weights and assign trend sub-items based on the data item weights. and period sub-items The trend sequence G and period sequence S are accumulated and expressed as follows:

[0034]

[0035]

[0036] In the formula, is the data item weight; MLP is a multi-layer perceptron.

[0037] Furthermore, the trend sequence G and the spatiotemporal fusion features Input to the encoder, and obtain the encoded features through the cross attention mechanism and feedforward neural network in the encoder to enhance the feature representation , including:

[0038] The trend sequence G is linearly transformed through the cross attention mechanism to obtain the first query item, and the spatiotemporal fusion features are Perform linear transformation to obtain the first key item and the first value item, and the expression formula is:

[0039]

[0040] In the formula, is the first query item; is the first key item; is the first value item; , and is a learnable matrix;

[0041] The first weight matrix is ​​calculated according to the first query item and the first key item, and the expression formula is:

[0042]

[0043] In the formula, is the first weight matrix; T represents the matrix transpose; is the characteristic dimension of the first key item; is the Softmax function;

[0044] Multiply the first weight matrix and the first value item and combine them with the spatiotemporal fusion feature Obtain the encoding intermediate features by adding residual connections , the expression formula is:

[0045] ;

[0046] Encoding intermediate features Normalization is performed to obtain coding standard features, the coding standard features are input into the feedforward neural network to obtain coding output features, and the coding output features are normalized to obtain coding features ; The expression formula is:

[0047]

[0048]

[0049]

[0050] In the formula, is the layer normalization function; To encode standard features; Output features for encoding; It is a feed-forward neural network.

[0051] Furthermore, the feature compression module is used to encode the features Perform redundant filtering to obtain coding compression features , including:

[0052] Encoding features Add channel dimension to obtain 3D encoding features ,in, , ; L is the sequence length; D is the feature dimension; C is the number of channels;

[0053] 3D coded features Input to the residual network for dimensionality reduction to obtain abstract encoding features

[0054] 3D coded features Convert to the second value item ; Abstract encoding features Convert to second key and the second query item ;

[0055] Use the pooling layer to filter the second query item Downsampling to obtain key semantic features ;in, is the compression ratio;

[0056] For the second value , key semantic features and the second key item Perform shape reshaping to obtain the reshaping value item , Reshape query items and reshape key items ;

[0057] Reshape value items according to , Reshape query items and reshape key items Generate encoding filter features, the expression formula is:

[0058]

[0059] In the formula, To encode the filtering features, ; T represents matrix transpose;

[0060] The encoded filter features are transformed into encoded compressed features through the reshaping function .

[0061] Furthermore, the decoded features are obtained by reconstructing the features through the KL sparse self-attention mechanism and the cross attention mechanism in the decoder. , specifically including:

[0062] Perform a linear transformation on the periodic sequence S to obtain the third value term , the third key and the third query item , the expression formula is:

[0063]

[0064] In the formula, , and is a learnable matrix;

[0065] The third value , the third key and the third query item Input to the KL sparse self-attention mechanism and calculate the third query item using the KL divergence measure The activity score of the query vector in is expressed as:

[0066]

[0067] In the formula, The third query item Middle query vector; The query vector activity score; L is the sequence length, The third key The rth key vector in ; g is the vector embedding dimension; T is represented by the matrix transpose;

[0068] According to the activity score in the third query Select query vectors to construct a sparse query matrix , according to the sparse query matrix , the third value item and the third key Generate decoding intermediate features, the expression formula is:

[0069]

[0070] In the formula, To decode the intermediate features; is the layer normalization function, is the characteristic dimension of the third key item; The third value item The mean vector of

[0071] Perform linear transformation on the decoded intermediate features to obtain the fourth query item ; Encoding compression features Perform a linear transformation to obtain the fourth value term and the fourth key , the expression formula is:

[0072]

[0073] The second weight matrix is ​​calculated according to the fourth query item and the fourth key item, and the expression formula is:

[0074]

[0075] In the formula, is the second weight matrix; T represents the matrix transpose; It is the characteristic dimension of the fourth key item; is the Softmax function;

[0076] The second weight matrix is ​​multiplied by the fourth value item and then added to the decoded intermediate feature through residual connection to obtain the decoded output feature , the expression formula is:

[0077]

[0078] Decoding output features Normalize to get decoding features .

[0079] Furthermore, the decoded features are analyzed by an anomaly detection unit. Anomaly detection is performed to obtain the abnormal state of multivariate time series data, including:

[0080] Decoding features through feedforward neural network Perform dimensionality reduction processing to obtain the time series reconstruction sequence, the expression formula is:

[0081]

[0082] In the formula, is the sequence length of the multivariate time series data, is the total number of attributes of the multivariate time series data; Reconstruct the sequence for time series; It is a feed-forward neural network;

[0083] The deviation sequence is obtained by calculating the reconstruction deviation between the time series reconstruction sequence and the multivariate time series data; the expression formula is:

[0084]

[0085]

[0086] In the formula, Reconstruction deviation between time series reconstruction sequence and multivariate time series data; is the deviation sequence; Enter features into multivariate time series data; The time series reconstruction feature in the time series reconstruction sequence; is the attribute serial number of multivariate time series data; is the length sequence number of the multivariate time series data;

[0087] The exponentially weighted moving average method is used to smooth the deviation sequence to obtain an abnormal score, and the expression formula is:

[0088]

[0089] In the formula, and Reconstruction bias and reconstruction bias Abnormal score of Tuning parameters for anomaly assessment;

[0090] The abnormal state of multivariate time series data is obtained based on the anomaly score and the preset anomaly judgment threshold.

[0091] Furthermore, it also includes: fitting the generalized Pareto distribution of the abnormal score sequence by maximum likelihood estimation to obtain the scale evaluation parameter and shape evaluation parameters , the expression formula is:

[0092]

[0093] In the formula, Fitting function for generalized Pareto distribution; Evaluate parameters for the location;

[0094] Evaluate parameters according to scale and shape evaluation parameters Update the abnormality judgment threshold, the expression formula is:

[0095]

[0096] In the formula, is the updated abnormality determination threshold, is the abnormality judgment threshold before updating, is the number of samples with abnormal status in multivariate time series data; is the total number of samples in the multivariate time series data; is the set balance parameter.

[0097] Furthermore, the training process of the time series anomaly detection model includes a reconstruction network training phase and a classification network training phase;

[0098] Acquire multivariate time series storage data as training samples, wherein the training samples are configured with real labels, and use the training samples to train a time series anomaly detection model;

[0099] In the reconstruction network training phase, the weight parameters of the anomaly detection unit are frozen; the training samples are input into the time series anomaly detection model, and the time series reconstruction unit outputs the time series reconstruction training features; the mean square error between the time series reconstruction training features and the training samples is calculated as the first training loss, which is expressed as:

[0100]

[0101] In the formula, is the mean square error; The number of training samples in the reconstruction network training phase or the classification network training phase; is the mth training sample; Reconstruct training features for the mth time series;

[0102] According to the first training loss, the weight parameters of the spatiotemporal feature fusion unit and the time series reconstruction unit are optimized, and the iterative reconstruction network training phase is repeated until the first training loss converges to output the time series anomaly detection model;

[0103] In the classification network training stage, the weight parameters of the anomaly detection unit are unfrozen, and the training samples are input into the time series anomaly detection model output in the reconstruction network training stage to obtain the anomaly determination training results. The binary cross entropy loss function is used to calculate the second training loss between the anomaly determination training results and the true label. The expression formula is:

[0104]

[0105] In the formula, BCE is the binary cross entropy loss value; is the true label of the mth training sample; is the mth abnormality judgment training result;

[0106] The weight parameters of the time series anomaly detection model are optimized according to the second training loss, and the classification network training phase is repeated until the second training loss converges to output the final time series anomaly detection model.

[0107] The second aspect of the present invention provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the neural network-based time series anomaly detection method described in the first aspect.

[0108] Compared with the prior art, the present invention has the following beneficial effects:

[0109] The present invention inputs the multivariate time series data into the spatiotemporal feature fusion unit, and the time domain convolution network and the graph attention influence network in the spatiotemporal feature fusion unit analyze the time domain information and attribute information of the multivariate time series data, and obtains the spatiotemporal fusion feature through feature fusion. ,Mining the high-dimensional latent state representation of multivariate time series data can capture the key ,information in the data more comprehensively, which is conducive to improving the ,prediction accuracy of the time series anomaly detection model.

[0110] The present invention adopts an adaptive time series decomposition algorithm to decompose multivariate time series data to obtain a trend sequence G and a period sequence S; the adaptive decomposition algorithm can dynamically adjust the decomposition process according to the characteristics of the data, so as to better adapt to different types of multivariate time series data, and can enhance the generalization ability of the time series anomaly detection model; at the same time, the trend sequence G and the period sequence S of the multivariate time series data are clearly separated, which is conducive to the time series anomaly detection model to more accurately capture the main features in the data.

[0111] The time series reconstruction unit in the present invention includes an encoder, a feature compression module and a decoder; the trend sequence G and the spatiotemporal fusion feature Input to the encoder, and obtain the encoded features through the cross attention mechanism and feedforward neural network in the encoder to enhance the feature representation ; Use feature compression module to encode features Perform redundant filtering to obtain coding compression features ; Encode the compressed features The periodic sequence S is input to the decoder, and the decoding features are obtained by feature reconstruction through the KL sparse self-attention mechanism and cross attention mechanism in the decoder ; It can expand the reconstruction deviation between potential abnormal data and reconstructed data, and realize fast and accurate detection of potential abnormal data in multidimensional time series. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Figure 1 This is a structural diagram of the time series anomaly detection model provided by Example 1 of the present invention.

[0113] Figure 2 is a flowchart of a time series anomaly detection method based on a neural network provided in Example 1 of the present invention;

[0114] Figure 3 is a structural diagram of a feature compression module provided in Example 1 of the present invention;

[0115] Figure 4 This is a flowchart of the KL sparse self-attention mechanism provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0116] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0117] Example 1

[0118] like Figures 1 to 4 As shown, this implementation provides a time series anomaly detection method based on a neural network, including:

[0119] Build a time series anomaly detection model, including:

[0120] The spatiotemporal feature fusion unit is composed of a temporal convolutional network (TCN) and a graph attention influence network (GAIN). In this embodiment, the spatiotemporal feature fusion unit includes a first temporal convolutional network, a graph attention influence network, and a second temporal convolutional network connected in sequence.

[0121] A time reconstruction unit is constructed by an encoder, a feature compression module and a decoder; the encoder comprises a cross attention mechanism, layer normalization and a feedforward neural network; the decoder comprises a KL sparse self-attention mechanism, layer normalization and a cross attention mechanism;

[0122] An anomaly detection unit is constructed based on the exponentially weighted moving average method, and a time series anomaly detection model is constructed according to the spatiotemporal feature fusion unit, the time series reconstruction unit and the anomaly detection unit.

[0123] The training process of the time series anomaly detection model includes a reconstruction network training stage and a classification network training stage;

[0124] Acquire multivariate time series storage data as training samples, wherein the training samples are configured with real labels, and use the training samples to train a time series anomaly detection model;

[0125] In the reconstruction network training phase, the weight parameters of the anomaly detection unit are frozen; the training samples are input into the time series anomaly detection model, and the time series reconstruction unit outputs the time series reconstruction training features; the mean square error between the time series reconstruction training features and the training samples is calculated as the first training loss, which is expressed as:

[0126]

[0127] In the formula, is the mean square error; The number of training samples in the reconstruction network training phase or the classification network training phase; is the mth training sample; Reconstruct training features for the mth time series;

[0128] According to the first training loss, the weight parameters of the spatiotemporal feature fusion unit and the time series reconstruction unit are optimized, and the iterative reconstruction network training phase is repeated until the first training loss converges to output the time series anomaly detection model;

[0129] In the classification network training stage, the weight parameters of the anomaly detection unit are unfrozen, and the training samples are input into the time series anomaly detection model output in the reconstruction network training stage to obtain the anomaly determination training results. The binary cross entropy loss function is used to calculate the second training loss between the anomaly determination training results and the true label. The expression formula is:

[0130]

[0131] In the formula, BCE is the binary cross entropy loss value; is the true label of the mth training sample; is the mth abnormality judgment training result;

[0132] The weight parameters of the time series anomaly detection model are optimized according to the second training loss, and the classification network training phase is repeated until the second training loss converges to output the final time series anomaly detection model.

[0133] In this embodiment, through phased training, the time series anomaly detection model can learn the characteristics of the data more efficiently. In the reconstruction network stage, the time series anomaly detection model focuses on learning the normal mode of the data, thereby reducing the interference of abnormal data on model training and the complexity of training. In the classification network stage, the time series anomaly detection model can learn the characteristics of abnormal data more efficiently, thereby improving the overall robustness of the model.

[0134] Use the trained time series anomaly detection model to perform anomaly detection on multivariate time series data, including:

[0135] Obtain the multivariate time series data to be detected and input it into a preset time series anomaly detection model;

[0136] The multivariate time series data is input into the spatiotemporal feature fusion unit, and the temporal convolutional network and graph attention influence network in the spatiotemporal feature fusion unit analyze the temporal information and attribute information of the multivariate time series data, and obtain the spatiotemporal fusion features through feature fusion. , including:

[0137] Input the multivariate time series data into the first time domain convolutional network to obtain the time series fusion feature, which is expressed as:

[0138]

[0139] In the formula, is the time series fusion feature at the tth moment, is the timestamp of the multivariate time series data, is the ReLU nonlinear activation function, is the number of data points involved in the convolution operation of the convolution kernel in the time domain convolution network; is the serial number of the convolution kernel in the time domain convolutional network; is the first time domain convolutional network The weight of the convolution kernel; is the expansion factor; is the bias term; is the multivariate time series data at time t; In accordance with Indexing multivariate time series data;

[0140] The temporal fusion features are input into the graph attention influence network. The graph attention influence network uses the attention mechanism to calculate the spatial structure associations between different nodes and their related weights, thereby updating the feature representation of each node, including:

[0141] The attribute information of the time series fusion feature is regarded as the node of the graph structure; a directed edge connection is established between each node to obtain a directed graph; the association strength of the directed edge between two adjacent nodes is calculated, and the expression formula is:

[0142]

[0143] In the formula, is the i-th node, recorded as the target node; is the jth node, recorded as the adjacent node; is the association strength between the target node and the adjacent nodes; T represents the matrix transpose; is the candidate node set of the target node, , is the set of nodes of a directed graph; Expressed as The node set after removing the target node;

[0144] Delete redundant directed edge connections, sort the association strengths of directed edges between the target node and adjacent nodes, and select the top The directed edge of the target node is expressed as:

[0145]

[0146] In the formula, is the adjacency matrix of the target node; Represented as the node's serial number; It is represented as the set of association strengths of directed edges between the target node and adjacent nodes; is the set of association strengths Sort and select first Directed edges to target nodes;

[0147] Traverse each node in the directed graph to obtain a simplified directed graph; use the SoftMax function to calculate the weight of the directed edge from the adjacent node to the target node, and the expression formula is:

[0148]

[0149] In the formula, The weight of the directed edge from the adjacent node to the target node; For the directed graph nodes; The target node and the node The strength of the association between To simplify the node set of a directed graph;

[0150] According to the directed edges and weights in the simplified directed graph, the target node is updated to obtain the spatiotemporal features. The expression formula is:

[0151]

[0152] In the formula, is the updated feature representation of the target node; is the learnable weight matrix; For Node The weight of the directed edge pointing to the target node; ReLU nonlinear activation function; || represents vector concatenation operation;

[0153] Use the second time domain convolutional network to perform time domain feature analysis on the time domain features to obtain time domain fusion features In this embodiment, the time domain convolutional network uses dilated convolution to expand the receptive field, and combined with residual connection, the stability and efficiency of model training can be improved.

[0154] Adaptive time series decomposition algorithm is used to decompose multivariate time series data to obtain trend series G and period series S, including:

[0155] Construct N seasonal trend decomposition units (Seasonal-Trend Decomposition using LOESS, referred to as STL units); decompose multivariate time series data through seasonal trend decomposition units to obtain trend sub-items and period sub-items , the expression formula is:

[0156]

[0157] In the formula, For the Hyperparameters of seasonal trend decomposition units; is a learnable matrix; is multivariate time series data; is the serial number of the seasonal trend decomposition unit; Decompose the units into seasonal trends;

[0158] Through the SoftMax function according to the periodic sub-item Generate data item weights and assign trend sub-items based on the data item weights. and period sub-items The trend sequence G and period sequence S are accumulated and expressed as follows:

[0159]

[0160]

[0161] In the formula, is the data item weight; MLP is a multi-layer perceptron.

[0162] In this embodiment, the adaptive decomposition algorithm can dynamically adjust the decomposition process according to the characteristics of the data, so as to better adapt to different types of multivariate time series data and enhance the generalization ability of the time series anomaly detection model; at the same time, it clearly separates the trend sequence G and the periodic sequence S of the multivariate time series data, which is conducive to the time series anomaly detection model to more accurately capture the main features in the data.

[0163] The trend sequence G and the spatiotemporal fusion features Input to the encoder, and obtain the encoded features through the cross attention mechanism and feedforward neural network in the encoder to enhance the feature representation , including:

[0164] The trend sequence G is linearly transformed through the cross attention mechanism to obtain the first query item, and the spatiotemporal fusion features are Perform linear transformation to obtain the first key item and the first value item, and the expression formula is:

[0165]

[0166] In the formula, is the first query item; is the first key item; is the first value item; , and is a learnable matrix;

[0167] The first weight matrix is ​​calculated according to the first query item and the first key item, and the expression formula is:

[0168]

[0169] In the formula, is the first weight matrix; T represents the matrix transpose; is the characteristic dimension of the first key item; is the Softmax function;

[0170] Multiply the first weight matrix and the first value item and combine them with the spatiotemporal fusion feature Obtain the encoding intermediate features by adding residual connections , the expression formula is:

[0171] ;

[0172] Encoding intermediate features Normalization is performed to obtain coding standard features, the coding standard features are input into the feedforward neural network to obtain coding output features, and the coding output features are normalized to obtain coding features ; The expression formula is:

[0173]

[0174]

[0175]

[0176] In the formula, is the layer normalization function; To encode standard features; Output features for encoding; It is a feedforward neural network; among them, layer normalization can standardize each activation value, reduce the gradient explosion or gradient disappearance problem, and improve the convergence speed of the model.

[0177] Using feature compression module to encode features Perform redundant filtering to obtain coding compression features , specifically including:

[0178] Encoding features Add channel dimension to obtain 3D encoding features ,in, , ; L is the sequence length; D is the feature dimension; C is the number of channels;

[0179] 3D coded features Input to the residual network for dimensionality reduction to obtain abstract encoding features

[0180] 3D coded features Convert to the second value item ; Abstract encoding features Convert to second key and the second query item ;

[0181] Use the pooling layer to filter the second query item Downsampling to obtain key semantic features ;in, is the compression ratio;

[0182] For the second value , key semantic features and the second key item Perform shape reshaping to obtain reshaping value items , Reshape query items and reshape key items ;

[0183] Reshape value items according to , Reshape query items and reshape key items Generate encoding filter features, the expression formula is:

[0184]

[0185] In the formula, To encode the filtering features, ; T represents the matrix transpose;

[0186] The encoded filter features are transformed into encoded compressed features through the reshaping function The feature compression module can reduce the redundant information in high-dimensional coding features while maintaining the same semantic information, solving the problem that high-dimensional coding features contain more redundant information and improving the reliability and efficiency of the decoding stage.

[0187] Encoding compression features The periodic sequence S is input to the decoder, and the decoding features are obtained by feature reconstruction through the KL (Kullback-Leibler) sparse self-attention mechanism and cross-attention mechanism in the decoder. , including:

[0188] Perform a linear transformation on the periodic sequence S to obtain the third value term , the third key and the third query item , the expression formula is:

[0189]

[0190] In the formula, , and is a learnable matrix;

[0191] The third value , the third key and the third query item Input to the KL sparse self-attention mechanism and calculate the third query item using the KL divergence measure The activity score of the query vector in is expressed as:

[0192]

[0193] In the formula, The third query item Middle query vector; The query vector activity score; L is the sequence length, The third key The rth key vector in ; g is the vector embedding dimension; T is represented by the matrix transpose;

[0194] According to the activity score in the third query Select query vectors to construct a sparse query matrix , according to the sparse query matrix , the third value item and the third key Generate decoding intermediate features, the expression formula is:

[0195]

[0196] In the formula, To decode the intermediate features; is the layer normalization function, is the characteristic dimension of the third key item; The third value item The mean vector of

[0197] Perform linear transformation on the decoded intermediate features to obtain the fourth query item ; Encoding compression features Perform a linear transformation to obtain the fourth value term and the fourth key , the expression formula is:

[0198]

[0199] The second weight matrix is ​​calculated according to the fourth query item and the fourth key item, and the expression formula is:

[0200]

[0201] In the formula, is the second weight matrix; T represents the matrix transpose; It is the characteristic dimension of the fourth key item; is the Softmax function;

[0202] The second weight matrix is ​​multiplied by the fourth value item and then added to the decoded intermediate feature through residual connection to obtain the decoded output feature , the expression formula is:

[0203]

[0204] Decoding output features Normalize to get decoding features ; The time series reconstruction unit can expand the reconstruction deviation between potential abnormal data and reconstructed data, which is conducive to the rapid and accurate detection of potential abnormal data in multidimensional time series.

[0205] Decoding features through anomaly detection unit Anomaly detection is performed to obtain the abnormal state of multivariate time series data, including:

[0206] Decoding features through feedforward neural network Perform dimensionality reduction processing to obtain the time series reconstruction sequence, the expression formula is:

[0207]

[0208] In the formula, is the sequence length of the multivariate time series data, is the total number of attributes of the multivariate time series data; Reconstruct the sequence for time series; It is a feed-forward neural network;

[0209] The deviation sequence is obtained by calculating the reconstruction deviation between the time series reconstruction sequence and the multivariate time series data; the expression formula is:

[0210]

[0211]

[0212] In the formula, Reconstruction deviation between time series reconstruction sequence and multivariate time series data; is the deviation sequence; Enter features into multivariate time series data; The time series reconstruction feature in the time series reconstruction sequence; is the attribute serial number of multivariate time series data; is the length sequence number of the multivariate time series data;

[0213] The exponentially weighted moving average method is used to smooth the deviation sequence to obtain an abnormal score, and the expression formula is:

[0214]

[0215] In the formula, and Reconstruction bias and reconstruction bias Abnormal score of Tuning parameters for anomaly assessment;

[0216] The abnormal state of multivariate time series data is obtained based on the anomaly score and the preset anomaly judgment threshold.

[0217] The scale evaluation parameters are obtained by fitting the generalized Pareto distribution to the abnormal score sequence through maximum likelihood estimation. and shape evaluation parameters , the expression formula is:

[0218]

[0219] In the formula, Fitting function for generalized Pareto distribution; Evaluate parameters for the location;

[0220] Evaluate parameters according to scale and shape evaluation parameters Update the abnormality judgment threshold, the expression formula is:

[0221]

[0222] In the formula, is the updated abnormality determination threshold, is the abnormality judgment threshold before updating, is the number of samples with abnormal status in multivariate time series data; is the total number of samples in the multivariate time series data; is the set balance parameter.

[0223] In this embodiment, the anomaly detection unit determines the abnormal state of the data by dynamically updating the anomaly threshold, which can better handle the fluctuation problem of the detection data and make the model more robust when facing complex and unstable environments.

[0224] Example 2

[0225] This embodiment provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the neural network-based time series anomaly detection method described in Example 1.

[0226] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0227] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0228] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0229] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0230] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A time series anomaly detection method based on neural network, characterized in that: include: Obtain the multivariate time series data to be detected and input it into a preset time series anomaly detection model; The time series anomaly detection model includes a spatiotemporal feature fusion unit, a time series reconstruction unit and an anomaly detection unit; The multivariate time series data is input into the spatiotemporal feature fusion unit, and the temporal convolutional network and graph attention influence network in the spatiotemporal feature fusion unit analyze the temporal information and attribute information of the multivariate time series data, and obtain the spatiotemporal fusion features through feature fusion. ; Adopting adaptive time series decomposition algorithm to decompose multivariate time series data to obtain trend series G and period series S; The time series reconstruction unit includes an encoder, a feature compression module and a decoder; the trend sequence G and the spatiotemporal fusion feature Input to the encoder, and obtain the encoded features through the cross attention mechanism and feedforward neural network in the encoder to enhance the feature representation ; Using feature compression module to encode features Perform redundant filtering to obtain coding compression features ; Encode the compressed features The periodic sequence S is input to the decoder, and the decoding features are obtained by feature reconstruction through the KL sparse self-attention mechanism and cross attention mechanism in the decoder ; Decoding features through anomaly detection unit Anomaly detection is performed to obtain abnormal states of multivariate time series data.

2. The time series anomaly detection method according to claim 1, characterized in that: The temporal convolutional network and graph attention influence network in the spatiotemporal feature fusion unit analyze the temporal information and attribute information of multivariate time series data, and obtain the spatiotemporal fusion features through feature fusion. , including: Input the multivariate time series data into the first time domain convolutional network to obtain the time series fusion feature, which is expressed as: ； In the formula, is the time series fusion feature at the tth moment, is the timestamp of the multivariate time series data, is the ReLU nonlinear activation function, is the number of data points involved in the convolution operation of the convolution kernel in the time domain convolution network; is the serial number of the convolution kernel in the time domain convolutional network; is the first time domain convolutional network The weight of the convolution kernel; is the expansion factor; is the bias term; is the multivariate time series data at time t; In accordance with Indexing multivariate time series data; The attribute information of the time series fusion feature is regarded as the node of the graph structure; a directed edge connection is established between each node to obtain a directed graph; the association strength of the directed edge between two adjacent nodes is calculated, and the expression formula is: ； In the formula, is the i-th node, recorded as the target node; is the jth node, recorded as the adjacent node; is the association strength between the target node and the adjacent nodes; T represents the matrix transpose; is the candidate node set of the target node, , is the set of nodes of a directed graph; Expressed as The node set after removing the target node; Sort the association strength of the directed edges between the target node and the adjacent nodes and select the top The directed edge of the target node is expressed as: ； In the formula, is the adjacency matrix of the target node; Represented as the node's serial number; It is represented as the set of association strengths of directed edges between the target node and adjacent nodes; is the set of association strengths Sort and select first Directed edges to target nodes; Traverse each node in the directed graph to obtain a simplified directed graph; use the SoftMax function to calculate the weight of the directed edge from the adjacent node to the target node, and the expression formula is: ； In the formula, The weight of the directed edge from the adjacent node to the target node; For the directed graph nodes; The target node and the node The strength of the association between To simplify the node set of a directed graph; According to the directed edges and weights in the simplified directed graph, the target node is updated to obtain the spatiotemporal features. The expression formula is: ； In the formula, is the updated feature representation of the target node; is the learnable weight matrix; For Node The weight of the directed edge pointing to the target node; ReLU nonlinear activation function; || represents vector concatenation operation; Use the second time domain convolutional network to perform time domain feature analysis on the time domain features to obtain time domain fusion features .

3. The time series anomaly detection method according to claim 1, characterized in that: Adaptive time series decomposition algorithm is used to decompose multivariate time series data to obtain trend series G and period series S, including: Construct N seasonal trend decomposition units; decompose multivariate time series data through seasonal trend decomposition units to obtain trend sub-items and period sub-items , the expression formula is: ； In the formula, For the Hyperparameters of seasonal trend decomposition units; is a learnable matrix; is multivariate time series data; is the serial number of the seasonal trend decomposition unit; Decompose the units into seasonal trends; Through the SoftMax function according to the periodic sub-item Generate data item weights and assign trend sub-items based on the data item weights. and period sub-items The trend sequence G and the period sequence S are accumulated and expressed as follows: ； ； In the formula, is the data item weight; MLP is a multi-layer perceptron.

4. The time series anomaly detection method according to claim 3, characterized in that: The trend sequence G and the spatiotemporal fusion features Input to the encoder, and obtain the encoded features through the cross attention mechanism and feedforward neural network in the encoder to enhance the feature representation , including: The trend sequence G is linearly transformed through the cross attention mechanism to obtain the first query item, and the spatiotemporal fusion features are Perform linear transformation to obtain the first key item and the first value item, and the expression formula is: ； In the formula, is the first query item; is the first key item; is the first value item; , and is a learnable matrix; The first weight matrix is ​​calculated according to the first query item and the first key item, and the expression formula is: ； In the formula, is the first weight matrix; T represents the matrix transpose; is the characteristic dimension of the first key item; is the Softmax function; Multiply the first weight matrix and the first value item and combine them with the spatiotemporal fusion feature Obtain the encoding intermediate features by adding residual connections , the expression formula is: ； Encoding intermediate features Normalization is performed to obtain coding standard features, the coding standard features are input into the feedforward neural network to obtain coding output features, and the coding output features are normalized to obtain coding features ; The expression formula is: ； ； ； In the formula, is the layer normalization function; To encode standard features; Output features for encoding; It is a feed-forward neural network.

5. The time series anomaly detection method according to claim 1, characterized in that: Using feature compression module to encode features Perform redundant filtering to obtain coding compression features , including: Encoding features Add channel dimension to obtain 3D encoding features ,in, , ; L is the sequence length; D is the feature dimension; C is the number of channels; 3D coded features Input to the residual network for dimensionality reduction to obtain abstract encoding features ; The three-dimensional coding feature Convert to the second value item ; Abstract encoding features Convert to second key and the second query item ; Use the pooling layer to filter the second query item Downsampling to obtain key semantic features ;in, is the compression ratio; For the second value , key semantic features and the second key item Perform shape reshaping to obtain the reshaping value item , Reshape query items and reshape key items ; Reshape value items according to , Reshape query items and reshape key items Generate encoding filter features, the expression formula is: ； In the formula, To encode the filtering features, ; T represents the matrix transpose; The encoded filter features are transformed into encoded compressed features through the reshaping function .

6. The time series anomaly detection method according to claim 1, characterized in that: The decoded features are obtained by reconstructing the features through the KL sparse self-attention mechanism and the cross attention mechanism in the decoder. , including: Perform a linear transformation on the periodic sequence S to obtain the third value term , the third key and the third query item , the expression formula is: ； In the formula, , and is a learnable matrix; The third value , the third key and the third query item Input to the KL sparse self-attention mechanism and calculate the third query item using the KL divergence measure The activity score of the query vector in is expressed as: ； In the formula, The third query item Middle query vector; The query vector activity score; L is the sequence length, The third key The rth key vector in ; g is the vector embedding dimension; T is represented by the matrix transpose; According to the activity score in the third query Select query vectors to construct a sparse query matrix , according to the sparse query matrix , the third value item and the third key Generate decoding intermediate features, the expression formula is: ； In the formula, To decode the intermediate features; is the layer normalization function, is the characteristic dimension of the third key item; The third value item The mean vector of Perform linear transformation on the decoded intermediate features to obtain the fourth query item ; Encoding compression features Perform a linear transformation to obtain the fourth value term and the fourth key , the expression formula is: ； The second weight matrix is ​​calculated according to the fourth query item and the fourth key item, and the expression formula is: ； In the formula, is the second weight matrix; T represents the matrix transpose; It is the characteristic dimension of the fourth key item; is the Softmax function; The second weight matrix is ​​multiplied by the fourth value item and then added to the decoded intermediate feature through residual connection to obtain the decoded output feature , the expression formula is: ； Decoding output features Normalize to get decoding features .

7. The time series anomaly detection method according to claim 1, characterized in that: Decoding features through anomaly detection unit Anomaly detection is performed to obtain the abnormal state of multivariate time series data, including: Decoding features through feedforward neural network Perform dimensionality reduction processing to obtain the time series reconstruction sequence, the expression formula is: ； In the formula, is the sequence length of the multivariate time series data, is the total number of attributes of the multivariate time series data; Reconstruct the sequence for time series; It is a feed-forward neural network; The deviation sequence is obtained by calculating the reconstruction deviation between the time series reconstruction sequence and the multivariate time series data; the expression formula is: ； ； In the formula, Reconstruction deviation between time series reconstruction sequence and multivariate time series data; is the deviation sequence; Enter features into multivariate time series data; The time series reconstruction feature in the time series reconstruction sequence; is the attribute serial number of multivariate time series data; is the length sequence number of the multivariate time series data; The exponentially weighted moving average method is used to smooth the deviation sequence to obtain an abnormal score, and the expression formula is: ； In the formula, and Reconstruction bias and reconstruction bias Abnormal score of Tuning parameters for anomaly assessment; The abnormal state of multivariate time series data is obtained based on the anomaly score and the preset anomaly judgment threshold.

8. The time series anomaly detection method according to claim 7, characterized in that: Also includes: The scale evaluation parameters are obtained by fitting the generalized Pareto distribution to the abnormal score sequence through maximum likelihood estimation. and shape evaluation parameters , the expression formula is: ； In the formula, Fitting function for generalized Pareto distribution; Evaluate parameters for the location; Evaluate parameters according to scale and shape evaluation parameters Update the abnormality judgment threshold, the expression formula is: ； In the formula, is the updated abnormality determination threshold, is the abnormality judgment threshold before updating, is the number of samples with abnormal status in multivariate time series data; is the total number of samples in the multivariate time series data; is the set balance parameter.

9. The time series anomaly detection method according to claim 1, characterized in that: The training process of the time series anomaly detection model includes a reconstruction network training stage and a classification network training stage; Acquire multivariate time series storage data as training samples, wherein the training samples are configured with real labels, and use the training samples to train a time series anomaly detection model; In the reconstruction network training phase, the weight parameters of the anomaly detection unit are frozen; the training samples are input into the time series anomaly detection model, and the time series reconstruction unit outputs the time series reconstruction training features; Calculate the mean square error of the time series reconstruction training features and training samples as the first training loss, expressed as: ； In the formula, is the mean square error; The number of training samples in the reconstruction network training phase or the classification network training phase; is the mth training sample; Reconstruct training features for the mth time series; According to the first training loss, the weight parameters of the spatiotemporal feature fusion unit and the time series reconstruction unit are optimized, and the iterative reconstruction network training phase is repeated until the first training loss converges to output the time series anomaly detection model; In the classification network training stage, the weight parameters of the anomaly detection unit are unfrozen, and the training samples are input into the time series anomaly detection model output in the reconstruction network training stage to obtain the anomaly determination training results. The binary cross entropy loss function is used to calculate the second training loss between the anomaly determination training results and the true label. The expression formula is: ； In the formula, BCE is the binary cross entropy loss value; is the true label of the mth training sample; is the mth abnormality judgment training result; The weight parameters of the time series anomaly detection model are optimized according to the second training loss, and the classification network training phase is repeated until the second training loss converges to output the final time series anomaly detection model.

10. An electronic device comprising a storage medium and a processor; the storage medium is used to store instructions; characterized in that: The processor is used to operate according to the instructions to execute the neural network-based time series anomaly detection method according to any one of claims 1 to 9.

Images