Multi-dimensional time sequence anomaly detection method based on graph attention network

Abstract

The invention provides a multidimensional time series anomaly detection method based on a graph attention network, and relates to the technical field of machine learning. Comprising the following steps: acquiring multi-dimensional time sequence data, preprocessing the data, extracting features by using a one-dimensional convolution layer, inputting a constructed MSED-GAT prediction model, capturing an association relationship and time information between dimensions by using two GAT layers, fusing the outputs of the two GAT layers and the output after one-dimensional convolution, and finally encoding the fused data by using LSTM. Capturing a long-term dependency relationship in the sequence so as to obtain implicit feature representation of the whole input data; then, multi-scale fusion integrated decoding with different lengths is defined for decoding, a loss function is adopted, model training and model optimization are carried out, and the function of anomaly detection is achieved; the problem of error accumulation is solved, the reconstruction error rate is reduced, the anomaly detection accuracy is improved, the data accuracy is further improved, the feature extraction speed is increased, and the fault early warning accuracy is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a multidimensional time series anomaly detection method based on a graph attention network. Background Art

[0002] With the rapid development of the Internet of Things and big data technologies, anomaly detection in multidimensional time series data is playing an increasingly important role in industrial equipment monitoring, financial risk control, and healthcare monitoring. Traditional anomaly detection methods are primarily based on statistical models, such as ARIMA and isolation forests. While these methods are computationally efficient, they perform poorly when processing high-dimensional, nonlinear time series data.

[0003] In recent years, deep learning-based anomaly detection methods have made significant progress. Existing methods can be broadly categorized into three main categories based on their modeling approach: First, recurrent neural network-based methods utilize recurrent neural networks (RNNs) such as LSTM and GRU to model temporal dependencies in time series. For example, OmniAnomaly combines a variational autoencoder with an LSTM to capture temporal patterns. However, these methods often ignore correlations between variables of different dimensions, resulting in limited detection effectiveness in complex systems. Second, autoencoder-based methods utilize an encoder-decoder architecture to learn normal patterns in the data. For example, USAD employs adversarial training to improve reconstruction quality. While these methods can capture global features, they are prone to error accumulation during the decoding process, making reconstruction of long sequences particularly challenging. Third, graph neural network-based methods, such as MTAD-GAT, attempt to use graph neural networks to model relationships between variables. While these methods consider variable correlation, they suffer from two major limitations in practical applications: First, they inadequately integrate spatiotemporal features, typically focusing on a single perspective within the feature or temporal dimension; and second, the decoding process still utilizes a fixed-scale decoder, making it difficult to adapt to anomaly patterns at different time scales. In addition, existing methods rely heavily on manual experience or fixed rules in selecting anomaly determination thresholds and lack adaptive capabilities, resulting in the need for repeated parameter adjustments in different scenarios, reducing the practicality of the method.

[0004] The current field of multidimensional time series anomaly detection still faces the following technical challenges: how to effectively integrate information from the time dimension and feature dimension, solve the problem of error accumulation in the decoding process, and achieve adaptive determination of anomaly thresholds. Therefore, new solutions are urgently needed. Summary of the Invention

[0005] The main purpose of this invention is to provide a multi-dimensional time series anomaly detection method based on graph attention network, so as to solve the technical problems that the current anomaly detection has deficiencies in spatiotemporal feature fusion and decoding robustness, resulting in limited anomaly detection accuracy and a small scope of application.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is: a multi-dimensional time series anomaly detection method based on graph attention network, comprising the following steps: S1: Obtain multidimensional time series data, perform data preprocessing, and save it as a first data set. The first data set is divided into a training set and a test set with a preset ratio, and then features are extracted using a one-dimensional convolutional layer. S2: Build the MSED-GAT prediction model, input the training set into the built MSED-GAT prediction model for model training, and save the optimal MSED-GAT prediction model; The MSED-GAT prediction model consists of a graph attention network encoder spliced with a multi-scale ensemble decoder. The graph attention network encoder consists of a one-dimensional convolutional layer module connected to a feature-oriented GAT layer and a time-oriented GAT layer. The outputs of the three are concatenated along the feature direction and then connected to an LSTM encoding layer. The multi-scale ensemble decoder consists of multiple decoders of different lengths connected in sequence. S3: The test set is input into the optimal MSED-GAT prediction model for anomaly detection. When it is greater than the preset anomaly threshold, the system is considered abnormal; otherwise, it is judged to be normal.

[0007] In the preferred embodiment, data preprocessing includes: S11: Data cleaning and anomaly filtering for multidimensional time series data: using a multivariate joint outlier detection method: calculating the preset quantile of the sliding window for each dimension, removing data points that exceed the threshold range, and then using the robust Mahalanobis distance to identify multivariate joint outliers for dimension-related anomalies; S12: Perform normalization processing, the formula is: ; Where, is the maximum value, is the minimum value, For input data.

[0008] In a preferred embodiment, the method for constructing the feature-oriented GAT layer includes: Each dimension of the multidimensional time series is regarded as a node in the graph. The node feature is the time series data on that dimension. The graph attention mechanism is used to adaptively calculate the attention weight of the node and its neighboring nodes, and the node feature is updated by aggregating the neighboring node information. Specifically: Graph neural networks are used to model the relationship between different dimensions in multidimensional time series. Graph attention mechanisms are introduced to perform aggregation operations on neighboring nodes. Time windows are used to segment data. The time series in each dimension is represented by a node. The presence of an edge between nodes indicates a correlation between the two dimensions. The node aggregation operation formula is: ; Where, For nodes The new feature vector after the GAT layer, is the activation function, For nodes The neighbor set of For nodes Its neighboring nodes The attention coefficient; Then use the weight parameter matrix to perform a linear transformation on the node features to obtain a scalar representing the node relevance. The formula is: ; In the formula, || is the concatenation operation of two vectors, is a learning parameter, LeakyReLU is a nonlinear activation function; Then normalize it through the softmax function to finally get the weight coefficient of the neighbor node information, the formula is: ; Where, For nodes With neighboring nodes 's relevance.

[0009] In a preferred embodiment, the method for constructing the time-oriented GAT layer includes: Treat all dimension data at each timestamp within the time window as a node in the graph; The graph attention mechanism is used to calculate the node dependencies in the time dimension, and the node features are updated by aggregating the information of neighboring nodes. The relationship between nodes represents the time dependency.

[0010] In the preferred embodiment, the LSTM encoding layer uses LSTM to model the time information in the sequence and outputs the implicit representation of the data, which is expressed as follows: ; ; Where LSTM is an LSTM unit, for The implicit state at the moment is given by Implicit state at all times Time input Joint decision-making; is the implicit state of the entire input data, which is formed by concatenating the implicit state at each moment along the time dimension; concat is the concatenation function, is a fully connected layer.

[0011] In the preferred solution, the construction of the S1 multi-scale integrated decoder includes decoding of the time series, specifically: Define multiple decoders of different lengths. Assume that the decoder set is , where decoders The output length of , the original sequence length is , The definition formula is: ; ; Where, is the decoder length coefficient, which is determined by the parameter definition, ; LSTM is used to decode the hidden layer. The decoding process formula is: ; ; Where, is the implicit variable at time t, Initialized to zero, For the The output of a decoder at time t is and are all learnable parameters; Depend on and Joint decision, It is artificially introduced noise that conforms to the normal distribution.

[0012] In the preferred solution, S2 constructs a multi-scale integrated decoder, using the output of the coarse-grained decoder to assist the fine-grained decoder in decoding, and the coarse and fine granularity are integrated to form an integrated output, specifically: Using a fully connected layer The output of the first decoder is the same as the The decoder outputs are concatenated and a hyperparameter is introduced To control the balance between the two, the formula is: ; Where, is the implicit variable at time t, and Fusion, parameters is the preset value in the decoder length definition, It is a two-layer fully connected network. represents the weight, as shown in formula (11), Hidden layer variables at time Depend on and Joint decision-making; ; Where, Same as the encoding process, this is the artificially introduced noise value.

[0013] In the preferred solution, in S2, the MSED-GAT prediction model is trained. The model loss function includes the reconstruction error between the reconstructed data and the original data and the similarity error between the decoder output and the original data. The sum of the reconstruction error and the similarity error is used as the loss function and minimized. The two loss values are added to continuously update the model parameters to obtain the optimal MSED-GAT prediction model, which is specifically: The reconstruction error formula is: ; Where, is the mean square error, is the sequence length; The smoothed DTW method is used to evaluate the difference between the input sequence and multiple output sequences of unequal length. The formula is: ; Where, and For two sequences, is the set of alignment matrices corresponding to all paths, represents the inner product of two matrices, The distance matrix records the cost of the route, where ; The smoothed DTW function is introduced to calculate the similarity of sequences of unequal lengths. The formula is: ; Where, ; After obtaining the similarity constraints between the outputs of different decoders and the original input, the gradient descent method is used for training. The formula is: ; The loss function is calculated as: ; Where, is a hyperparameter representing the weight value of the temporal pattern similarity error.

[0014] In the preferred solution, S3 inputs the test set into the optimal MSED-GAT prediction model for anomaly detection, including: The first data set is divided into a validation set. After the validation set is used to reconstruct the data, the reconstruction error is fitted to a normal distribution, and the anomaly probability of the test data is calculated. The anomaly threshold is dynamically defined based on the product of the maximum anomaly score in the validation set and the hyperparameter, specifically: The normal distribution of the error set fitting is , the probability formula for judging whether the data is abnormal is: ; Where, is the total number of dimensions of the time series data, then the definition formula of the anomaly score is: ; The threshold uses the maximum value of the anomaly score in the validation set and the consistent hyperparameters The product of is: ; Where, is the threshold, is the maximum value of the anomaly score in the validation set, is the threshold scaling parameter.

[0015] In a preferred embodiment, the convolution kernel size of the one-dimensional convolution layer is determined according to the periodicity of the data set.

[0016] The present invention provides a multi-dimensional time series anomaly detection method based on a graph attention network, comprising the following steps: obtaining multi-dimensional time series data, performing data preprocessing, and then extracting features using a one-dimensional convolutional layer, inputting a constructed MSED-GAT prediction model, using two GAT layers to respectively capture the correlation relationship and time information between dimensions, and fusing the outputs of the two GAT layers with the output after the one-dimensional convolution, and finally encoding the fused data using an LSTM to capture the long-term dependency in the sequence, thereby obtaining an implicit feature representation of the entire input data; then defining multi-scale fusion integrated decoding of different lengths for decoding, adopting a loss function, performing model training, optimizing the model, and realizing the function of anomaly detection; by simultaneously modeling the correlation relationship and time dependency between variables through a feature-oriented GAT+a time-oriented GAT, feature extraction is more comprehensive, the error accumulation problem of traditional autoencoders is effectively solved, the reconstruction error rate is reduced, and the accuracy of anomaly detection is improved; the parallel computing of multiple decoders improves the consistency of multi-scale outputs, further improves the accuracy of data, increases the speed of feature extraction, and greatly improves the accuracy of fault warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below with reference to the accompanying drawings and examples: Figure 1 is a flow chart of the anomaly detection method of the present invention; Figure 2 Schematic diagram of the MSED-GAT prediction model structure of the present invention; Figure 3 This is a diagram of the processing process of one-dimensional convolution in time series data of the present invention; Figure 4 It is a graphical representation of multidimensional time series data of the present invention. DETAILED DESCRIPTION

[0018] Example 1 like Figure 1-4 As shown in FIG, a multi-dimensional time series anomaly detection method based on a graph attention network includes the following steps: S1: Obtain multidimensional time series data, perform data preprocessing, and save it as a first data set. Divide the first data set into a training set and a test set with a preset ratio, and then use a one-dimensional convolutional layer to extract features.

[0019] S2: Build an MSED-GAT (Multivariate Time Series Anomaly Detection Based on Graph Attention Network and Multi-scale Ensemble Decoding) prediction model, input the training set into the built MSED-GAT prediction model for model training, and save the optimal MSED-GAT prediction model.

[0020] The MSED-GAT prediction model includes a graph attention network encoder spliced with a multi-scale integrated decoder, wherein: the graph attention network encoder includes a one-dimensional convolutional layer module connected to the feature-oriented GAT (Graph Attention Network) layer and the time-oriented GAT layer respectively, and the outputs of the three are spliced along the feature direction and then connected to the LSTM (Long Short-Term Memory) encoding layer; the multi-scale integrated decoder includes multiple decoders of different lengths connected in sequence.

[0021] S3: The test set is input into the optimal MSED-GAT prediction model for anomaly detection. When it is greater than the preset anomaly threshold, the system is considered abnormal; otherwise, it is judged to be normal.

[0022] This embodiment obtains multidimensional time series data, performs data preprocessing, and then uses a one-dimensional convolution layer to extract features. The data is input into the established MSED-GAT prediction model, and two GAT layers are used to capture the correlation relationship and time information between dimensions respectively. The outputs of the two GAT layers are fused with the output after the one-dimensional convolution. Finally, LSTM is used to encode the fused data to capture the long-term dependency in the sequence, thereby obtaining the implicit feature representation of the entire input data; multi-scale fusion integrated decoding of different lengths is then defined for decoding, and a loss function is used to train and optimize the model to realize the function of anomaly detection; by simultaneously modeling the correlation relationship and time dependency between variables through feature-oriented GAT + time-oriented GAT, feature extraction is more comprehensive, effectively solving the error accumulation problem of traditional autoencoders, reducing the reconstruction error rate, and improving the accuracy of anomaly detection. The parallel computing of multiple decoders improves the consistency of multi-scale output, further improves the accuracy of data, increases the speed of feature extraction, and greatly improves the accuracy of fault warning.

[0023] like Figure 2 As shown in the figure, the overall framework of the method of this application can be divided into two parts: graph attention network encoder and multi-scale integrated encoder.

[0024] The main function of the graph attention network encoder is to obtain an effective implicit representation of the input data by deeply mining the data features. The main processing steps are as follows: 1) The data is first preprocessed and then input into the one-dimensional convolution layer, which uses the lightweight and fast computational characteristics of one-dimensional convolution to extract local high-level features of the time series.

[0025] 2) The output after one-dimensional convolution is processed as the input of two GAT layers. The time-oriented GAT layer extracts the temporal information in the sequence, and the feature-oriented GAT layer extracts the relationship between dimensions in the sequence.

[0026] 3) Merge the output of the one-dimensional convolution and the output of the GAT layer, and then use LSTM to capture the sequence features to obtain the implicit representation of the entire input data.

[0027] The main function of the multi-scale ensemble encoder is to reconstruct the input data using the implicit representation obtained by the encoder. The main processing steps are as follows: 1) Define multiple decoders of different lengths to capture information at different scales of the time series. Specifically, decoders with shorter decoding lengths focus more on the macroscopic characteristics of the sequence, while decoders with longer decoding lengths can capture local temporal patterns.

[0028] 2) The outputs of different decoders are fused, using the outputs of decoders with shorter decoding lengths to assist decoders with longer decoding lengths. The final decoder layer has the longest output length, and its output serves as the output of the entire multi-scale ensemble decoder, completing the reconstruction of the input data.

[0029] Each step in the method of this embodiment is described in detail. First, local features are extracted from the acquired time series data.

[0030] In the preferred embodiment, data preprocessing includes: S11: Data cleaning and anomaly filtering for multidimensional time series data: adopt the method of multivariate joint outlier detection: that is, calculate the preset quantile of the sliding window for each dimension, eliminate data points that exceed the threshold range, and then use the robust Mahalanobis distance to identify multivariate joint outliers for dimension-related anomalies.

[0031] S12: Perform normalization processing, the formula is: ; Where, is the maximum value, is the minimum value, For input data.

[0032] In this embodiment, outlier detection is based on moving quantiles: the 25% to 75% quantiles of the sliding window (1 hour) are calculated for each dimension, and data points exceeding [Q1-3IQR, Q3+3IQR] are eliminated; then multivariate detection is combined (for dimension-related anomalies).

[0033] Since the data range of each dimension in the multidimensional time series is inconsistent, in order to obtain the same measurement scale without changing the characteristics of the data itself, data normalization is chosen.

[0034] like Figure 3 As shown in the figure, after the data is normalized, one-dimensional convolution is used to extract local high-level features of the input data. This is because convolution operations are good at local feature engineering within the sliding window. Then, combined with LSTM for learning, it is effective in processing long time series data. Figure 3 The one-dimensional convolution process of a single convolution kernel on time series data is shown. The convolution kernel slides along the time axis. Every time it moves to a fixed position, the result is the multiplication of the convolution kernel and the value of the corresponding position of the input data and then the summation.

[0035] In a preferred embodiment, the convolution kernel size of the one-dimensional convolution layer is determined according to the periodicity of the data set.

[0036] The definition of the convolution kernel size needs to be combined with the actual data set situation, focusing on the local time characteristics of the sequence. Therefore, the convolution kernel should not be too large. For data with a certain periodicity, the period value or other related values are taken. For data without obvious periodicity, the actual experimental results shall prevail.

[0037] A graph attention network encoder is constructed, including: a feature-oriented GAT layer that captures the relationship between dimensions of multi-dimensional time series, a time-oriented GAT layer that captures temporal dependencies, and an encoding layer for fusion information.

[0038] Since the dimensions of multidimensional time series are interconnected, changes in data in one dimension may cause the data in the associated dimensions to change at the same time. This association is very similar to the relationship between a node and its neighboring nodes in graph data. Therefore, the multidimensional time series can be regarded as a graph, and the data of a single dimension is a node in the graph. The graph neural network is used to model the relationship between different dimensions in the multidimensional time series, and the graph attention mechanism is introduced to perform aggregation operations on neighboring nodes to improve the expression and learning ability of the graph neural network model.

[0039] like Figure 4 As shown in the figure, this is the representation of multidimensional time series data after being converted into graph data. The nodes in the graph represent the time series in each dimension, and the edges between nodes represent the relationship between the two dimensions. Specifically, since it is complicated to represent the data of the entire time period in a dimension as nodes, time windows are used to segment the data.

[0040] In the preferred embodiment, the method for constructing the feature-oriented GAT layer includes: Each dimension of the multidimensional time series is regarded as a node in the graph. The node feature is the time series data on that dimension. The graph attention mechanism is used to adaptively calculate the attention weight of the node and its neighboring nodes, and the node feature is updated by aggregating the neighboring node information. Specifically: We use graph neural networks to model the relationship between different dimensions in multidimensional time series, introduce graph attention mechanisms, perform aggregation operations on neighboring nodes, and use time windows to segment data. The time series on each dimension is represented by a node. The existence of edges between nodes indicates that there is a correlation between the two dimensions. First, the size of the time window is cut off. ,enter express, Represents the sequence data of each dimension, that is, the feature vector of each node, is the total number of dimensions of the multidimensional time series. For example, the new node feature is The features of the node itself and the features of all neighboring nodes are aggregated, and different neighboring nodes have different weight coefficients.

[0041] The node aggregation operation formula is: ; Where, For nodes The new feature vector after the GAT layer, is the activation function, For nodes The neighbor set of For nodes Its neighboring nodes The attention coefficient.

[0042] In the process of node aggregation, the graph attention mechanism can be used to adaptively assign weights to all neighboring nodes of each node. Specifically, the node features are linearly transformed using the weight parameter matrix to obtain a scalar representing the node relevance, as follows: ; In the formula, || is the concatenation operation of two vectors, is a learning parameter, and LeakyReLU is a nonlinear activation function.

[0043] Then normalize it through the softmax function to finally get the weight coefficient of the neighbor node information, the formula is: .

[0044] Then, we construct a time-oriented GAT layer. This embodiment uses a time-oriented GAT to capture the hidden temporal dependencies in the sequence. Specifically, all time series data within the time window are regarded as a complete graph. Unlike the feature-oriented GAT, the data of all dimensions on a single timestamp are regarded as a node of the graph. The relationship between nodes represents the temporal dependencies. Assuming the time window size is , the input data can be expressed as , is the total number of dimensions of the multidimensional time series. Consequently, the input of the time-oriented GAT is exactly the transpose of the input of the feature-oriented GAT, because the two GAT layers examine the same data from the feature dimension and the time dimension respectively.

[0045] In the preferred solution, the method for constructing the time-oriented GAT layer includes: Treat all dimension data at each timestamp within the time window as a node in the graph; The graph attention mechanism is used to calculate the node dependencies in the time dimension, and the node features are updated by aggregating the information of neighboring nodes. The relationship between nodes represents the time dependency.

[0046] The calculation process of the attention coefficient and the node aggregation process are similar to those of the feature-oriented GAT layer above, so we will not repeat them here. Similarly, the data size does not change after the input passes through the time-oriented GAT layer, but since this layer is a graph constructed from the time dimension, the output data size is , it needs to be transposed and then fused with the feature-oriented GAT output and the output of the convolutional layer, and finally the LSTM encoder is used to learn the implicit representation of the sequence.

[0047] After feature-oriented GAT, we can get sequence , represents the association relationship between the original sequence dimensions, where is the total number of dimensions of the multidimensional time series, is the time window size; after the time-oriented GAT, the size can be obtained as sequence , represents the time information of the original sequence. After transposition and And the output of the one-dimensional convolution layer By splicing along the feature direction, we can get a size of sequence . It integrates the original sequence itself and the correlation between dimensions in the sequence, as well as time information, enriches the data features, and is more conducive to obtaining effective implicit representations.

[0048] Effective implicit representation of the original sequence is the key to anomaly detection in multidimensional time series. LSTM, based on the RNN (Recurrent Neural Network), adds filtering of past states and can select which states are more influential in the current state.

[0049] In the preferred solution, the LSTM encoding layer uses LSTM to model the time information in the sequence and output the implicit representation of the data. The formula is: ; ; Where LSTM is an LSTM unit, for The implicit state at the moment is given by Implicit state at all times Time input Joint decision-making; is the implicit state of the entire input data, which is formed by concatenating the implicit state at each moment along the time dimension; concat is the concatenation function, is the fully connected layer.

[0050] Assume that the dimension of the hidden layer is , after LSTM encoding, the output of the hidden layer can be obtained The implicit representation is a high-level abstraction of the temporal information and the relationship between dimensions in the original data, which will be used by the decoder to reconstruct the input data.

[0051] In the decoding stage of this embodiment, LSTM is used to decode the hidden layer. Due to the sequential decoding characteristics of the autoencoder, errors are easily accumulated. To alleviate this problem, decoders of different lengths are defined.

[0052] In the preferred solution, the construction of the S1 multi-scale integrated decoder includes decoding of the time series, specifically: Define multiple decoders of different lengths. Assume that the decoder set is , where decoders The output length of , the original sequence length is , The definition formula is: ; ; Where, is the decoder length coefficient, which is determined by the parameter definition( ), especially , which ensures that the maximum decoder output length is According to the actual application, the recurrent neural network can be adaptively selected in the time series decoding process.

[0053] LSTM is used to decode the hidden layer. The decoding process formula is: ; ; Where, is the implicit variable at time t, Initialized to zero, For the The output of a decoder at time t is and are all learnable parameters; Depend on and Joint decision, The noise that conforms to the normal distribution is artificially introduced to avoid overfitting of the model.

[0054] The output lengths of different decoders are different, so it is not possible to simply add up the outputs of each decoder and take the average value. Therefore, a suitable fusion strategy is needed. According to the relevant formula, , two decoders and middle, It can better capture the macro characteristics of the sequence and obtain coarse-grained information; It captures the local characteristics of the sequence and obtains more fine-grained information.

[0055] In the preferred solution, S2 constructs a multi-scale integrated decoder, using the output of the coarse-grained decoder to assist the fine-grained decoder in decoding, and the coarse and fine granularity are integrated to form an integrated output, specifically: Using a fully connected layer The output of the first decoder is the same as the The decoder outputs are concatenated and a hyperparameter is introduced To control the balance between the two, the formula is: ; Where, is the implicit variable at time t, and Fusion, parameters is the preset value in the decoder length definition, It is a two-layer fully connected network. represents the weight, as shown in formula (11), Hidden layer variables at time Depend on and Joint decision-making; ; In the formula Same as the encoding process, both are artificially introduced noise values.

[0056] Combining the formula of the time series decoding process, we can get the reconstructed sequences of different scales. Among them, the highest scale decoder The output of is a reconstruction of the input sequence.

[0057] In this embodiment, the loss of the entire model is given by two aspects: the error between the reconstructed data and the original data, and the similarity constraint between the decoder output and the original data. The objective function below is the loss function.

[0058] The output of the autoencoder-based anomaly detection model is a reconstruction of the input data. The reconstruction error is the difference between the reconstructed data and the real data, which can indicate how well the model reconstructs the input data. Expressed as , the original input sequence It can be expressed as .

[0059] In the preferred solution, in step S2, the MSED-GAT prediction model is trained. The model loss function includes the reconstruction error between the reconstructed data and the original data and the similarity error between the decoder output and the original data. The loss function is used to calculate the sum of the reconstruction error and the similarity error and minimize it. The two loss values are added together to continuously update the model parameters to obtain the optimal MSED-GAT prediction model, which is specifically: The reconstruction error formula is: ; Where, is the mean square error, is the sequence length.

[0060] The output sequences of multiple decoders in ensemble decoding have different lengths. Therefore, it is necessary to introduce similarity constraints between the outputs of different decoders and the original input sequences to ensure the similarity of the temporal patterns learned by decoders of different lengths.

[0061] This example uses the smooth DTW (Dynamic Time Warping) method to evaluate the difference between the input sequence and multiple output sequences of unequal length. The goal of model training is to minimize the difference. Suppose there are two sequences and , the lengths are and , introduce a matrix Indicates the alignment of two sequences. and Alignment, , otherwise it is zero, and the matrix is initialized to as well as , which means that the endpoints of the two sequences are always aligned. Another distance matrix Record the cost on the route, where Valid paths must be monotonic over time, so they can only start from the upper left corner and move in the right, down, or lower-right direction until they reach the lower-right end point. The DTW algorithm, on the other hand, requires selecting the path with the lowest cost among all possible paths. Because the cost of a path (the current state) is determined by the cost of the previous step (the state), and the set of paths is finite, dynamic programming is often used in practice.

[0062] The formula of the smoothed DTW method is: ; Where, and For two sequences, is the set of alignment matrices corresponding to all paths, is the inner product of two matrices.

[0063] because The function is a non-differentiable operation and cannot be directly introduced into the model training. The smoothed DTW function is introduced to calculate the similarity of sequences of unequal lengths. The formula is: ; Where, ; According to the limit principle, when When it approaches 0 infinitely, this formula is equivalent to directly using function.

[0064] Based on the smoothed DTW distance, we can obtain the similarity constraints between the outputs of different decoders and the original input, and use the gradient descent method for training. The formula is: ; The formula for the sum of reconstruction error and similarity error is: ; Where, is a hyperparameter representing the weight value of the temporal pattern similarity error.

[0065] In the anomaly detection of this embodiment, the distribution of anomaly scores is approximately in line with the normal distribution, that is, the majority of the test data are normal data, and the outliers only account for a small part of the entire set. After completing the reconstruction of the data in the validation set, the error set is fitted into a normal distribution .

[0066] In the preferred solution, S3 inputs the validation set into the optimal MSED-GAT prediction model for anomaly detection, including: The first data set is divided into a validation set. After the validation set is used to reconstruct the data, the reconstruction error is fitted to a normal distribution, and the anomaly probability of the test data is calculated. The anomaly threshold is dynamically defined based on the product of the maximum anomaly score in the validation set and the hyperparameter, specifically: The normal distribution of the error set fitting is , the probability formula for judging whether the data is abnormal is: ; Where, is the total number of dimensions of the time series data, then the definition formula of the anomaly score is: .

[0067] Use the maximum value of the validation set anomaly score and consistent hyperparameters The product of is: ; Where, is the threshold, is the maximum value of the anomaly score in the validation set, is the threshold scaling parameter.

[0068] In this embodiment, the information obtained during the training phase completes the automatic definition of the threshold.

[0069] In order to verify the effectiveness of the method in this embodiment, experiments were conducted on five real data sets and one artificially generated data set. The results of the real data sets are shown in Table 1.

[0070] Table 1 Real dataset

[0071] In the comparative experiment, a total of 6 comparison methods were selected, as shown in Table 2. They were divided into three aspects: the classification of the method, the definition of the anomaly score, and the definition of the threshold, which intuitively shows the basic situation of each comparison method.

[0072] Table 2 Basic information of comparison methods

[0073] Table 3 Experimental results of F1-score indicator

[0074] Table 4 Experimental results of precision index

[0075] Table 5 Experimental results of recall rate index

[0076] Table 6 Comparison of MSED-GAT experimental results and other comparison methods

[0077] Among them, Win is the number of times the evaluation index in all data sets is higher than that of the comparison method; Lose is the number of times the evaluation index in all data sets is lower than that of the comparison method.

[0078] By observing and analyzing the data in the table, we can know that: (1) From the comparison results of the table data, it can be seen that the method of this embodiment achieves better results than the comparison method in most cases, and is effective in detecting anomalies in time series data.

[0079] (2) From the perspective of method classification, graph neural network-based anomaly detection algorithms such as MTAD-GAT and MSED-GAT can achieve good results on most data sets, indicating that modeling the relationship between dimensions through graph neural networks is effective in assisting anomaly detection.

[0080] (3) The WADI dataset not only has the longest time series length but also contains the most data dimensions. Most methods do not perform well on this dataset. The MSED-GAT method in this embodiment performs best, which proves that this method is relatively less affected by the size of the test dataset and improves generalization ability while ensuring model accuracy.

[0081] In use, this embodiment uses a graph attention network to model inter-dimensional relationships and adaptively assign weights to different neighboring nodes, providing better representation of nodes and improving the anomaly detection algorithm. Multi-scale ensemble decoding, by combining multiple decoders, fully utilizes the multi-scale information of time series, improving both model accuracy and generalization.

[0082] The above embodiments are merely preferred technical solutions of the present invention and should not be construed as limiting the present invention. The scope of protection of the present invention shall be the technical solutions set forth in the claims, including equivalent alternatives to the technical features of the technical solutions set forth in the claims. In other words, equivalent alternatives and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A multi-dimensional time series anomaly detection method based on graph attention network, characterized by: The following steps are involved: S1: Obtain multidimensional time series data, perform data preprocessing, and save it as a first data set. The first data set is divided into a training set and a test set with a preset ratio, and then features are extracted using a one-dimensional convolutional layer. S2: Build the MSED-GAT prediction model, input the training set into the built MSED-GAT prediction model for model training, and save the optimal MSED-GAT prediction model; The MSED-GAT prediction model consists of a graph attention network encoder spliced with a multi-scale ensemble decoder. The graph attention network encoder consists of a one-dimensional convolutional layer module connected to a feature-oriented GAT layer and a time-oriented GAT layer. The outputs of the three are concatenated along the feature direction and then connected to an LSTM encoding layer. The multi-scale ensemble decoder consists of multiple decoders of different lengths connected in sequence. S3: The test set is input into the optimal MSED-GAT prediction model for anomaly detection. When it is greater than the preset anomaly threshold, the system is considered abnormal; otherwise, it is judged to be normal.

2. The multidimensional time series anomaly detection method based on graph attention network according to claim 1, characterized in that: Data preprocessing includes: S11: Data cleaning and anomaly filtering for multidimensional time series data: using a multivariate joint outlier detection method: calculating the preset quantile of the sliding window for each dimension, removing data points that exceed the threshold range, and then using the robust Mahalanobis distance to identify multivariate joint outliers for dimension-related anomalies; S12: Perform normalization processing, the formula is: ； Where, is the maximum value, is the minimum value, For input data.

3. The multidimensional time series anomaly detection method based on graph attention network according to claim 1 is characterized in that The method for constructing the feature-oriented GAT layer includes: Each dimension of the multidimensional time series is regarded as a node in the graph. The node feature is the time series data on that dimension. The graph attention mechanism is used to adaptively calculate the attention weight of the node and its neighboring nodes, and the node feature is updated by aggregating the neighboring node information. Specifically: Graph neural networks are used to model the relationship between different dimensions in multidimensional time series. Graph attention mechanisms are introduced to perform aggregation operations on neighboring nodes. Time windows are used to segment data. The time series in each dimension is represented by a node. The presence of an edge between nodes indicates a correlation between the two dimensions. The node aggregation operation formula is: ； Where, For nodes The new feature vector after the GAT layer, is the activation function, For nodes The neighbor set of For nodes Its neighboring nodes The attention coefficient; Then use the weight parameter matrix to perform a linear transformation on the node features to obtain a scalar representing the node relevance. The formula is: ； In the formula, || is the concatenation operation of two vectors, is a learning parameter, LeakyReLU is a nonlinear activation function; Then normalize it through the softmax function to finally get the weight coefficient of the neighbor node information, the formula is: ； Where, For nodes With neighboring nodes 's relevance.

4. The multidimensional time series anomaly detection method based on graph attention network according to claim 1, characterized in that: The method for constructing the time-oriented GAT layer includes: Treat all dimension data at each timestamp within the time window as a node in the graph; The graph attention mechanism is used to calculate the node dependencies in the time dimension, and the node features are updated by aggregating the information of neighboring nodes. The relationship between nodes represents the time dependency.

5. The multi-dimensional time series anomaly detection method based on graph attention network according to claim 4 is characterized in that The LSTM encoding layer uses LSTM to model the time information in the sequence and outputs the implicit representation of the data. The formula is: ； ； Where LSTM is an LSTM unit, for The implicit state at the moment is given by Implicit state at all times Time input Joint decision-making; is the implicit state of the entire input data, which is formed by concatenating the implicit state at each moment along the time dimension; concat is the concatenation function, is the fully connected layer.

6. The multidimensional time series anomaly detection method based on graph attention network according to claim 1 is characterized in that The construction of the S1 multi-scale integrated decoder includes time series decoding, specifically: Define multiple decoders of different lengths. Assume that the decoder set is , where decoders The output length of , the original sequence length is , The definition formula is: ； ； Where, is the decoder length coefficient, which is determined by the parameter definition, ; LSTM is used to decode the hidden layer. The decoding process formula is: ； ； Where, is the implicit variable at time t, Initialized to zero, For the The output of a decoder at time t is and are all learnable parameters; Depend on and Joint decision, It is artificially introduced noise that conforms to the normal distribution.

7. The multidimensional time series anomaly detection method based on graph attention network according to claim 6 is characterized in that S2 builds a multi-scale integrated decoder, using the output of the coarse-grained decoder to assist the fine-grained decoder for decoding. The coarse and fine granularity are fused to form an integrated output, specifically: Using a fully connected layer The output of the first decoder is the same as the The decoder outputs are concatenated and a hyperparameter is introduced To control the balance between the two, the formula is: ； Where, is the implicit variable at time t, and Fusion, parameters is the preset value in the decoder length definition, It is a two-layer fully connected network. represents the weight, as shown in formula (11), Hidden layer variables at time Depend on and Joint decision-making; ； Where, Same as the encoding process, this is the artificially introduced noise value.

8. The multi-dimensional time series anomaly detection method based on graph attention network according to claim 1, characterized in that: In S2, the MSED-GAT prediction model is trained. The model loss function includes the reconstruction error between the reconstructed data and the original data, and the similarity error between the decoder output and the original data. The sum of the reconstruction error and the similarity error is used as the loss function and minimized. The two loss values are added together to continuously update the model parameters to obtain the optimal MSED-GAT prediction model, which is specifically: The reconstruction error formula is: ； Where, is the mean square error, is the sequence length; The smoothed DTW method is used to evaluate the difference between the input sequence and multiple output sequences of unequal length. The formula is: ； Where, and For two sequences, is the set of alignment matrices corresponding to all paths, represents the inner product of two matrices, The distance matrix records the cost of the route, where ; The smoothed DTW function is introduced to calculate the similarity of sequences of unequal lengths. The formula is: ； Where, ; After obtaining the similarity constraints between the outputs of different decoders and the original input, the gradient descent method is used for training. The formula is: ； The loss function is calculated as: ； Where, is a hyperparameter representing the weight value of the temporal pattern similarity error.

9. The multidimensional time series anomaly detection method based on graph attention network according to claim 1 is characterized in that In S3, the test set is input into the optimal MSED-GAT prediction model for anomaly detection, including: The first data set is divided into a validation set. After the validation set is used to reconstruct the data, the reconstruction error is fitted to a normal distribution, and the anomaly probability of the test data is calculated. The anomaly threshold is dynamically defined based on the product of the maximum anomaly score in the validation set and the hyperparameter, specifically: The normal distribution of the error set fitting is , the probability formula for judging whether the data is abnormal is: ； Where, is the total number of dimensions of the time series data, then the definition formula of the anomaly score is: ； The threshold uses the maximum value of the anomaly score in the validation set and the consistent hyperparameters The product of is: ； Where, is the threshold, is the maximum value of the anomaly score in the validation set, is the threshold scaling parameter.

10. The multi-dimensional time series anomaly detection method based on graph attention network according to claim 1 is characterized in that The convolution kernel size of the one-dimensional convolution layer is determined according to the periodicity of the data set.

Images