A time series anomaly detection method based on neighborhood information fusion attention mechanism

Abstract

The application discloses a time sequence anomaly detection method based on neighborhood information fusion attention mechanism, and comprises the following steps: step 1, pre-processing time sequence data in a data set; step 2, building a neural network model based on sequence decomposition and attention mechanism, and completing feature extraction and reconstruction of training data through the network; step 3, unsupervised pre-training the network by using a deep learning algorithm; and step 4, using the finally obtained model to perform anomaly detection. The method provided by the application can significantly improve the anomaly detection effect of the model in an unsupervised scene by combining sequence decomposition and attention mechanism, fusing neighborhood information and strengthening the modeling capability of the attention module to the edge part of the input window data.

Description

Technical Field

[0001] This invention relates to a method for detecting temporal anomalies, and more particularly to a method for detecting temporal anomalies based on a neighborhood information fusion attention mechanism. Background Technology

[0002] Time series anomaly detection is a classic time series analysis method with significant implications for various fields. It learns patterns from historical time series data and monitors real-time data, triggering alarms when patterns abruptly change. This task can be applied in areas such as intelligent operation and maintenance, and industrial automation. For example, it can enable real-time monitoring of network devices and automated alarms when devices malfunction; it can also monitor indicators of large-scale industrial equipment, automatically triggering alarms and assisting technicians in troubleshooting when equipment anomalies occur.

[0003] Traditional time series anomaly detection methods assume that the data follows a certain distribution, and data outside this distribution is considered anomaly. However, due to the dynamic nature of time series data, its distribution shifts over time, rendering traditional time series anomaly detection methods ineffective for this task. Summary of the Invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a temporal anomaly detection method based on a neighborhood information fusion attention mechanism, which addresses the shortcomings of the existing technology.

[0005] To address the aforementioned technical problems, this invention discloses a temporal anomaly detection method based on a neighborhood information fusion attention mechanism, comprising the following steps:

[0006] Step 1: Preprocess the text data containing time series data in the dataset, including:

[0007] First, normalize each time series in the dataset to shrink all data to the same range. Second, window the series, with each window having a length of T and a window sliding step of T, that is, divide the time series into non-overlapping sub-windows.

[0008] Step 2: Build a neural network model based on sequence decomposition and attention mechanism, namely an anomaly detection model. The anomaly detection model uses a neural network to extract multi-level features of the input data and reconstructs the input data using the extracted multi-level features to obtain the reconstruction result of the input data.

[0009] The aforementioned construction of the anomaly detection model includes:

[0010] Step 2-1: Construct the sequence decomposition attention module, which consists of three parts: a sequence decomposition layer, an anomaly attention layer, and a fully connected layer, specifically including:

[0011] Step 2-1-1, Construct the sequence decomposition layer:

[0012] The preprocessed data in step 1 is decomposed into sequences, including information fusion and the generation of two components: a trend component and a seasonal component.

[0013] Information fusion involves concatenating the data of the current window (the original window) with the data from the preceding and following windows.

[0014]

[0015] in, It is the spliced ​​data. This refers to the data of the current window, i.e., the original window. It's data from the previous window. This refers to the data in the next window; Concat{·} is the concatenation operation; the window in question is the window obtained during the preprocessing in step 1.

[0016] After the data is stitched together, the trend component is extracted from the stitched data using average pooling. It is expressed as follows:

[0017]

[0018] AvgPool(·) is the average pooling operation with a step size of 3;

[0019] Finally, the seasonal component is obtained by subtracting the trend component from the original window data. It is expressed as follows:

[0020]

[0021] Step 2-1-2, Build the anomaly attention layer:

[0022] The attention mechanism is calculated using the weight matrix. σ matrix:

[0023]

[0024] in, These are the matrix components in the attention mechanism, where σ is the learnable Gaussian kernel parameter. It is the input data of the model. These are the query weight matrix, key-value weight matrix, and attention weight matrix in the attention mechanism, W. σ It is the Gaussian kernel parameter weight matrix;

[0025] The seasonal component extracted in step 2-1-1 is used as input to model prior association and sequence association respectively, which are used to define the loss function in step 3 and the anomaly score in step 4.

[0026] The aforementioned modeling prior associations specifically include:

[0027] Prior association uses a learnable Gaussian kernel to calculate prior information about relative temporal distance. The specific modeling method is as follows:

[0028]

[0029] in, Represents prior association; Rescale(·) refers to scaling the data based on the Gaussian kernel; L represents the length of the input data window; i,j represent the indices of any two data points on the same data line; σ i Let represent the Gaussian kernel parameter of the i-th data, and exp() represent the exponential function with the natural constant e as the base.

[0030] The aforementioned modeling sequence association is based on an attention mechanism for extraction, and the modeling method is as follows:

[0031]

[0032] in, Indicates sequence association. This indicates the transpose operation. d represents the key-value matrix in the attention mechanism. model The hyperparameter represents the dimension of the model, and Softmax() represents the softmax function.

[0033] Global attention weights are extracted from the input data through sequence association, and attention components are obtained.

[0034]

[0035] Matrix components storing attention weights It is the first output of the sequence decomposition attention module.

[0036] Step 2-1-3, build the fully connected layer:

[0037] The fully connected layer is used to extract the trend component obtained in step 2-1-1. It consists of a three-layer fully connected neural network and a ReLU function, as shown below:

[0038]

[0039] Here, FeedForward() represents the fully connected layer, and S1 is the output of the fully connected layer, which is the second output of the sequence decomposition attention module.

[0040] Step 2-2: Build an anomaly detection model based on the sequence decomposition attention module, including a two-layer sequence decomposition attention module, a residual connection module, and a reconstruction module, specifically including:

[0041] Two sequence decomposition attention modules are stacked. The input to the first sequence decomposition attention module is the concatenated data in step 2-1-1. The output of the anomaly attention layer in the first sequence decomposition attention module is edge-padded and used as the input to the second sequence decomposition attention module. In addition, the original window data is input into a separate anomaly attention layer as a residual connection module. Finally, the outputs of the above three modules are concatenated by the reconstruction module and input into the fully connected layer as the output layer of the anomaly detection model.

[0042] The aforementioned edge padding involves padding the data with 0 values ​​to a length of three times.

[0043] Step 3, train the anomaly detection model described in Step 2, including: defining the loss function and neural network hyperparameters, iteratively optimizing the parameters in the anomaly detection model based on an unsupervised reconstruction task, and obtaining the trained anomaly detection model;

[0044] The aforementioned training anomaly detection model specifically includes:

[0045] Step 3-1: Design the loss function, including reconstruction loss and attention correlation difference loss. The specific loss function is as follows:

[0046]

[0047] in, Represents the model reconstruction result, |||| 2 Let λ represent the L2 norm of the vector, λ be a hyperparameter, and AssDis(·) be the correlation difference. The specific calculation formula is as follows:

[0048]

[0049] Among them, KL(·||·) is composed of and The KL divergence is calculated from the two discrete distributions corresponding to each row, where L represents the length of the output data window.

[0050] Step 3-2, Define the experimental configuration: Before training the model, first define the hyperparameters and the model optimizer used to update the model parameters, and then set the learning rate scheduler and the maximum number of iterations or related early stopping strategies;

[0051] Step 3-3: Perform unsupervised training on the anomaly detection model, reconstruct the data in the dataset, and update the gradient based on the loss function defined in Step 3-1 to obtain the trained anomaly detection model.

[0052] Step 4: Using the anomaly detection model trained in Step 3, process the time series data to be tested. Calculate the anomaly score based on the output of the anomaly detection model and determine whether the input time series data to be tested is anomaly. Specifically, this includes:

[0053] Step 4-1: Design the outlier score calculation method, which calculates the outlier score based on the association differences designed in Step 3-1, as follows:

[0054]

[0055] in, This represents outlier scores, Softmax() represents the Softmax function, and ⊙ represents the element-wise product of matrices. Denotes the L-2 norm;

[0056] Step 4-2: Use the anomaly detection model trained in Step 3 to process the data in the dataset, and use the anomaly score calculation method in Step 4-1 to calculate the anomaly score of each data in the dataset.

[0057] Step 4-3: Determine the anomaly detection threshold by using quantiles. If the anomaly score exceeds the threshold, the data is considered anomaly data.

[0058] Step 5: Apply the methods described in steps 1 to 4 to a dataset of a technical system containing time-series text data to perform anomaly detection on the time-series text data in the dataset.

[0059] Beneficial effects:

[0060] Compared with previous time series anomaly detection methods, the advantages of this invention are: by introducing sequence decomposition and anomaly attention mechanism, the model's modeling of window edge data is optimized without increasing model complexity. Furthermore, by continuously extracting trend terms with similar sequence associations and prior associations through the overlay module, the association differences of anomaly attention can better reflect the degree of anomaly of periodic anomalies, while enabling the model to extract deep semantic features. Attached Figure Description

[0061] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0062] Figure 1This is a flowchart of the present invention.

[0063] Figure 2 This is a diagram of the anomaly detection network structure.

[0064] Figure 3 This is a schematic diagram of sequence decomposition and comparison.

[0065] Figure 4 This is a schematic diagram illustrating the actual effect of the present invention on a server monitoring dataset. Detailed Implementation

[0066] Given the powerful feature extraction capabilities of neural networks, deep learning-based time series anomaly detection methods have gradually become mainstream. These methods can be further divided into anomaly detection methods based on prediction errors and anomaly detection methods based on reconstruction errors. The main idea of ​​anomaly detection methods based on prediction errors is to predict time series data using a time series prediction model, then calculate the deviation between the predicted and actual values. If the deviation exceeds a certain threshold, the sequence is considered anomaly. Anomaly detection based on reconstruction errors is based on autoencoders, which compress and encode the original data into a low-dimensional code using an encoder, and then use a decoder to decode and restore the original data. The goal of decoding is to completely reconstruct the original data.

[0067] like Figure 1 As shown, this invention discloses a time-series anomaly detection method based on a neighborhood information fusion attention mechanism, comprising the following steps:

[0068] Step 1: Preprocess the text data containing time series data in the dataset;

[0069] Step 2: Build a neural network model based on sequence decomposition and attention mechanism, use the network to extract multi-level features, and use the extracted features to reconstruct the input data to obtain the reconstruction result.

[0070] Step 3, train the model described in Step 2, including: defining the loss function and experimental settings, iteratively optimizing the model parameters in the network model based on the unsupervised reconstruction task, and obtaining the anomaly detection model;

[0071] Step 4: Use the anomaly detection model trained in Step 3 to process the time series data to be tested, calculate the anomaly score based on the model output, and determine whether it is an anomaly.

[0072] Step 1 specifically includes the following steps:

[0073] Normalize each time series in the dataset, scaling each time series to a minimum value of 0 and a maximum value of 1:

[0074]

[0075] Secondly, the sequence is windowed, with each window having a length of T and a window sliding step of T. That is, each time series is divided into k non-overlapping sub-windows of length T, which facilitates the subsequent model to extract time-dependent features and perform anomaly detection.

[0076] Step 2 specifically includes the following steps:

[0077] Step 2-1: Build the sequence decomposition attention module, which consists of three parts: a sequence decomposition layer, an anomaly attention layer, and a fully connected layer;

[0078] Step 2-2: Build a complete anomaly detection network, such as... Figure 2 As shown, it includes a two-layer sequence decomposition attention module, a residual connection module, and a reconstruction module;

[0079] Step 2-1 specifically includes the following steps:

[0080] Step 2-1-1: The sequence decomposition layer decomposes the original data to fuse more information and generate two components, such as... Figure 3 As shown. First, the original window data is concatenated with the data from the preceding and following windows:

[0081]

[0082] in, It is the input window data. It's data from the previous window. It contains the data from the next window; Concat{·} is the concatenation operation.

[0083] After the data is stitched together, average pooling is used to merge the data from the two windows into the original data, and trend components are extracted.

[0084]

[0085] Here, AvgPool(·) refers to the average pooling operation with a step size of 3. These are the trend components after sequence decomposition, which can reflect the long-term development trend or state of the sequence.

[0086] Finally, the seasonal component is obtained by subtracting it from the original window:

[0087]

[0088] That is, the seasonal component, which reflects the part of the sequence that changes rapidly in the short term;

[0089] Step 2-1-2: First, a suitable attention module is selected (using Anomaly Transformer, an anomaly attention module; reference: XU J, WU H, WANG J, et al. Anomaly transformer: Time series anomaly detection with association discrepancy[J]. ArXiv preprint arXiv:2110.02642,2021.). The anomaly attention layer takes the seasonal component extracted in step 2-1-1 as input and models prior association and sequence association respectively. First, the attention mechanism is calculated using the weight matrix (reference: Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).). σ:

[0090]

[0091] in, σ is the matrix component in the Transformer architecture, and σ is the learnable Gaussian kernel parameter.

[0092] First, a prior association is modeled. A learnable Gaussian kernel is used to calculate prior information about relative temporal distances. This design leverages the unimodal nature of the Gaussian kernel to focus attention on adjacent regions. A learnable scaling parameter is used in the Gaussian kernel to adapt the prior association to various time series patterns, such as different lengths of outlier segments. The specific modeling method is as follows:

[0093]

[0094] Rescale(·) refers to scaling the data based on the Gaussian kernel, generating a prior unimodal weight for each data point according to its Gaussian kernel parameters, with closer data points having larger weights. Prior association can force the model to focus its attention on the neighborhood of data points.

[0095] Then, sequence association is modeled, which is extracted based on an attention mechanism. The modeling method is as follows:

[0096]

[0097] Global attention weights are extracted from the input data through sequence association, and attention components are obtained.

[0098]

[0099] The attention component is one of the outputs of the sequence decomposition-attention module and is fed into the next layer of the network;

[0100] Step 2-1-3: The fully connected layer is mainly used to extract the trend components obtained through sequence decomposition in step 2-1-1. The fully connected layer consists of a three-layer fully connected neural network and a ReLU function.

[0101]

[0102] The output S1 of the fully connected layer is used as the second output of the sequence decomposition attention module and fed into the next layer of the network;

[0103] Step 2-2 specifically includes the following steps:

[0104] Build an anomaly detection network, such as Figure 2 As shown, after obtaining the sequence decomposition attention module in step 2-1, it is stacked in two layers. The input of the first layer is the concatenation result of the original window sequence data and the data of the two preceding and following windows. The output of the attention layer in the first layer is filled and used as the input of the second layer. In addition, the original window data is also input into a separate attention layer as a residual connection module. Finally, the outputs of these three modules are concatenated and input into a fully connected layer as the output layer of the model.

[0105] Step 3 specifically includes the following steps:

[0106] Step 3-1: Design the loss function for the reconstruction task, including reconstruction loss and attention association difference loss. Specifically, the loss function consists of two parts: reconstruction loss and attention association difference loss. The reconstruction loss reflects the model's reconstruction effect on the sequence and can guide the model to find the most informative attention weights. The attention association difference loss guides the distribution of sequence associations to be as different as possible from the prior association distribution, strengthening the model's ability to identify outliers. The specific loss function is as follows:

[0107]

[0108] Where AssDis(·) refers to the association difference, which represents the information gain between prior association and sequence association. It can be used to quantify the difference between two distributions, and the specific calculation formula is as follows:

[0109]

[0110] Among them, KL(·||·) is composed of and The KL divergence is calculated from the two discrete distributions corresponding to each row.

[0111] Step 3-2: Define the experimental configuration. Before training the model, first define the hyperparameters and the model optimizer used to update the model parameters. The length L of the non-overlapping sliding window is set to 100, and the model hyperparameter d of the anomaly attention module is defined. model The learning rate was set to 512, the hyperparameter λ in the loss function was set to 3, and the training batch size was 32. Then, the learning rate scheduler and the maximum number of iterations or related early stopping strategies were set, using the ADAM optimizer (reference: Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980 (2014).), with an initial learning rate of 10. -4 The early stop strategy is set to end within 10 rounds.

[0112] Step 3-3: Perform unsupervised training on the model, reconstruct the data in the dataset, and update the gradient based on the loss function defined in Step 3-1 to obtain the trained anomaly detection model.

[0113] Step 4 specifically includes the following steps:

[0114] Step 4-1: Design the formula for calculating the anomaly score. The anomaly score can be evaluated using the association differences designed in Step 3-1. The association differences of anomalous data will be greater than those of normal data.

[0115]

[0116] Step 4-2: Use the network trained in Step 3 to process the original time series, and use the anomaly score calculation formula in Step 6-1 to calculate the anomaly score for each data point.

[0117] Step 4-3: Determine the anomaly detection threshold by using quantiles. The quantiles of the SWaT dataset are set to 0.1%, and the quantiles of the WADI dataset are set to 1% (the UCR dataset does not need to be set with quantiles because each data point in this dataset has only one anomaly). If the anomaly score of a data point calculated in step 6-2 exceeds the threshold, the point is determined to be an anomaly; otherwise, it is determined to be a normal point.

[0118] Example 1:

[0119] This embodiment was tested on the standard public datasets UCR, SWAT, and WADI. These datasets contain time-series text data stored in a computer in formats such as CSV. This invention is compared with mainstream methods such as AnomalyTransformer (proposed at ICLR 2022), LSTM-VAE (proposed at ICRA 2018), TS2Vec (proposed at AAAI 2022), Deep-SVDD (proposed at ICML 2018), LSTM (proposed at SIGKDD 2018), and Isolation Forest (proposed at ICDM 2008). The experimental results are shown in Tables 1 and 2. Table 1 shows the accuracy, recall, and F1 score of different methods on the SWAT and WADI datasets. Table 2 shows the Acc@1, Acc@2, and Acc@3 metrics of different methods on the UCR dataset.

[0120]

[0121] Table 1

[0122]

[0123] Table 2

[0124] As can be seen, compared with public datasets in different fields, this invention achieves better results. From the F1 metric, which can comprehensively evaluate anomaly detection capabilities, this invention achieves the best results on all datasets. Furthermore, on the complex UCR dataset, its Acc@1 (top 1 accuracy), Acc@2 (top 2 accuracy), and Acc@3 (top 3 accuracy) metrics all achieve the best results, effectively demonstrating the anomaly detection performance of this invention.

[0125] Example 2:

[0126] The temporal anomaly detection method based on neighborhood information fusion attention mechanism described in this invention can be applied to server equipment fault monitoring. This embodiment introduces the implementation steps and effects of this method on a real-world server performance monitoring dataset, specifically including:

[0127] Step 1 involves preprocessing the time-series data in the dataset. The dataset consists of 45 days of actual server performance monitoring data collected in CSV format and stored on a computer. It contains 19 server metrics (including processor-related metrics, memory-related metrics, network metrics, etc.), collected every 5 minutes. This step includes:

[0128] Normalize each time series in the dataset, scaling each time series to a minimum value of 0 and a maximum value of 1:

[0129]

[0130] Secondly, the sequence is windowed, with each window having a length of T and a window sliding step of T. That is, each time series is divided into k non-overlapping sub-windows of length T, which facilitates the subsequent model to extract time-dependent features and perform anomaly detection.

[0131] Step 2 involves building a neural network model based on sequence decomposition and attention mechanisms, i.e., an anomaly detection model. This anomaly detection model uses a neural network to extract multi-level features from the input data and reconstructs the input data using these extracted multi-level features. This step includes:

[0132] Step 2-1: Build the sequence decomposition attention module, which consists of three parts: a sequence decomposition layer, an anomaly attention layer, and a fully connected layer. This step includes:

[0133] Step 2-1-1, Construct the sequence decomposition layer:

[0134] The preprocessed data in step 1 is decomposed into sequences, including information fusion and the generation of two components: a trend component and a seasonal component.

[0135] Information fusion involves concatenating the data of the current window (the original window) with the data from the preceding and following windows.

[0136]

[0137] in, It is the spliced ​​data. This refers to the data of the current window, i.e., the original window. It's data from the previous window. This refers to the data in the next window; Concat{·} is the concatenation operation; the window in question is the window obtained during the preprocessing in step 1.

[0138] After the data is stitched together, the trend component is extracted from the stitched data using average pooling. It is expressed as follows:

[0139]

[0140] AvgPool(·) is the average pooling operation with a step size of 3;

[0141] Finally, the seasonal component is obtained by subtracting the trend component from the original window data. It is expressed as follows:

[0142]

[0143] Step 2-1-2, Build the anomaly attention layer:

[0144] Using the seasonal components extracted in step 2-1-1 as input, prior association and sequence association were modeled respectively.

[0145] First, the attention mechanism is calculated using the weight matrix (reference: Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017)). σ matrix:

[0146]

[0147] in, σ is the matrix component in the attention mechanism, and σ is the learnable Gaussian kernel parameter.

[0148] First, a prior association is modeled. A learnable Gaussian kernel is used to calculate prior information about relative temporal distances. This design leverages the unimodal nature of the Gaussian kernel to focus attention on adjacent regions. A learnable scaling parameter is used in the Gaussian kernel to adapt the prior association to various time series patterns, such as different lengths of outlier segments. The specific modeling method is as follows:

[0149]

[0150] Rescale(·) refers to scaling the data based on the Gaussian kernel, generating a prior unimodal weight for each data point according to its Gaussian kernel parameters, with closer data points having larger weights. Prior association can force the model to focus its attention on the neighborhood of data points.

[0151] Then, sequence association is modeled, which is extracted based on an attention mechanism. The modeling method is as follows:

[0152]

[0153] Global attention weights are extracted from the input data through sequence association, and attention components are obtained.

[0154]

[0155] The attention component is the first output of the sequence decomposition attention module;

[0156] Step 2-1-3, build the fully connected layer:

[0157] The fully connected layer is used to extract the trend component obtained in step 2-1-1. It consists of a three-layer fully connected neural network and a ReLU function, as shown below:

[0158]

[0159] Here, FeedForward() represents the fully connected layer, and S1 is the output of the fully connected layer, which is the second output of the sequence decomposition attention module.

[0160] Step 2-2: Build an anomaly detection model based on the sequence decomposition attention module, including a two-layer sequence decomposition attention module, a residual connection module, and a reconstruction module. This step includes:

[0161] Two sequence decomposition attention modules are stacked. The input to the first sequence decomposition attention module is the concatenated data in step 2-1-1. The output of the anomaly attention layer in the first sequence decomposition attention module is padded with zero values ​​to three times its length, and this padded data is used as the input to the second sequence decomposition attention module. In addition, the original window data is input to a separate anomaly attention layer as a residual connection module. Finally, the outputs of the above three modules are concatenated by the reconstruction module and input into the fully connected layer as the output layer of the anomaly detection model.

[0162] Step 3, training the anomaly detection model described in Step 2, includes: defining the loss function and neural network hyperparameters, iteratively optimizing the parameters in the anomaly detection model based on an unsupervised reconstruction task, and obtaining the trained anomaly detection model. This step includes:

[0163] Step 3-1: Design the loss function for the reconstruction task, including reconstruction loss and attention association difference loss. Specifically, the loss function consists of two parts: reconstruction loss and attention association difference loss. The reconstruction loss reflects the model's reconstruction effect on the sequence and can guide the model to find the most informative attention weights. The attention association difference loss guides the distribution of sequence associations to be as different as possible from the prior association distribution, strengthening the model's ability to identify outliers. The specific loss function is as follows:

[0164]

[0165] Where AssDis(·) refers to the association difference, which represents the information gain between prior association and sequence association. It can be used to quantify the difference between two distributions, and the specific calculation formula is as follows:

[0166]

[0167] Among them, KL(·||·) is composed of and The KL divergence is calculated from the two discrete distributions corresponding to each row.

[0168] Step 3-2: Define the experimental configuration. Before training the model, first define the hyperparameters and the model optimizer used to update the model parameters. The length L of the non-overlapping sliding window is set to 100, and the model hyperparameter d of the anomaly attention module is defined. model The learning rate is set to 512, the hyperparameter λ in the loss function is set to 3, and the training batch size is 32. Then, the learning rate scheduler and the maximum number of iterations or related early stopping strategies are set, using the ADAM optimizer with an initial learning rate of 10. -4 The early stop strategy is set to end within 10 rounds.

[0169] Step 3-3: Perform unsupervised training on the model, reconstruct the data in the dataset, and update the gradient based on the loss function defined in Step 3-1 to obtain the trained anomaly detection model.

[0170] Step 4: Using the anomaly detection model trained in Step 3, process the time series data to be tested. Calculate the anomaly score based on the output of the anomaly detection model and determine whether the input time series data to be tested is anomaly. This step includes:

[0171] Step 4-1: Design the formula for calculating the anomaly score. The anomaly score can be evaluated using the association differences designed in Step 3-1. The association differences of anomalous data will be greater than those of normal data.

[0172]

[0173] Step 4-2: Use the network trained in Step 3 to process the original time series, and use the anomaly score calculation formula in Step 6-1 to calculate the anomaly score for each data point.

[0174] Step 4-3: Determine the anomaly detection threshold using quantiles. Here, the quantile is set to 1%. If the anomaly score exceeds the threshold, the data is considered an anomaly. Figure 4 The diagram demonstrates the effectiveness of some anomaly detection methods. The four metrics in the figure represent CPU utilization (metric 1), network throughput (metrics 2 / 3), and disk read / write activity (metric 4), with the shaded areas representing anomalies identified by this invention. The first identified anomaly exhibits significant network fluctuations, while the second anomaly involves brief periods of high CPU load, network fluctuations, and high disk load—all valid anomalies. Therefore, this invention can accurately identify anomalies, quickly triggering alerts and assisting professionals in troubleshooting when a server malfunctions.

[0175] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a temporal anomaly detection method based on a neighborhood information fusion attention mechanism, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0176] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0177] This invention provides an idea and method for temporal anomaly detection based on a neighborhood information fusion attention mechanism. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A time series anomaly detection method based on neighborhood information fusion attention mechanism, characterized in that, The method comprises the following steps: Step 1, preprocessing text data containing time series in a data set; Step 2, building a neural network model based on sequence decomposition and attention mechanism, i.e. an anomaly detection model, which uses a neural network to extract multi-level features of input data and uses the extracted multi-level features for reconstruction to obtain a reconstruction result of the input data; Step 3, training the anomaly detection model of step 2, including: defining a loss function and neural network hyperparameters, iteratively optimizing the parameters in the anomaly detection model based on an unsupervised reconstruction task, and obtaining a trained anomaly detection model; Step 4, using the trained anomaly detection model obtained in step 3 to process the time series data to be tested, calculating an anomaly score according to the output of the anomaly detection model, and determining whether the input time series data to be tested is abnormal; Step 5, applying the method of steps 1 to 4 to the data set of the technical system containing time series text data to perform anomaly detection on the time series text data in the data set; Wherein, the preprocessing of step 1 comprises: Firstly, normalize each time series in the data set to scale all data to the same range, and secondly, window the sequence, with each window length T and window sliding step T, i.e. divide the time series into non-overlapping sub-windows; Step 2 builds an anomaly detection model, including: Step 2-1, building a sequence decomposition attention module, including three parts: a sequence decomposition layer, an anomaly attention layer, and a fully connected layer; Step 2-2, building an anomaly detection model based on the sequence decomposition attention module, including a double-layer sequence decomposition attention module, a residual connection module, and a reconstruction module; Step 2-1 builds a sequence decomposition attention module, specifically including: Step 2-1-1, building a sequence decomposition layer: Perform sequence decomposition on the preprocessed data in step 1, including information fusion and generating two components, i.e. a trend component and a seasonal component; Wherein, information fusion is to concatenate the current window, i.e. the original window data, with the data of the previous and next two windows: ； wherein, is the spliced data, is the current window, i.e., the original window data, is the previous window data, is the next window data, is the splicing operation; the window is the window obtained by dividing in the preprocessing of step 1. After the stitching is completed, the stitched data is extracted for a trend component through an average pooling operation is expressed as follows: ； wherein is an average pooling operation with a stride of 3; Finally, the seasonal component is obtained by differencing the trend component from the original window data and is represented as follows: ； Step 2-1-2, building an anomaly attention layer: The attention mechanism in the weight matrix is calculated by matrix: ； wherein, is a matrix component in the attention mechanism, is a learnable Gaussian kernel parameter, is input data of the model, , , are a query weight matrix, a key-value weight matrix, an attention weight matrix in the attention mechanism, respectively, is a Gaussian kernel parameter weight matrix; Take the seasonal component extracted in step 2-1-1 as input, model prior correlation and sequence correlation respectively, and use them to define the loss function in step 3 and the anomaly score in step 4; Step 2-1-3, building a fully connected layer: The fully connected layer is used to extract the trend component obtained in step 2-1-1 and is composed of three layers of fully connected neural networks and ReLu functions, represented as follows: ； wherein, denotes a fully connected layer, is the output of the fully connected layer, which is the second output of the sequence decomposition attention module; Step 2-2 builds an anomaly detection model based on the sequence decomposition attention module, specifically including: The two-layer sequence decomposition attention module is superimposed, the input of the first-layer sequence decomposition attention module is the spliced data in step 2-1-1, and the output of the abnormal attention layer in the first-layer sequence decomposition attention module is edge padded and used as the input of the second-layer sequence decomposition attention module; in addition, the original window data is input into a separate abnormal attention layer and used as a residual connection module; finally, the outputs of the three modules are spliced through a reconstruction module and input into a full connection layer as an output layer of the abnormality detection model.

2. The time series anomaly detection method based on neighborhood information fusion attention mechanism according to claim 1, characterized in that, The modeling prior correlation in step 2-1-2 specifically includes: The prior correlation adopts a learnable Gaussian kernel to calculate the prior information of the relative time distance, and the specific modeling manner is as follows: ； where, represents the prior correlation, means scaling the data based on the Gaussian kernel, represents the input data window length, represents any two data point numbers on the same data, represents the Gaussian kernel parameter of the th data, represents the exponential function with the natural constant e as the base.

3. The time series anomaly detection method based on neighborhood information fusion attention mechanism according to claim 2, characterized in that, The sequence correlation in step 2-1-2 is modeled based on an attention mechanism for extraction, and the modeling manner is as follows: ； wherein, denotes sequence association, denotes transpose operation, denotes key-value matrix in attention mechanism, denotes dimension of the model, denotes softmax function; extracting global attention weights of input data by sequence correlation, and obtaining attention components : ； Matrix components storing attention weights is the first output of the sequence decomposition attention module.

4. The time series anomaly detection method based on neighborhood information fusion attention mechanism according to claim 3, characterized in that, Step 3 trains the abnormality detection model, specifically including: Step 3-1, a loss function is designed, including a reconstruction loss and an attention correlation difference loss, and the specific loss function is as follows: ； wherein, denotes the model reconstruction result, denotes the L2 norm of a vector, is a hyperparameter, is the correlation difference, and the specific calculation formula is as follows: ； wherein, is calculated from and the KL divergence of the two discrete distributions corresponding to each row, L denotes the output data window length; Step 3-2, define the experimental related configuration: before training the model, first define the hyperparameters and the model optimizer used to update the model parameters, and then set the learning rate scheduler and the maximum number of iterations or the related early stopping strategy; Step 3-3, the abnormality detection model is unsupervised trained, the data in the data set is reconstructed, and the gradient is updated based on the loss function defined in step 3-1 to obtain the trained abnormality detection model.

5. The time series anomaly detection method based on neighborhood information fusion attention mechanism according to claim 4, characterized in that, Step 4 judges whether the input time series data to be tested is abnormal, specifically including: Step 4-1, an abnormal score calculation method is designed, that is, the abnormal score is calculated by the correlation difference designed in step 3-1, and the specific method is as follows: ； wherein, represents an abnormal score, represents function, represents multiplication of corresponding elements of matrices, represents L-2 norm; Step 4-2, the trained abnormality detection model in step 3 is used to process the data in the data set, and the abnormal score calculation method in step 4-1 is used to calculate the abnormal score of each data in the data set; Step 4-3, the abnormality judgment threshold is determined by the quantile, if the abnormal score exceeds the threshold, the data is determined as abnormal data.

6. The time series anomaly detection method based on neighborhood information fusion attention mechanism according to claim 5, characterized in that, The edge padding in step 2-2 is that the data is padded to three times the length with 0 values.

Images