Multi-granularity reconstruction difference time sequence anomaly detection method

Abstract

The invention discloses a multi-granularity reconstruction difference time sequence anomaly detection method, which belongs to the technical field of time sequence anomaly detection, and comprises the following steps of: preprocessing a time sequence to obtain a local initial input vector and a global initial input vector; inputting the local initial input vector and the global initial input vector into a dual-path Transform coding module, performing local reconstruction and global reconstruction, and generating a first local reconstruction output and a first global reconstruction output; performing linear layer mapping on the first local reconstruction output and the first global reconstruction output to obtain a second local reconstruction output and a second global reconstruction output, and comparing the second local reconstruction output with the second global reconstruction output to obtain an anomaly detection result of the time series data; according to the method, the global information and the local information are utilized, the accuracy and robustness of time sequence anomaly detection are improved, and the anomaly detection performance is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series anomaly detection, and in particular to a multi-granularity reconstruction difference time series anomaly detection method. Background Art

[0002] Time series refers to a series of observation data or samples arranged in chronological order, usually data collected at equal time intervals. Time series data essentially reflects the trend of one or more random variables changing over time.

[0003] Time series anomaly detection aims to identify data points that are significantly different from most data. It is the process of identifying abnormal events or behaviors from normal time series. It is used to identify and mark data points or time periods that significantly deviate from normal behavior patterns. It is widely used in finance, network security, industrial maintenance, medical health, traffic management and other fields. Analyzing anomalies in time series helps to promptly discover potential risks and problems, help prevent losses, optimize operational efficiency, and improve the reliability and security of the overall system.

[0004] Traditional time series anomaly detection technologies include ARIMA-based prediction models and k-means-based clustering models. These methods have achieved certain success in univariate time series anomaly detection, but they fail to consider the temporal dependency in time series. In recent years, with the widespread application of deep learning methods, there are more and more anomaly detection methods based on deep learning methods, including anomaly detection methods based on convolutional neural networks or recurrent neural networks. However, the above methods ignore the relationship between local and global features, and cannot make good use of the correlation between global information and local anomalies, resulting in low detection accuracy. Summary of the invention

[0005] In view of the above-mentioned prior art, the present invention provides a multi-granularity reconstruction difference time series anomaly detection method, which mainly solves the technical problems existing in the above-mentioned background technology.

[0006] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0007] The multi-granularity reconstruction difference time series anomaly detection method includes the following steps:

[0008] After preprocessing the time series, a local initial input vector and a global initial input vector are obtained;

[0009] Inputting the local initial input vector and the global initial input vector into the dual-path Transformer encoding module, performing local reconstruction and global reconstruction, and generating a first local reconstruction output and a first global reconstruction output;

[0010] The first local reconstruction output and the first global reconstruction output are mapped through a linear layer to obtain a second local reconstruction output and a second global reconstruction output. The second local reconstruction output is compared with the second global reconstruction output to obtain an anomaly detection result of the time series data.

[0011] Optionally, the dual-path Transformer encoding module includes a plurality of dual-path Transformer encoding modules, and the plurality of dual-path Transformer encoding modules are stacked and connected;

[0012] The dual-path Transformer encoding module includes a multi-granularity attention module, a first normalization layer, two convolutional layers, and a second normalization layer. The dual-path Transformer encoding module is internally connected using residual connections.

[0013] Optionally, performing local reconstruction and global reconstruction to generate a first local reconstruction output and a first global reconstruction output specifically includes:

[0014] The local information in the local initial input vector and the global information in the global initial vector are captured through the multi-granularity attention module to generate intermediate local reconstruction output, intermediate global reconstruction output, local granularity attention matrix and global granularity attention matrix; the intermediate local reconstruction output and the intermediate global reconstruction output are normalized by the first normalization layer, and then input into the convolution layer to extract local features and process global features respectively; the intermediate local reconstruction output and the intermediate global reconstruction output after the convolution layer are normalized by the second normalization layer to generate the first local reconstruction output and the first global reconstruction output.

[0015] Optionally, the multi-granularity attention module includes a local granularity attention submodule and a global granularity attention submodule, the local granularity attention submodule outputs a local granularity attention matrix used as a local reconstruction output, and the global granularity attention submodule outputs a global granularity attention matrix used as a global reconstruction output;

[0016] The local granular attention submodule is used to learn the association weights between each data point in the time series and its adjacent data points using the attention mechanism, and acts on local reconstruction; the local granular attention submodule includes a Gaussian kernel function, which is used to enhance the fitting ability of the local attention submodule;

[0017] The global granular attention submodule is used to directly learn the association weights between each data point and other data points in the time series using the attention mechanism, and acts on the global reconstruction.

[0018] Optionally, it also includes: optimizing using a loss function and a two-stage partial gradient stop optimization strategy;

[0019] The expression of the loss function is:

[0020]

[0021] in, is the mean square error loss term of the local reconstruction output, is the second local reconstruction output, X is the time series, is the mean square error loss term of the global reconstruction output, Reconstruct the output for the second global, is the local granular attention matrix, is the global granular attention matrix, λ is the hyperparameter of the KL divergence constraint term of the loss function, and M is the number of Transformer encoding modules.

[0022] Optionally, in the process of optimizing using the loss function, the root mean square error between the mean square error loss term of the local reconstruction output and the mean square error term of the global reconstruction output is also used to quantify the anomaly, and the expression is:

[0023]

[0024] Where D is the variable dimension of the local and global reconstruction output data, is the local reconstruction output on the i-th variable dimension, is the global reconstruction output on the i-th variable dimension.

[0025] Optionally, the two-stage partial gradient stopping optimization strategy includes a first-stage optimization and a second-stage optimization;

[0026] In the first stage of optimization, the gradient back propagation is stopped for the second local reconstruction output and the local granular attention matrix, forcing the global granular attention matrix to move away from the local granular attention matrix and pay more attention to the information between non-adjacent time points in the time series;

[0027] In the second stage of optimization, the gradient back propagation is stopped for the second global reconstruction output and the global granular attention matrix, focusing on the local area and paying more attention to the information between adjacent time points in the time series;

[0028] The expression for the first stage optimization is:

[0029] Phase 1:

[0030] The expression for the second stage optimization is:

[0031] Phase 2:

[0032] Among them, L1 is the loss function of the first stage optimization process, and L2 is the loss function of the second stage optimization process.

[0033] Optionally, it also includes an evaluation experiment on the multi-granularity reconstruction difference time series anomaly detection method, specifically using different methods to perform anomaly detection on seven different data sets, and scoring, and comparing the obtained scoring results, where the evaluation indicators are precision, recall rate and F1 score.

[0034] The beneficial effects of the present invention are as follows: the multi-granularity reconstruction difference time series detection method provided by the present invention effectively decomposes the time series into two granularity levels of reconstruction, captures the global and local features of the time series through global reconstruction and local reconstruction, and simultaneously utilizes global information and local information to improve the accuracy and robustness of time series anomaly detection, thereby effectively improving the performance of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of the multi-granularity reconstruction difference time series anomaly detection method provided in an embodiment of the present invention;

[0036] Figure 2 It is an overall block diagram of the multi-granularity reconstruction difference time series anomaly detection method provided in an embodiment of the present invention;

[0037] Figure 3 A comparison chart of the ablation test results of gradient stop and local KL divergence in the evaluation experiment of the present invention;

[0038] Figure 4 This is a comparison chart of the results of abnormal score analysis in the evaluation experiment of the present invention;

[0039] Figure 5 This is a comparison chart of the effect of Gaussian kernel width σ on anomaly detection performance in the evaluation experiment of the present invention;

[0040] Figure 6 This is a comparison chart of the impact of λ on anomaly detection performance in the evaluation experiment of the present invention;

[0041] Figure 7 This is a comparison chart of the results of the impact of window size on anomaly detection performance in the evaluation experiment of the present invention;

[0042] Figure 8 This is a comparison chart of the results of the impact of the number of Transformer layers on anomaly detection performance in the evaluation experiment of the present invention;

[0043] Fig. 9 This is a comparison chart of the results of the impact of the embedding dimension on the anomaly detection performance in the evaluation experiment of the present invention;

[0044] Fig.10 This is a result diagram of visualizing the anomaly score in the evaluation experiment of the present invention;

[0045] Fig.11 The figure is a detection result diagram of five kinds of anomalies in the evaluation experiment of the present invention;

[0046] Fig.12 This is a comparison chart of the results of visualizing anomaly scores in the evaluation experiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is further elaborated in detail below in conjunction with the drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. In the following description, the expression "some embodiments" is related to a subset of all possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0048] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0049] It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments proposed herein. On the contrary, providing these embodiments will make the disclosure thorough and complete, and the scope of the present invention will be fully conveyed to those skilled in the art. And the purpose of the terms used herein is only to describe specific embodiments and is not intended to be a limitation of the present invention. When used herein, the singular forms of "one", "one" and "said / the" are also intended to include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "compose" and / or "include" when used in this specification determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0050] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only implementation method.

[0051] In order to fully understand the present invention, a detailed structure will be proposed in the following description to illustrate the technical solution proposed by the present invention. The optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementations.

[0052] Example

[0053] Please refer to Figure 1 and Figure 2 The present invention provides a multi-granularity reconstruction difference time series anomaly detection method, comprising the following steps:

[0054] After preprocessing the time series, a local initial input vector and a global initial input vector are obtained;

[0055] Specifically, the time series X is cleaned to remove noise and fill missing values, providing a high-quality data basis for subsequent time series anomaly detection. The cleaned time series X is passed through the embedding layer to obtain the local initial input vector and the global initial input vector The specific expression is:

[0056]

[0057] Embedding(X)=Dropout(PE+VE(X))

[0058] in, d represents the embedding dimension, PE represents the position encoding using the sine function, and VE(~) represents the convolutional neural network for feature extraction;

[0059] Inputting the local initial input vector and the global initial input vector into the dual-path Transformer encoding module, performing local reconstruction and global reconstruction, and generating a first local reconstruction output and a first global reconstruction output;

[0060] Specifically, the local initial input vector and the global initial input vector The input is sent to the dual-path Transformer encoding module. The Transformer encoding module captures the time dependency and the correlation between sequences in the time series data through the self-attention mechanism. The input sequence is processed by the Transformer encoding module to extract the high-order features of the input. Then, the anomaly score can be calculated by comparing the difference between the input data and the reconstructed data. The data points with higher scores are considered to be abnormal. In the present invention, a dual-path Transformer encoding module is used, and the Transformer encoding module under one path is used to process the local initial input vector Perform local reconstruction to generate the first local reconstruction output The Transformer encoding module under another path is used to process the global initial input vector Perform global reconstruction and generate the first global reconstruction output

[0061] The first local reconstruction output and the first global reconstruction output are mapped through a linear layer to obtain a second local reconstruction output and a second global reconstruction output, and an anomaly detection result of the time series data is obtained by comparing the second local reconstruction output with the second global reconstruction output;

[0062] Specifically, the first local reconstruction output and the first global reconstruction output Input to the linear layer, the linear layer can reconstruct the output of the first local and the first global reconstruction output Transform and scale, and map to obtain the second local reconstruction output and the second global reconstruction output Reconstruct the second local output and the second global reconstruction output Perform error calculation. If the second local reconstruction output Reconstruct output with the second global A large error between may indicate that there is an anomaly at that time point, because anomalies often lead to an increase in the reconstruction error of the model;

[0063] Exemplarily, the error between the second local reconstruction output and the second global reconstruction output is calculated, and an error threshold is set. If the difference exceeds the threshold, it is considered that the time point may be abnormal, and finally the position information of the abnormal time point is output.

[0064] It should be noted that since a time series is composed of a series of data points arranged in a time series, if the time series contains normal time points (referring to data points in the time series that conform to the expected pattern or behavior), then the normal time points usually follow certain rules or are distributed within a specific statistical range, that is, normal time points should have similar statistical characteristics over a long period of time, so that normal time points can establish connections with more distant time points;

[0065] Abnormal time points (data points in a time series that deviate from expected patterns or behaviors) may be caused by system failures, external interference, emergencies, or other abnormal factors, and often exhibit statistical characteristics that are significantly different from normal time points, such as sudden peaks, valleys, or sudden changes in trends, making abnormal time points more concerned with adjacent time points with similar abnormal behaviors.

[0066] Therefore, in the entire time series, the correlation between normal time points and the entire time series is usually strong, while the correlation between abnormal time points is mainly concentrated on adjacent time points due to their sparsity, and it is difficult to establish a strong correlation with the entire time series; in the local process, abnormal time points often do not match the local pattern, and the correlation between abnormal time points and adjacent time points may show obvious inconsistency; through this local and global inconsistency, normal time points and abnormal time points can be distinguished; therefore, by comparing the second local reconstruction output Reconstruct output with the second global The difference can be used to obtain the anomaly detection results of time series data.

[0067] As an optional implementation, the dual-path Transformer encoding module includes a plurality of dual-path Transformer encoding modules, and the plurality of dual-path Transformer encoding modules are stacked and connected;

[0068] The dual-path Transformer encoding module includes a multi-granularity attention module, a first normalization layer, two convolutional layers, and a second normalization layer. The dual-path Transformer encoding module is internally connected using residual connections.

[0069] Performing local reconstruction and global reconstruction to generate a first local reconstruction output and a first global reconstruction output specifically includes:

[0070] Specifically, M dual-path Transformer encoding modules are stacked, and the output of the previous dual-path Transformer encoding module is the input of the next dual-path Transformer encoding module, until the mth layer, the input of the mth layer of the dual-path Transformer After being processed by the multi-granularity attention module, local reconstruction and global reconstruction are performed. Specifically, the multi-granularity attention module captures the local information in the local initial input vector and the global information in the global initial vector. The multi-granularity attention module first outputs the local granularity attention matrix Global Granular Attention Matrix Intermediate local reconstruction output and the intermediate global reconstruction output Reconstruct the intermediate local output and the intermediate global reconstruction output After the first normalization layer is applied for normalization, the output is input into the convolution layer to extract local features and process global features respectively. The intermediate local reconstruction output and the intermediate global reconstruction output after the convolution layer are normalized by the second normalization layer to generate the first local reconstruction output. and the first global reconstruction output The specific expression of the above process is:

[0071]

[0072] Among them, MGA(~) represents the multi-granularity attention module, T∈{L,G}, indicating that they involve local and global cases, L is the local representation, G is the global representation, Represents the last layer output of the dual-path Transformer, and finally obtains the reconstructed output of X through a linear layer mapping

[0073] As an optional implementation, the multi-granularity attention module includes a local granularity attention submodule and a global granularity attention submodule, the local granularity attention submodule outputs a local granularity attention matrix used as a local reconstruction output, and the global granularity attention submodule outputs a global granularity attention matrix used as a global reconstruction output;

[0074] The local granular attention submodule is used to learn the association weights between each data point in the time series and its adjacent data points using the attention mechanism, and acts on local reconstruction; the local granular attention submodule includes a Gaussian kernel function, which is used to enhance the fitting ability of the local attention submodule;

[0075] The specific expression is:

[0076]

[0077] in, is the local value matrix in the m-th layer model; represents the value of the Gaussian matrix G at the i-th row and j-th column, is the width of the Gaussian function, and the neighborhood focus range at the i-th point is defined as: Where |j-1| represents the relative time distance, i,j∈{1,2,...,T}; represents the reconstructed output of the local granular attention at the mth layer, Q represents Query, and K represents Key;

[0078] Since the local granular attention submodule focuses on the information of the local area, the correlation between adjacent time points in the local initial input vector is stronger than the correlation between distant time points. Therefore, the local granular attention submodule learns the association weights between each data point and the adjacent data points in the local initial input vector. For each data point in the local initial input vector, only the attention weights between it and the adjacent positions are calculated, and then the results in the local range are summed. The result of this weighted summation represents the information of the given data point, while considering the influence of the data points in its adjacent positions in the local range, so that the output contains the information of other data points in the local range, realizing local association;

[0079] The global granular attention submodule is used to directly learn the association weights between each data point and other data points in the time series using the attention mechanism, and acts on the global reconstruction;

[0080] The specific expression is:

[0081]

[0082] in, is a matrix of query, key, and value. represents the model parameters, represents the global granular attention matrix, represents the reconstructed output of the global granular attention at the mth layer, is the global value matrix in the m-th layer model;

[0083] Since the global granular attention submodule is used for global reconstruction, in order to better associate contextual information and capture the long-distance dependencies between data points, the global granular attention submodule is used to learn the association weights between each data point and other data points in the time series, calculate the attention weights between each data point and other data points, and then perform weighted summation on the attention weights of data points at all positions. The result of the weighted summation represents the information of the data point, while considering the influence of other data points on it. These weighted summation results are then applied to global reconstruction, so that in the output, each data point contains the information of other data points in the entire time series data set, realizing global association;

[0084] Specifically, the reconstructed outputs of the global granular attention submodule and the local granular attention submodule of the previous layer are used as the inputs of the global granular attention submodule and the local granular attention submodule of the current layer after mapping, matrix calculation, scaling, Softmax and other operations. The input of the global granular attention submodule of the current layer is scaled, and the input of the local granular attention submodule of the current layer is processed by Gaussian kernel function constraints. The data constrained by the Gaussian kernel function is concatenated with the data of the global granular attention submodule after scaling operations, so that the Gaussian matrix is ​​combined with the global attention weight, guiding the attention matrix weight to concentrate on the neighboring area, retaining the original attention features of the data, and then normalizing to obtain the local granular attention matrix. The local granular attention matrix is ​​multiplied by the local value matrix of the current layer to obtain the intermediate local reconstruction output;

[0085] The input of the global granular attention submodule of the current layer is scaled and normalized to obtain the global granular attention matrix. The global granular attention matrix is ​​multiplied by the global value matrix of the current layer to obtain the intermediate global reconstruction output.

[0086] Adding Gaussian kernel function constraints in the local granular attention submodule can retain the unimodal characteristics of the local granular attention matrix and pay more attention to neighborhood data; splicing the data constrained by the Gaussian kernel function with the data after scaling operation of the global granular attention submodule can improve the convergence speed of the local granular attention submodule during the optimization process, thereby enhancing the fitting ability of the local attention submodule.

[0087] As an optional implementation, it also includes: optimizing using a loss function and a two-stage partial gradient stop optimization strategy;

[0088] The expression of the loss function is:

[0089]

[0090] in, is the mean square error loss term of the local reconstruction output, is the second local reconstruction output, X is the time series, is the mean square error loss term of the global reconstruction output, Reconstruct the output for the second global, is the local granular attention matrix, is the global granular attention matrix, λ is the hyperparameter of the KL divergence constraint term of the loss function, and M is the number of Transformer encoding modules;

[0091] The loss function includes the mean square error loss term of the local reconstruction output, the mean square error term of the global reconstruction output, and the KL divergence constraint loss term;

[0092] The mean square error loss term of the local reconstruction output is used to minimize the difference between the time series and the second local reconstruction output;

[0093] The mean square error term of the global reconstruction output is used to minimize the difference between the time series and the second global reconstruction output;

[0094] The KL divergence constraint loss term is used as a regularization term to minimize the KL divergence between the local granular attention matrix and the global granular attention matrix;

[0095] By calculating the loss term between the second local reconstruction output and the actual time series, and the loss term between the second global reconstruction output and the actual time series, if the MSE loss term of the local reconstruction output or the global reconstruction output is significantly higher than the normal level, this may indicate that there is an abnormality at the corresponding time point, because the abnormal time point often leads to an increase in the reconstruction error of the model, thereby increasing the MSE loss term; at the same time, by comparing the MSE loss terms of the local reconstruction output and the global reconstruction output, the inconsistency of local and global information is utilized, and the abnormal points in the time series can be comprehensively identified and located, effectively distinguishing normal time points from abnormal time points;

[0096] In order to prevent the over-coupling between the global granular attention submodule and the local granular attention submodule, the KL divergence constraint is used for decoupling. The KL divergence is used to measure the difference between the second local reconstruction output and the second global reconstruction output to ensure that they learn complementary features, which helps to improve the detection performance of time series anomalies.

[0097] Through the loss function, on the one hand, the original time series X is decomposed into local reconstructions and global refactoring Two parts, the second part reconstructs the output The noise in the time series X is effectively eliminated; on the other hand, the second global reconstruction output It mainly captures the overall structural information of the time series X and ignores local changes.

[0098] As an optional implementation, in the process of optimizing using the loss function, the root mean square error between the mean square error loss term of the local reconstruction output and the mean square error term of the global reconstruction output is also used to quantify the anomaly, and the expression is:

[0099]

[0100] Where D is the variable dimension of the local and global reconstruction output data, is the local reconstruction output on the i-th variable dimension, is the global reconstruction output on the i-th variable dimension.

[0101] As an optional implementation, the two-stage partial gradient stop optimization strategy includes a first-stage optimization and a second-stage optimization;

[0102] In the first stage of optimization, the gradient back propagation is stopped for the second local reconstruction output and the local granular attention matrix, forcing the global granular attention matrix to move away from the local granular attention matrix and pay more attention to the information between non-adjacent time points in the time series;

[0103] In the second stage of optimization, the gradient back propagation is stopped for the second global reconstruction output and the global granular attention matrix, focusing on the local area and paying more attention to the information between adjacent time points in the time series;

[0104] The expression for the first stage optimization is:

[0105] Phase 1:

[0106] The expression for the second stage optimization is:

[0107] Phase 2:

[0108] Among them, L1 is the loss function of the first stage optimization process, and L2 is the loss function of the second stage optimization process.

[0109] In order to effectively control the optimization process of two types of attention matrices, the global granular attention matrix and the local granular attention matrix, and obtain better disentangled representation, the optimization process is updated to a two-stage partial gradient stopping optimization method, and only one type of attention is optimized in each stage; by optimizing the global granular attention matrix and the local granular attention matrix respectively at different stages, the mutual interference between the two can be reduced, so that the model can focus more on learning the unique feature representation of each attention matrix; so as to better capture local and global features, thereby improving the performance of time series anomaly detection.

[0110] As an optional implementation, it also includes an evaluation experiment on the multi-granularity reconstruction difference time series anomaly detection method, specifically using different methods to perform anomaly detection on seven different time series data sets, and scoring, and comparing the obtained scoring results, wherein the evaluation indicators are precision, recall and F1 score;

[0111] Among them, the seven different time series datasets are:

[0112] (1)HAI (HAI Security Dataset): collected from an ICS test platform with a HIL simulator.

[0113] (2) MSL (Mars Science Laboratory Rover): obtained from the public dataset provided by the National Aeronautics and Space Administration (NASA), containing sensor data and actuator data of the Mars rover.

[0114] (3) PSM (Pooled Server Metrics): collected from each server node within the eBay infrastructure.

[0115] (4) SMAP (Soil Moisture Active Passive Satellite): A dataset of soil samples and telemetry information from Mars rovers provided by NASA.

[0116] (5) SMD (Server Machine Dataset): Data collected by a large Internet company over five weeks, containing stacked traces of resource utilization from 28 machines in a computing cluster.

[0117] (6) SWaT (Secure Water Treatment): collected from a water treatment test bed, including network traffic data and readings from 51 sensors and actuators. The data is collected over 11 consecutive days, of which 7 days are normal operation and 4 days have abnormal system behavior.

[0118] (7) WADI (Water Distribution): The WADI dataset is an extension of the SWaT system and consists of data collected over 16 days, including 14 days of normal operation and 2 days of attack scenarios. It contains readings from 123 sensors and actuators.

[0119] The multi-granularity reconstruction difference time series anomaly detection method (MGRD) provided by the present invention is compared with ten deep learning-based baseline models for time series anomaly detection.

[0120] Among them, the ten models are as follows:

[0121] (1) DAGMM (2018): An end-to-end optimized density estimation method using deep autoencoders and Gaussian mixture models. It can effectively balance reconstruction, latent representation density estimation, and regularization.

[0122] (2) OmniAnomaly (2019): Uses random variable connections and flat normalization flows to learn robust representations of normal patterns. This method identifies anomalies by reconstructing input data based on VAE and GRU.

[0123] (3) MSCRED (2019): It uses multi-scale matrices, convolutional encoders, attention-based ConvLSTM networks, and residual signature matrices to capture temporal dependencies and inter-sensor correlations. It identifies anomalies through the residual signature matrix.

[0124] (4) USAD (2020): An autoencoder-based model that uses adversarial training to detect anomalies.

[0125] (5) MTAD-GAT (2020): A self-supervised framework that uses graph attention layers to capture relationships in both temporal and feature dimensions. It jointly optimizes prediction and reconstruction to obtain better representations.

[0126] (6) CAE-M (2023): Uses deep convolutional autoencoders and memory networks to capture spatiotemporal correlations in multi-sensor time series data and distinguish between normal, abnormal, and noisy data.

[0127] (7)Anomaly Transformer (2022): A model for detecting anomalies by associating differences.

[0128] (8) GDN (2021): Utilizes graph neural networks, structured learning, and attention mechanisms to capture relationships between sensors and provide root cause analysis.

[0129] (9) TimesNet (2023): A general time series representation learning algorithm that expands the representation of complex temporal changes by converting one-dimensional time series into two-dimensional tensors, using two-dimensional convolutional kernels to model intra-cycle and inter-cycle variations of the data.

[0130] (10) TranAD (2022): A Transformer-based model that leverages attention-based sequence encoders, self-regulation, adversarial training, and model-agnostic meta-learning for simultaneous anomaly detection and diagnosis.

[0131] Specifically, the data set information is shown in Table 1:

[0132] Table 1 Information of seven datasets

[0133]

[0134]

[0135] The experiment uses non-overlapping sliding windows to obtain input subsequences. To ensure the validity of the experimental results, the size of a single sliding window is set to 100 and applied to all data sets. Avoid using anomaly adjustment strategies in the experimental results because this strategy tends to overestimate the actual detection performance. The value range of the r ratio is determined according to the actual anomaly ratio (as shown in Table 1). When the anomaly score exceeds the threshold δ, the corresponding data point is marked as anomaly.

[0136] MGRD uses a 3-layer Transformer (M=3) with a learning rate of 10 -4 , the loss function hyperparameter λ is different values ​​(1, 3 and 6), early stopping is performed within 20 epochs, and the experiment is run in the PyTorch 1.10 environment and uses the DCU Z100SM accelerator card.

[0137] (I) Performance comparison

[0138] Tables 2 and 3 compare the results of Precision, Recall, and F1 score (the harmonic mean of precision and recall) of the MGRD method provided by the present invention and ten deep learning models on seven data sets, and the F1 score is used as the evaluation index of focus. Among them, the best results are indicated in bold, and the suboptimal results are indicated by underscores. Some results are not included in the table because the memory requirements of some models exceed the capacity of the computing card.

[0139] From the experiments, we can see that MGRD obtains the best F1 score on all seven datasets, while the time series anomaly detection model TranAD and the time series representation model TimesNet obtain suboptimal results on the MSL, SWaT datasets and the HAI, SMAP, SMD datasets, respectively. This is because MGRD further models the noise and abnormal features. MGRD emphasizes the difference between noise and anomalies, while TranAD focuses on enhancing the model's robustness to noise through adversarial learning, without considering structural mutations in the data. TimesNet can obtain competitive results on different datasets because it establishes a robust representation of time series data, indicating that it is very important to establish a robust representation of normal time series.

[0140] MGRD has similar conditional assumptions as Anomaly Transformer, that is, noise is ubiquitous, but anomalies appear as local mutations. However, Anomaly Transformer mainly relies on the deviation between local and global associations as anomaly scores, and produces the worst results on datasets such as HAI, MSL, and PSM due to the lack of supervision on the local association matrix.

[0141] Table 2. Comparison of anomaly detection performance

[0142]

[0143]

[0144] Table 3. Comparison of anomaly detection performance

[0145]

[0146] (II) Ablation experiment

[0147] (1) Module analysis: In order to verify the contribution of specific modules in MGRD, a series of ablation experiments were conducted to analyze the impact of these modules on the F1 score. w / o gradient-stopping means removing some gradients to stop the optimization strategy, and w / olocal KL divergence means removing the local KL divergence constraint. The comparison results are shown in Figure 2. Figure 3 As shown in the figure, the KL divergence constraint improves the performance of the model, especially on the HAI, PSM, SMD and WADI datasets, and the partial gradient stop optimization method further improves the overall performance of the model. The experimental results verify the effectiveness of these two parts in improving model performance.

[0148] (2) Anomaly score analysis: Use of MGRD model As an anomaly score, and as anomaly scores for comparison. Figure 4 Experimental results for these three types of anomaly scores.

[0149] from Figure 4 It can be seen that The performance is the best on MSL, PSM, SMAP, SMD, and SWaT datasets, and the performance on HAI and WADI is second best but comparable to the best The performance is basically the same, The performance is the worst on almost all data sets. Based on the above observations, the following conclusions can be drawn:

[0150] 1. and The results show that under abnormal conditions, global granular reconstruction Deviation from local granularity reconstruction (or observation).

[0151] 2. When reconstructing local granularity, the information of adjacent points is used to reconstruct the observation points, which effectively realizes denoising and local smoothing of data. Performance is better than good.

[0152] 3. middle, Only using local areas for reconstruction, although it has a certain degree of robustness, it is difficult to detect significant structural changes, resulting in poor performance.

[0153] (3) Gaussian kernel width analysis: The Gaussian kernel function plays a vital role in local granularity reconstruction. Its role is to control the focus range of the attention weight, thereby obtaining correlations of different granularities. The scale parameter σ is responsible for determining the width of the Gaussian kernel curve. When σ is larger, the curve is wider, making the focus range of local granularity correlation wider.

[0154] In order to examine its contribution, Figure 5 The performance under adaptive conditions and five fixed value settings are analyzed in

[14] ; the results show that under adaptive conditions, MGRD can adapt to different data patterns and achieve the best performance. Therefore, the adaptive Gaussian bandwidth effectively enhances the fitting ability of local reconstruction, thereby improving the performance of anomaly detection.

[0155] (III) Parameter analysis

[0156] λ is the only parameter in the loss function. In order to analyze the impact of λ on the detection performance, Figure 6 The F1 score of the model is given for different λ values. As λ increases, the F1 score of the model remains stable. When λ is 6, the performance of the model improves slightly on the SMD dataset and significantly on the WADI dataset. Observation of stability with different parameter settings shows that parameter configuration does not usually reduce the performance of the model, and further refinement through experience may produce better results.

[0157] According to the settings of the MGRD model, there are three important parameters: the window size of the time series reconstruction, the number of layers in the Transformer (M), and the dimension of the embedding vector (d). The larger the window of time series reconstruction, the wider the reconstruction range of the model, but it also leads to an increase in computational complexity. Conversely, if the reconstruction window is set too small, it will lead to insufficient reference data for the global granularity sequence. Both the number of Transformer layers and the dimension of the embedding vector need to be weighed: if it is too large, the risk of overfitting increases, and if it is too small, the risk of underfitting increases.

[0158] Figure 7 The performance of the MGRD model under different reconstruction window sizes is shown. As the window size increases, the performance of the model shows a slight improvement trend. However, since an overly large window will bring higher computing requirements, considering the limitations of computing memory and time, the present invention selects a window size of 100 for all experiments.

[0159] In order to evaluate the sensitivity of model parameters, the impact of model layers M and embedding dimension d on model performance is analyzed. The experimental results are shown in Figure 8 and Fig. 9 As shown. Generally, increasing the depth and embedding dimension of the model can learn deeper information and richer feature representations, but at the cost of increasing computing time and memory usage. In the experiments of this invention, the number of model layers for all datasets is set to 3. The embedding size of PSM and WADI is 64, the embedding size of HAI and SWaT is 128, the embedding size of MSL and SMD is 256, and the embedding size of SMAP is 512.

[0160] (IV) Visualization Analysis

[0161] Provide explanations of anomaly detection results by showing visualizations of time series and anomaly scores. Fig.10 The visualization results of anomaly detection of MGRD of the present invention on PSM, SMD, HAI, and SWaT datasets are given, and each subgraph consists of two or three rows: the first row of the subgraph with highlighted areas represents the abnormal data part of the input sequence, while the second row represents the anomaly score of MGRD and highlights the areas predicted to be abnormal.

[0162] from Fig.10 As can be seen in Figure 2, MGRD successfully detects most of the regions where local structural anomalies occur. These anomalies are mainly manifested as significant deviations between the local structure and the global structure. In some regions, there are local anomalies caused by noise. However, due to the smoothing effect of local reconstruction, our method is robust to this noise. Therefore, the anomaly score does not show significant fluctuations in these noisy regions.

[0163] exist Fig.10 In (d), although most of the abnormal regions do not have local deviations, the time series pattern deviates greatly from the training data (third row). The global granular reconstruction has difficulty identifying new patterns (anomalies), resulting in large fluctuations in the corresponding anomaly scores, thus detecting anomalies in this area.

[0164] In summary, the results show that: 1. MGRD has strong local anomaly detection capabilities. 2. MGRD enhances robustness to noise through local granularity reconstruction. 3. MGRD can identify abnormal patterns (unknown patterns) through global granularity reconstruction.

[0165] Five practical anomaly detection methods

[0166] The five types of anomalies are: global point anomaly, contextual point anomaly, trend anomaly, shape anomaly, and seasonal anomaly. Fig.11The detection results of five types of anomalies. The first line shows the observed data, with the abnormal data highlighted. The local and global reconstruction outputs of MGRD are also shown to illustrate the reconstruction differences under different anomalies. The second line shows the anomaly score of MGRD, and the detected abnormal area is also highlighted.

[0167] Global anomalies: Anomalies mainly manifest as local deviations. Such anomalies, characterized by sudden changes in local structures, can be easily detected by MGRD. Local reconstruction and global reconstruction It is difficult to perfectly fit the global point abnormal area. The global reconstruction error is large, and the local reconstruction error is small. The reconstruction error between the two detects this type of abnormal point.

[0168] Contextual anomalies: Anomalies are point anomalies that occur in a specific environment. MGRD is able to detect these sudden changes involving a single point, and thus performs anomaly detection through the difference between local and global reconstructions. Similar to global anomalies, the difference between local and global reconstruction capabilities reflects this type of anomaly.

[0169] Trend anomaly: Anomaly refers to the change of the overall trend. The trend anomaly changes in a large range and has little impact on local reconstruction, but the global reconstruction that relies on the overall trend is difficult to reconstruct the area after the trend change. Therefore, there will be obvious fluctuations in the global reconstruction in the trend anomaly area.

[0170] Shape anomalies: Anomalies are caused by small changes in local shape. When the shape changes, the global reconstruction of MGRD shows the anomaly, while the local reconstruction can still adequately reconstruct the abnormal area. The difference between the two can effectively detect the anomaly in the area.

[0171] Seasonal anomaly: Anomaly is similar to shape anomaly, but covers a wider area. By properly adjusting the range of global reconstruction, the global reconstruction still has errors in the seasonal area. Since the local reconstruction has a strong fitting ability, it does not show reconstruction differences in such a large structural change, making the global and local reconstructions of MGRD diverge again and anomalies are found in this area.

[0172] In summary, MGRD effectively detects five common anomaly types through the error differences between local and global reconstructions.

[0173] 6. Performance on Univariate Datasets

[0174] In order to further evaluate the detection performance of MGRD on univariate data, the present invention selects the univariate datasets NAB and UCR for further analysis. The NAB dataset is open sourced by Numenta and contains real-world and artificially synthesized time series data. The UCR dataset is a comprehensive dataset provided for the KDD 2021 time series anomaly detection competition, including real-world and synthetic datasets. Each dataset has only one dimension, contains an anomaly segment, and is pre-divided into a test set and a training set. Following the experimental settings in TranAD, only real-world datasets are used here. The statistical information of the two datasets is shown in Table 4, and the experimental results are shown in Table 5.

[0175] Table 4 Univariate data sets

[0176]

[0177] Table 5 Performance on univariate datasets

[0178]

[0179] As shown in Table 5, MGRD can still achieve the best detection performance in the univariate dataset. On these two datasets, if the point adjustment strategy is adopted, the performance of all methods almost reaches above 0.95, and it is almost impossible to evaluate the difference between the models (the present invention still does not adopt the point adjustment strategy on the univariate dataset and directly reports the detection performance).

[0180] The results show that TranAD achieves suboptimal results on the UCR dataset, while its performance on the NAB dataset is comparable to that of GDN, TimesNet, and OmniAnomaly, indicating that it has strong generalization ability on univariate data. Some algorithms, such as DAGMM, USAD, MTAD-GAT, and CAE-M, do not work properly on the UCR dataset. The potential reason may be that implicit reconstruction easily leads to overfitting of deep models on univariate data. In contrast, the method of the present invention maintains good generalization on univariate data through global reconstruction.

[0181] (VII) Anomaly score analysis

[0182] In order to further analyze the anomaly score mechanism of the model, a segment of the FIT101 variable of the SWaT dataset was intercepted for visual analysis, because this segment contains an extremely long continuous anomaly segment, which can effectively demonstrate the anomaly detection ability of each algorithm. Fig.12 The detection results of the MGRD of the present invention were compared with those of six models: Anomaly Transformer, TranAD, USAD, TimesNet, MSCRED and CAE-M.

[0183] from Fig.12 It can be seen that in the abnormal area highlighted by the rectangular box, both MGRD and Anomaly Transformer failed to identify the early abnormal area. From the waveform of the input data, it can be seen that the area is visually similar to the normal data area. The area may be inappropriately marked as abnormal. In contrast, other algorithms will generate premature alarms, that is, in the normal data area outside the rectangular box, all algorithms will generate false positive predictions. However, the method proposed in the present invention will generate fewer false positives, indicating better robustness to noise.

[0184] The time series representation model TimesNet provides anomaly scores in the form of discrete sparse distribution, which cannot detect continuous long-term anomaly segments, resulting in an F1 score of only 0.1024. However, as shown in Table 7, under the anomaly adjustment strategy, the F1 score is 0.7296, which is improved by more than 700%, which also verifies the rationality of not using the anomaly adjustment strategy in this invention.

[0185] In summary, compared with the six baseline models, MGRD has the strongest detection ability in long-term continuous anomaly detection and is more robust to noise, producing the lowest false positive rate in normal data areas.

[0186] Since some anomaly detection methods use point adjustment strategies in the final performance report (if a single anomaly point is detected in a continuous anomaly segment, the anomaly segment is considered to be detected in its entirety), the results of using the adjustment strategy are also shown in Tables 6 and 7, with the F1 score as the main evaluation indicator. The best results are highlighted in bold, and the suboptimal F1 score results are highlighted in underline.

[0187] Table 6 Performance comparison using adjustment strategies

[0188]

[0189] Table 7 Performance comparison using adjustment strategies

[0190]

[0191]

[0192] Comparing the results in Tables 6 and 7 with those in Tables 2 and 3, it can be seen that applying the tuning strategy can significantly improve the detection performance metrics. Notably, the F1 score of SMAP improves from 0.08 to 0.8, while the F1 score of MSL improves from 0.1 to 0.69, a performance improvement of nearly 10 times. After applying the tuning strategy, Anomaly Transformer produces highly competitive results, especially achieving the best performance on SMAP and the second best performance on PSM, while also providing competitive results on the remaining datasets.

[0193] The time series representation model TimesNet shows the second best performance on the HAI, SMD and WADI datasets, and remains competitive on other datasets, further verifying the importance of robust time series representation in anomaly detection.

[0194] Even with the tuning, MGRD performs best on five of the seven datasets and second best on the remaining two. These results demonstrate the superiority of our algorithm.

[0195] (VIII) Comparison of other performance

[0196] In addition to algorithms designed specifically for anomaly detection tasks, the present invention has found through research that methods for time series prediction and feature representation are also applicable to anomaly detection tasks. The selected models include Autoformer, Crossformer, DLinear, ETSformer, Informer, MICN, Non-stationary Transformer, Pyraformer, and TimesNet.

[0197] The F1 score without using the anomaly adjustment strategy is used as the evaluation criterion, and the average F1 of all datasets is provided as the final evaluation indicator. The experimental results are shown in Table 8. The experimental results show that MGRD performs the best or second-best performance on all datasets and performs best in the average value of F1. According to the average F1 score in the last column, the second-best methods are Informer and Pyraformer, both of which are based on the Transformer architecture and are designed for long-range prediction tasks. Informer uses a probabilistic sparsity method to constrain the attention matrix, and Pyraformer uses a multi-scale pyramid structure to enhance the model's prediction ability for long- and short-distance patterns.

[0198] Table 8 Performance comparison with representation-based models

[0199]

[0200]

[0201] In summary, the present invention proposes a time series anomaly detection method based on multi-granularity reconstruction differences. The method designs a multi-granularity attention mechanism, decomposes the time series into local and global reconstructions, and detects anomalies by quantifying the differences between these reconstructions. In order to increase the degree of dissociation between the two reconstructions at the semantic level, the KL divergence constraint and the two-stage partial gradient stopping optimization strategy are integrated together, and the time series anomaly detection method of the present invention is compared with other methods. The time series anomaly detection method of the present invention obtains good detection effects and shows superior performance on multiple data sets.

[0202] Experimental evaluation of the present invention shows that MGRD outperforms the performance of ten existing anomaly detection techniques. Specifically, local granularity reconstruction enhances robustness to noise, global granularity reconstruction enhances the ability to detect previously unknown patterns (anomalies), and the deviation between these reconstructions excels in detecting mutations (local anomalies). These results demonstrate the applicability of the method in engineering and other fields.

[0203] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-granularity reconstruction difference time series anomaly detection method, characterized in that: The following steps are involved: After preprocessing the time series, a local initial input vector and a global initial input vector are obtained; Inputting the local initial input vector and the global initial input vector into a dual-path Transformer encoding module, performing local reconstruction and global reconstruction, and generating a first local reconstruction output and a first global reconstruction output; The first local reconstruction output and the first global reconstruction output are mapped through a linear layer to obtain a second local reconstruction output and a second global reconstruction output, and the anomaly detection result of the time series data is obtained by comparing the second local reconstruction output with the second global reconstruction output.

2. The multi-granularity reconstruction difference time series anomaly detection method according to claim 1 is characterized in that: The dual-path Transformer encoding module includes a plurality of dual-path Transformer encoding modules, and the plurality of dual-path Transformer encoding modules are stacked and connected; The dual-path Transformer encoding module includes a multi-granularity attention module, a first normalization layer, two convolutional layers and a second normalization layer. The dual-path Transformer encoding module is internally connected using a residual connection.

3. The multi-granularity reconstruction difference time series anomaly detection method according to claim 2 is characterized in that: The performing of local reconstruction and global reconstruction to generate a first local reconstruction output and a first global reconstruction output specifically includes: The multi-granularity attention module is used to capture local information in the local initial input vector and global information in the global initial vector to generate an intermediate local reconstruction output, an intermediate global reconstruction output, a local granularity attention matrix and a global granularity attention matrix; the intermediate local reconstruction output and the intermediate global reconstruction output are normalized by the first normalization layer, and then input into the convolution layer to extract local features and process global features respectively; the intermediate local reconstruction output and the intermediate global reconstruction output processed by the convolution layer are normalized by the second normalization layer to generate the first local reconstruction output and the first global reconstruction output.

4. The multi-granularity reconstruction difference time series anomaly detection method according to claim 3 is characterized in that: The multi-granularity attention module includes a local granularity attention submodule and a global granularity attention submodule, wherein the local granularity attention submodule outputs the local granularity attention matrix used as a local reconstruction output, and the global granularity attention submodule outputs the global granularity attention matrix used as a global reconstruction output; The local granularity attention submodule is used to learn the association weight between each data point in the time series and its adjacent data points by using the attention mechanism, and act on the local reconstruction; the local granularity attention submodule includes a Gaussian kernel function, and the Gaussian kernel function is used to enhance the fitting ability of the local attention submodule; The global granular attention submodule is used to directly learn the association weights between each data point and other data points in the time series using the attention mechanism, and act on the global reconstruction.

5. The multi-granularity reconstruction difference time series anomaly detection method according to claim 3 is characterized in that: It also includes: optimization using a loss function and a two-stage partial gradient stopping optimization strategy; The expression of the loss function is: in, is the mean square error loss term of the local reconstruction output, is the second local reconstruction output, X is the time series, is the mean square error loss term of the global reconstruction output, Reconstruct the output for the second global, is the local granular attention matrix, is the global granular attention matrix, λ is the hyperparameter of the KL divergence constraint term of the loss function, and M is the number of Transformer encoding modules.

6. The multi-granularity reconstruction difference time series anomaly detection method according to claim 5 is characterized in that: In the process of optimizing using the loss function, the root mean square error between the mean square error loss term of the local reconstruction output and the mean square error term of the global reconstruction output is also used to quantify the anomaly, and the expression is: Where D is the variable dimension of the local and global reconstruction output data, is the local reconstruction output on the i-th variable dimension, is the global reconstruction output on the i-th variable dimension.

7. The multi-granularity reconstruction difference time series anomaly detection method according to claim 5 is characterized in that: The two-stage partial gradient stop optimization strategy includes a first-stage optimization and a second-stage optimization; The first stage optimization stops the gradient back propagation on the second local reconstruction output and the local granular attention matrix, forcing the global granular attention matrix to move away from the local granular attention matrix and pay more attention to the information between non-adjacent time points in the time series; The second stage optimization stops the gradient back propagation on the second global reconstruction output and the global granular attention matrix, focuses on the local area, and pays more attention to the information between adjacent time points in the time series; The expression of the first stage optimization is: The expression of the second stage optimization is: Among them, L1 is the loss function of the first stage optimization process, and L2 is the loss function of the second stage optimization process.

8. The multi-granularity reconstruction difference time series anomaly detection method according to any one of claims 1 to 7, characterized in that: It also includes an evaluation experiment on the multi-granularity reconstruction difference time series anomaly detection method, specifically using different methods to perform anomaly detection on seven different data sets, and scoring, and comparing the obtained scoring results, where the evaluation indicators are precision, recall rate and F1 score.

Images