A lightweight time series anomaly detection method that combines point-level and sequence-level reconstruction associations

Abstract

This invention discloses a lightweight time series anomaly detection method that combines point-level and sequence-level reconstruction correlations. The method first constructs a lightweight pure MLP dual-branch reconstruction network using two MLPs with only a few parameters as the backbone to quickly learn the local and global correlations of each timestamp, and uses the difference between the local and global correlations as the reconstruction correlation. Next, correlation reconstruction learning is performed simultaneously from two complementary perspectives at the point and sequence levels, compensating for the shortcomings of the lightweight architecture through complementary learning. Finally, an anomaly scoring mechanism based on reconstruction correlations is proposed, combining point-level and sequence-level reconstruction correlations to accurately identify explicit and implicit anomalies in the time series. This lightweight time series anomaly detection method based on a pure MLP architecture achieves high speed and high accuracy using only four two-layer MLPs and can be easily deployed on resource-constrained IoT edge devices.

Description

Technical Field

[0001] This invention relates to the field of time series anomaly detection technology, and in particular to a lightweight time series anomaly detection method that combines point-level and sequence-level reconstruction correlations. Background Technology

[0002] Time series data exhibits several new characteristics: more complex and variable features, more numerous and coupled variables, and scarcer sequence labels. These characteristics lead to poor performance, or even failure, of traditional time series anomaly detection models. Specifically, traditional time series anomaly detection models based on clustering, statistics, and machine learning struggle to accurately extract complex features from time series data. Simultaneously, traditional time series anomaly detection models based on supervised and semi-supervised learning struggle to handle scarce labels. Therefore, unsupervised deep learning-based time series anomaly detection models, such as those based on convolutional neural networks (CNNs), Transformers, and recurrent neural networks (RNNs), have become a research hotspot in recent years. However, existing unsupervised deep learning-based time series anomaly detection models face two new challenges.

[0003] (1) Challenge 1: Existing anomaly scoring schemes are not powerful enough. Typically, a time series anomaly detection model requires an anomaly scoring scheme to evaluate the observations at each timestamp and determine whether they are anomalies. Existing scoring schemes can be categorized into those based on reconstruction error, prediction error, and a mixture of errors. Specifically, reconstruction error-based scoring schemes typically use the reconstruction error of each timestamp as the anomaly score. This scheme is fast and simple, but ignores the contextual information of each timestamp and its neighbors. Therefore, this scheme can cooperate with shallow neural networks to achieve high timeliness, but cannot guarantee accuracy. Prediction error-based scoring schemes use the model's prediction error as the anomaly score. This scheme considers the contextual information of each timestamp and is highly compatible with deep time series anomaly detection models. Although this type of scheme achieves high accuracy, its timeliness is poor. Mixed error-based schemes combine the above two methods, making the model more complex, and the timeliness is also unsatisfactory. Therefore, building a more powerful anomaly scoring mechanism that can balance high speed and high accuracy is a key challenge for existing time series anomaly detection models.

[0004] To address the first challenge, a novel anomaly scoring scheme based on correlation difference was proposed in 2022. This scheme posits that "compared to normal points, anomalies have stronger correlations with their local neighbors and weaker correlations with their global neighbors." Over the past two years, numerous time-series anomaly detection models based on correlation difference have been proposed and proven to have excellent performance. Typically, correlation difference-based detection models construct two branches to learn the correlations between each timestamp and its local and global neighbors, respectively, and then calculate the difference between these two correlations to determine whether a timestamp is an anomaly. For example, the AnomalyTransformer model first uses a Gaussian kernel to learn the correlation between a timestamp and its local neighbors; simultaneously, it employs a multi-head attention mechanism to learn the correlation between the timestamp and its global neighbors; finally, it designs a min-max training strategy to amplify the difference between the two correlations. Building on this, the MAN-QSM model adds a mask learning mechanism to more accurately generate correlation differences. Similarly, the DCdetector model designs a two-branch attention contrastive learning architecture, using two multi-head attention mechanisms to learn the local and global correlations of each timestamp. The Dual-TF model considers fine-grained correlation differences in both the time and frequency domains for more accurate anomaly identification. The SiET model further learns the spatial correlations of multiple variables and combines spatiotemporal correlation differences for anomaly detection. Building on this, the STAT model further considers the order of temporal and spatial feature learning to better capture spatiotemporal features and achieve more accurate anomaly detection. The GLAD model attempts to learn spatiotemporal correlation differences from both point-level and sequence-level perspectives.

[0005] In summary, anomaly scoring schemes based on correlation differences can effectively learn the long-term and short-term contextual information of each timestamp, thus effectively identifying anomalies in time series. However, correlation computation is typically a fine-grained operation, which leads to long training and detection times for these models, as well as significant CPU / GPU and storage resources. Therefore, to build a highly efficient time series anomaly detection model based on correlation differences, it is necessary to abandon the traditional bulky architecture and fine-grained computation mode, and instead adopt a lightweight, shallow architecture with fewer parameters and a coarse-grained correlation computation scheme.

[0006] (2) Challenge 2: Difficulty in simultaneously achieving high timeliness and high accuracy. The rapid development of industrial intelligence has led to increasingly finer-grained supervision of underlying equipment, forcing time-series anomaly detection models to achieve both high timeliness and high accuracy. Specifically, an ideal time-series anomaly detection model should detect anomalies as quickly and accurately as possible and provide early warnings, thus allowing time for subsequent intelligent operation and maintenance. However, high speed and high accuracy are often difficult to achieve simultaneously. For example, time-series data often exhibits complex characteristics such as periodicity, seasonality, and scale. To accurately capture these features, existing deep time-series anomaly detection models tend to use deep structures and large parameters to enhance their feature extraction capabilities. In this case, these models require significant computational and storage resources, as well as substantial training and detection time. Therefore, these models cannot achieve both high speed and high timeliness. Conversely, while traditional machine learning-based and statistical learning-based models offer high speed, their accuracy is unsatisfactory. Therefore, balancing high speed and high accuracy is a challenge for existing time-series anomaly detection systems.

[0007] To address the second challenge, many deep time-series anomaly detection models have adopted lightweight MLP networks in recent years to replace traditional time-consuming or resource-intensive modules, thereby compressing the model, reducing model parameters, and improving model timeliness. For example, the PatchAD model uses an MLP to replace the attention module in the Transformer network, thus reducing the time spent on anomaly detection. Similarly, the ANNet model uses an MLP to optimize LSTM units, thereby improving model speed. In the field of time-series prediction, the use of MLP networks is even more extensive. Several excellent time-series analysis models based on pure MLPs have been proposed and have shown superior performance. For example, the RTSF model has undergone extensive experiments to demonstrate that MLP networks can replace CNN networks and attention networks. Based on this finding, the Solar-Mixer and TS-Mix models attempt to stack multiple MLP layers to construct an All-MLP architecture for fast and accurate time-series prediction. To improve prediction performance, the HDMixer model designs a multi-layer MLP network to learn the temporal and spatial features between multiple variables. The STHMLP model further develops a hierarchical MLP network to learn multi-scale spatiotemporal features. To further utilize frequency domain information, the FreTS model employs only shallow MLP networks to learn time series features directly in the frequency domain. Inspired by the FreTS model, the FITS model directly uses MLP networks to simulate the interpolation process in the frequency domain to achieve accurate time series prediction. Notably, the FITS model has fewer than 10k parameters, achieving the dual goals of high timeliness and high accuracy.

[0008] In summary, simple MLP networks are a good choice for improving model timeliness. However, research on MLP networks in time series anomaly detection is still in its early stages and not as in-depth as in the field of time series prediction. Specifically, existing MLP-based time series anomaly detection models only utilize MLP compression to improve timeliness while maintaining high accuracy. However, the compressed models are also structurally complex and have large parameters, and the improvement in timeliness is not satisfactory. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention provides a lightweight time series anomaly detection method that combines point-level and sequence-level reconstruction correlation. It utilizes a lightweight pure MLP architecture with only shallow structure and a small number of parameters, along with a customized reconstruction difference scoring strategy, to achieve both high accuracy and timeliness in anomaly detection.

[0010] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0011] A lightweight time series anomaly detection method combining point-level and sequence-level reconstruction correlations is proposed. First, a lightweight pure MLP dual-branch reconstruction network is constructed using two two-layer MLPs as the backbone to learn the local and global correlations of each timestamp, and the difference between the local and global correlations is used as the reconstruction correlation. Next, correlation reconstruction learning is performed simultaneously from two complementary perspectives at the point and sequence levels, compensating for the shortcomings of the lightweight architecture through complementary learning. Finally, an anomaly scoring mechanism based on reconstruction correlations is designed, combining point-level and sequence-level reconstruction correlations to accurately identify explicit and implicit anomalies in the time series.

[0012] The anomaly detection method utilizes the idea that "compared to normal points, anomalies often have strong correlations with local neighbors and only weak correlations with global neighbors" to guide the implementation of lightweight anomaly detection based on a shallow pure MLP architecture.

[0013] A further improvement to the above technical solution is as follows:

[0014] Preferably, the anomaly detection method specifically includes the following:

[0015] Step S1, Point-level Reconstruction Association Learning: Construct a lightweight first dual-branch reconstruction network consisting of two two-layer MLPs, learn the local and global associations of each timestamp from a point-level perspective, and generate point-level reconstruction associations;

[0016] Step S2, Sequence-level Reconstruction Association Learning: Construct another lightweight pure MLP dual-branch network to complement the first dual-branch reconstruction network, learn the local and global associations of each timestamp from a sequence-level perspective, and generate sequence-level reconstruction associations;

[0017] Step S3, Anomaly scoring based on reconstructed associations: Combine point-level and sequence-level reconstructed associations to generate anomaly scores for each timestamp and determine whether the current timestamp is anomaly.

[0018] Preferably, step S1 specifically includes the following steps:

[0019] S1-1, Point-level sampling: First, multiple sequences of length WIN_SIZE are randomly selected in parallel from the time series; second, each sequence is further divided into... Non-overlapping segments, each segment containing A timestamp, and The product of these equals WIN_SIZE; then, The stacked fragments form a size of × A 2D matrix is ​​generated; finally, the local and global neighbors of each timestamp are quickly generated from the 2D matrix; for a current timestamp in the i-th row and j-th column, the local neighbors are composed of other timestamps in the i-th row, and the global neighbors are other timestamps in the j-th column;

[0020] S1-2, Local Association Learning: A two-layer MLP network with a lightweight dual-branch reconstruction network is used as the first branch to learn the association between each timestamp and its local neighbors;

[0021] S1-3, Global Association Learning: Use another two-layer MLP as the second branch of a lightweight dual-branch reconstruction network to learn the association between each timestamp and its global neighbors;

[0022] S1-4, Loss Function: The local association learning process and the global association learning process constitute two contrastive branches. The MSE function is used as the loss function for contrastive learning to align the reconstructed values ​​of the two branches.

[0023] Preferably, in S1-2 and S1-3, the two-layer MLP network includes a layer with... The network consists of an input layer with d neurons, a hidden layer with d neurons (default values ​​for the hidden layer being 64 or 128), and an output layer with 1 neuron. The input and hidden layers together form the first fully connected network. There are trainable parameters; the hidden layer and the output layer form a second fully connected network, with There are 1 trainable parameters; the hidden layer is equipped with dropout and ReLU activation functions.

[0024] Preferably, step S2 specifically includes the following steps:

[0025] S2-1, Sequence-level sampling: First, multiple sequences of length W_SEQ are randomly and in parallel extracted from the time series; then, each sequence is divided into... Each segment is divided into non-overlapping segments; at the same time, each segment is divided into... Non-overlapping sub-segments, each sub-segment containing A timestamp, , and The product of these equals W_SEQ; then, The stacked fragments constitute a size of The 2D matrix is ​​used; finally, the sequence-level local and global neighbors of each timestamp are generated from the 2D matrix. For a current timestamp located in the i-th row and j-th column, the sub-segment of its sequence is used as the replacement sequence, the other sub-segments in row i are used as local neighbors, and the sub-segments in column j corresponding to other timestamps are used as global neighbors.

[0026] S2-2, Local Association Learning: A two-layer MLP network is used as the first reconstruction branch to learn the association between each timestamp and its local neighbors from a sequence perspective;

[0027] S2-3, Global Association Learning: Using another two-layer MLP network as the second reconstruction branch, we learn the association between each timestamp and its global neighbors from a sequence perspective;

[0028] S2-4, Loss Function: The local association learning process and the global association learning process constitute two contrastive branches, and the MSE function is used as the contrastive learning loss function.

[0029] Preferably, in S2-2 and S2-3, the two-layer MLP network includes a layer with... The system consists of an input layer with d neurons and a hidden layer with d neurons, the hidden layer having a default value of 64 or 128, and a layer with... The output layer contains 1 neuron; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer form a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer form a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer constitute a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer constitute a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer constitute a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a second fully connected network; the hidden layer and output layer constitute a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a second fully connected network; the hidden layer and output layer constitute a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a third ... There are 10 training parameters.

[0030] Preferably, step S3 specifically includes:

[0031] S3-1, set a timestamp in the test set Simultaneously, inputs are fed into point-level and sequence-level reconstruction association learning to perform parallel learning;

[0032] S3-2, Parallel correlation calculations are performed on the M variables at the t-th timestamp in point-level reconstruction correlation learning and sequence-level reconstruction correlation learning; after the calculations are completed, M reconstruction correlation values ​​at the t-th timestamp are generated respectively.

[0033] S3-3, calculate the average of the M reconstruction association values ​​in each reconstruction association learning process to generate point-level reconstruction associations at timestamp t. Related to sequence-level reconstruction ;

[0034] S3-4, combining point-level reconstruction association and sequence-level reconstruction association to generate the anomaly score for the t-th timestamp, is:

[0035]

[0036] in, These are preset parameters, ranging from [0 to 1]; when When only sequence-level reconstruction associations are considered; when At that time, only point-level reconstruction of associations is considered; This represents the point-level reconstructed association on the m-th variable. This represents a sequence-level reconstructed association on the m-th variable;

[0037] S3-5, after obtaining the anomaly score, determine whether the t-th timestamp is an anomaly using the following formula:

[0038]

[0039] in, It is a preset threshold, ranging from 0 to 1; if , indicates that the t-th timestamp is abnormal; otherwise, the t-th timestamp is normal.

[0040] The lightweight time series anomaly detection method based on joint point-level and sequence-level reconstruction correlation provided by this invention has the following advantages compared with the prior art:

[0041] (1) The lightweight time series anomaly detection method of the present invention, which combines point-level and sequence-level reconstruction correlations, abandons the traditional approach of constructing cumbersome architectures with deep structures and large parameters. Instead, it designs a lightweight pure MLP architecture consisting of only two layers and a small number of parameters to achieve both high speed and high accuracy in anomaly detection. Specifically, the method employs two two-layer MLPs with only a small number of parameters from both point-level and sequence-level perspectives to construct a lightweight dual-branch reconstruction network for learning the point-level and sequence-level reconstruction correlations for each timestamp. This is the first lightweight time series anomaly detection model based on pure MLP.

[0042] (2) The lightweight time series anomaly detection method of the present invention, which combines point-level and sequence-level reconstruction correlation, proposes an anomaly scoring scheme based on reconstruction correlation. This scheme combines point-level and sequence-level reconstruction correlation to accurately identify explicit and implicit anomalies. This is also the first work to attempt to integrate a scoring scheme based on reconstruction error and a scoring scheme based on correlation difference. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of explicit and implicit anomalies in a time series.

[0044] Figure 2 This is a framework diagram of anomaly detection in this invention.

[0045] Figure 3 This is a schematic diagram of point-level sampling and sequence-level sampling in this invention.

[0046] Figure 4 This is a schematic diagram comparing the anomaly scoring scheme based on reconstructed association in this invention with the traditional scoring scheme.

[0047] Figure 5 This is a flowchart of the anomaly scoring process based on reconstructed association in this invention.

[0048] Figure 6(a) is a visualization of anomaly detection within timestamps 200-1200 of the first variable in the MSL dataset during the experimental verification of this invention.

[0049] Figure 6(b) is a visualization of anomaly detection within timestamps 26400-27400 of the univariate MITDB dataset in the experimental verification of this invention.

[0050] Figure 7 This is the deployment result of ten comparative models on a Raspberry Pi 4b edge device in the experimental verification of this invention. Detailed Implementation

[0051] The following provides a detailed description of specific embodiments of the present invention. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the invention.

[0052] The lightweight time series anomaly detection method of the present invention, which combines point-level and sequence-level reconstruction associations, constructs a lightweight bi-branch reconstruction network using two two-layer MLPs from both point-level and sequence-level perspectives, thereby generating point-level and sequence-level reconstruction associations for each timestamp. Then, the two reconstruction associations are combined for fast and accurate time series anomaly detection.

[0053] The lightweight time series anomaly detection method of the present invention, which combines point-level and sequence-level reconstruction correlation, is based on the principle that: using M sensors to perform T observations on a dynamic system, the collected data constitutes a time series, which can be represented as follows: A time series consists of two dimensions: the time dimension and the variable dimension. Regarding the time dimension, ,in This represents the observations from M sensors at time t. For the variable dimension, ,in This represents the observations of the m-th sensor at all T timestamps. In this invention, a time series... Divided into unlabeled training sets and labeled test sets Then, using the unlabeled training set To train a lightweight time series anomaly detection model; then use the trained model on the test set. Each timestamp in the data is used to score anomalies:

[0054] (1)

[0055] in yes The observed values ​​of M variables at the t-th time stamp;

[0056] Finally, the score of this anomaly rating is used to determine whether the t-th timestamp is an anomaly, as defined below:

[0057] (2)

[0058] in It is a preset threshold, ranging from 0 to 1; if , indicates that the t-th timestamp is abnormal; otherwise, the t-th timestamp is normal.

[0059] This invention presents a lightweight time series anomaly detection method based on joint point-level and sequence-level reconstruction correlations. It utilizes an unsupervised lightweight time series anomaly detection model (LFTSAD) based on joint point-level and sequence-level reconstruction correlations. LFTSAD abandons the traditional deep architecture with its large number of parameters, using only two two-layer MLPs to construct a lightweight pure MLP dual-branch reconstruction network. It learns the reconstruction correlations for each timestamp from both point-level and sequence-level perspectives. Furthermore, LFTSAD designs a novel scoring mechanism based on reconstruction correlations, improving model accuracy by jointly using point-level and sequence-level reconstruction correlations, thus addressing the shortcomings of the lightweight architecture.

[0060] The lightweight time series anomaly detection method of the present invention, which combines point-level and sequence-level reconstruction correlation, has the following LFTSAD model framework: Figure 2 As shown, it includes the following:

[0061] Step S1, Point-level Reconstruction Association Learning

[0062] In order to identify explicit anomalies in time series ( Figure 1 As shown in (a), a lightweight pure MLP dual-branch network is constructed to learn the local and global associations of each timestamp from a point-level perspective, and to generate the final point-level reconstructed associations, as shown in (a). Figure 2 As shown in (a) in the figure.

[0063] First, a sampling strategy is designed to generate local and global neighbors for each timestamp. Then, a lightweight dual-branch reconstruction network is constructed using two two-layer MLPs to simultaneously reconstruct the same timestamp using local and global neighbors respectively. Finally, the difference between the reconstruction values ​​of the two branches is the point-level reconstruction association for each timestamp.

[0064] Specifically, the following steps are included:

[0065] S1-1, Point-level sampling

[0066] The sampling strategy is designed to generate local and global neighbors for each timestamp. In this strategy, adjacent timestamps are considered local neighbors, while timestamps that are far apart are considered global neighbors. Figure 3 As shown in (a) above, the detailed sampling process is as follows:

[0067] First, multiple sequences of length WIN_SIZE are randomly extracted in parallel from the time series. Second, each sequence is divided into... Non-overlapping segments, each segment containing A timestamp, and The product of these equals WIN_SIZE. Next, [the product of these two numbers] will... The fragments are stacked together to form a size of × This is a 2D matrix. Finally, the local and global neighbors of each timestamp are quickly generated from this 2D matrix. For example, for a timestamp in row i and column j, its local neighbors consist of the other timestamps in row i, while its global neighbors are the other timestamps in column j.

[0068] This sampling strategy boasts advantages of high speed and low resource consumption. Furthermore, in a 2D matrix, all timestamps in the same row share the same set of local neighbors, and all timestamps in the same column share the same set of global neighbors. Therefore, all timestamps in the 2D matrix can share the same storage space, significantly reducing resource consumption. Moreover, since it is not necessary to calculate the local and global neighbors separately for each timestamp, the sampling speed is greatly accelerated. In addition, when processing multivariate time series, the sampling process for each variable is completely independent and can be executed in parallel to further accelerate the process.

[0069] S1-2, Local Association Learning

[0070] Association calculations are typically fine-grained operations, requiring the evaluation of the association between each timestamp and each neighbor. For example, if a timestamp has... If there are several neighbors, then it is necessary to calculate... This involves several correlations. However, the lightweight MLP network in this embodiment cannot perform such fine-grained computation. To address this issue, all neighbors are used to reconstruct each timestamp. If a timestamp can be accurately reconstructed, it means that it has a strong correlation with its neighbors; otherwise, the correlation is weak. This approach reduces fine-grained correlation computation to one operation per timestamp, greatly simplifying the computation process.

[0071] A two-layer MLP network is used as the first branch of the MLP dual-branch network to learn the association between each timestamp and its local neighbors. This two-layer MLP network includes a... The network consists of an input layer with 1 neuron, a hidden layer with d neurons, and an output layer with 1 neuron. The input and hidden layers form the first fully connected network. There are 1 trainable parameters. The hidden layer and the output layer form a second fully connected network, with 12 trainable parameters. There are 10 trainable parameters. To speed up training, the hidden layers use dropout and ReLU activation functions.

[0072] Time series The m-th variable in For example, the t-th timestamp is... Local neighbors are , among which The timestamps up to t share the same local neighbors.

[0073] The local association calculation process for the t-th timestamp is as follows:

[0074] S1-2-1, will Each local neighbor is input into the first fully connected network to learn relevant features.

[0075] (3)

[0076] in It is the output of the first fully connected network. This is the parameter of the first fully connected network; For activation functions; Represents the local neighbors of timestamp t.

[0077] S1-2-2, will Input to a second fully connected network for reconstruction As shown below:

[0078] (4)

[0079] in It is a locally reconstructed value. These are the parameters for the second fully connected network.

[0080] S1-3, Global Association Learning

[0081] Similar to local association learning, another two-layer MLP was used as the second branch of a lightweight dual-branch network to learn the association between each timestamp and its global neighbors. This MLP has the same structure as the first MLP, consisting only of two fully connected networks and parameter.

[0082] Furthermore, with the m-th variable For example, the t-th timestamp is... Global neighbors are All timestamps share the same global neighbor.

[0083] The global association calculation process for the t-th timestamp is as follows:

[0084] S1-3-1, will Each global neighbor is input into the first fully connected network to learn relevant features, as shown below:

[0085] (5)

[0086] in It is the output. This is the parameter for the first fully connected network.

[0087] S1-3-2, what will be learned Input into a second fully connected network for reconstruction :

[0088] (6)

[0089] in It is a global refactoring value. It is the second fully connected network parameter.

[0090] All timestamps of the same variable share two MLPs of a lightweight dual-branch network. Furthermore, all variables in the time series also share these two MLP networks.

[0091] S1-4, Loss Function

[0092] The local association learning process and the global association learning process constitute two contrastive branches. The MSE function is used as the contrastive learning loss to align the reconstructed values ​​of the two branches:

[0093] (7)

[0094] in, This represents the point-level contrastive loss, where M is the number of variables and WIN_SIZE is the window size for one sample. It is the locally reconstructed value of the m-th variable at the t-th timestamp. It is a global refactoring value.

[0095] Step S2, sequence-level reconstruction association learning.

[0096] In order to identify explicit anomalies in time series ( Figure 1 As shown in (b) of the example, this embodiment constructs another lightweight pure MLP dual-branch network as a supplement to the first dual-branch network. It learns the local and global associations of each timestamp from a sequence-level perspective, generating the final sequence-level reconstructed associations, as shown in (b). Figure 2 As shown in (b) of the diagram.

[0097] Furthermore, firstly, a sequence-level sampling strategy is adopted to generate sequence-level local and global neighbors for each timestamp; then, a lightweight dual-branch network is constructed to simultaneously reconstruct alternative subsequences for each timestamp using local and global neighbors; finally, the difference between the reconstructed values ​​of the two subsequences is the sequence-level reconstruction association for each timestamp.

[0098] In an implicit anomaly spanning multiple timestamps, some timestamps exhibit neighbor association patterns similar to normal timestamps, while others display distinctly different patterns. This means that point-level reconstruction association learning alone cannot accurately and perfectly identify all anomalous timestamps. To address this issue, this embodiment constructs a sequence-level reconstruction association learning process as a supplement to the point-level process, used for accurate identification of implicit anomalies.

[0099] S2-1, Sequence-level sampling

[0100] The sequence-level sampling strategy designed in this embodiment first starts from the time series... Multiple sequences of length W_SEQ are randomly and in parallel drawn from the sample; then, each sequence is divided into... Each segment is further divided into non-overlapping segments; at the same time, each segment is further divided into Non-overlapping sub-segments, each sub-segment containing A timestamp, , and The product of these equals W_SEQ; then, The stacked fragments constitute a size of 2D matrix, such as Figure 3 As shown in (b) in the diagram; finally, the sequence-level local and global neighbors for each timestamp can be quickly generated from this 2D matrix.

[0101] For example, for a timestamp located in row i and column j, its sub-segment is used as the replacement sequence, other sub-segments in row i are used as its local neighbors, and sub-segments in column j corresponding to other timestamps are used as its global neighbors.

[0102] S2-2, Local Association Learning

[0103] Similar to point-level learning, a two-layer MLP network is used as the first branch to learn the association between each timestamp and its local neighbors from a sequence perspective. Unlike previous MLPs, this MLP has an input layer with... There are d neurons in the hidden layer and d neurons in the output layer. There are 100 neurons. The input layer and hidden layer form the first fully connected network, and the hidden layer and output layer form the second fully connected network. The MLP network contains two fully connected networks and 100 neurons. One parameter.

[0104] Furthermore, with the m-th variable For example, the t-th timestamp is... Its substitution subsequence is Local neighbors are In this subsequence, all timestamps share the same local neighbors. The sequence-level local association learning process for the t-th timestamp is as follows:

[0105] S2-2-1, will Each local neighbor is input into the first fully connected network to learn association features, as follows:

[0106] (8)

[0107] in It is the output of the first fully connected network. These are the parameters of the first fully connected network.

[0108] S2-2-2, will The input is fed into a second fully connected network to reconstruct the alternative subsequence of the t-th timestamp. .

[0109] (9)

[0110] in, It is a locally reconstructed value. These are the parameters of the second fully connected network.

[0111] S2-3, Global Association Learning

[0112] A two-layer MLP network is used as the second branch to learn sequence-level global correlations. In this MLP, the input layer has... There are d neurons in the hidden layer and d neurons in the output layer. This MLP has a total of [number] neurons. parameter.

[0113] Taking the t-th timestamp of the m-th variable as an example, its substitution subsequence is: Its global neighbors are The sequence-level global association learning process for the t-th timestamp is as follows:

[0114] S2-3-1, will Global neighbors are input into the first fully connected network.

[0115] (10)

[0116] in This is the output of the first fully connected network. These are the parameters of the first fully connected network.

[0117] S2-3-2, will The input is fed into a second fully connected network to reconstruct the alternative subsequence of the t-th timestamp.

[0118] (11)

[0119] in It is a global refactoring value. These are the parameters of the second fully connected network.

[0120] S2-4 uses the MSE function as the contrastive learning loss function.

[0121] (12)

[0122] in This represents the sequence-level contrastive loss, where M is the number of variables and W_SEQ is the window size for one sample. It is a locally reconstructed value. It is a global refactoring value.

[0123] Step S3: Anomaly scoring based on reconstructed associations.

[0124] The anomaly scoring scheme in this embodiment combines point-level and sequence-level reconstruction correlations to generate a final anomaly score for each timestamp and determine whether it is an anomaly. This embodiment's scoring scheme is the first to attempt to fuse a reconstruction error-based scoring scheme and a correlation difference-based scoring scheme, adapting it to a lightweight MLP-based network architecture, thereby serving the lightweight model of this embodiment. Figure 4 As shown.

[0125] like Figure 5 As shown, the detailed process of the anomaly scoring scheme in this embodiment is as follows:

[0126] S3-1, set a timestamp in the test set Simultaneously, inputs are fed into point-level and sequence-level reconstruction association learning to perform parallel learning;

[0127] In S3-2, parallel correlation calculations are performed on the M variables at the t-th timestamp in both point-level and sequence-level reconstruction correlation learning. After the calculations are completed, M reconstructed correlation values ​​at the t-th timestamp are generated respectively.

[0128] S3-3, calculate the average of the M reconstruction association values ​​in each reconstruction association learning process to generate point-level reconstruction associations at timestamp t. Related to sequence-level reconstruction ;

[0129] S3-4, combining point-level reconstruction association and sequence-level reconstruction association to generate the anomaly score for the t-th timestamp, is:

[0130] (13)

[0131] in, These are preset parameters, ranging from [0 to 1]; when When only sequence-level reconstruction associations are considered; when At that time, only point-level reconstruction of associations is considered; This represents the point-level reconstructed association on the m-th variable. This represents a sequence-level reconstructed association on the m-th variable;

[0132] S3-5, after obtaining the abnormal score, determine whether the t-th timestamp is abnormal by formula (2).

[0133] This invention proposes an unsupervised, lightweight time-series anomaly detection method that combines point-level and sequence-level reconstruction correlations. It abandons the traditional approach of constructing cumbersome architectures with deep structures and large parameters, instead designing a lightweight architecture composed of shallow structures and a small number of parameters to balance high speed and high accuracy. First, a lightweight bi-branch reconstruction network is constructed using two MLPs with only two layers and a small number of parameters to quickly learn the local and global correlations of each timestamp, using the difference between the local and global correlations as the reconstruction correlation. Second, based on the lightweight pure MLP bi-branch reconstruction network, two complementary point-level and sequence-level reconstruction learning mechanisms are designed to improve model accuracy and mitigate the shortcomings of the lightweight architecture. Finally, an anomaly scoring mechanism based on reconstruction correlations is proposed, combining point-level and sequence-level reconstruction correlations to more accurately identify anomalies.

[0134] Experimental verification

[0135] (1) Experimental data

[0136] The experiment used 13 publicly available real-world time series datasets, including 6 univariate datasets and 7 multivariate datasets, as detailed in Table 1. These datasets span multiple fields, such as industry, finance, healthcare, and manufacturing, to ensure the fairness and impartiality of the experimental results.

[0137] 1) Multivariate Datasets. The MSL dataset captures the operational status of multiple sensors or controllers on the Mars rover. The GECCO dataset stores drinking water quality data from multiple IoT sensors. The SWAN dataset contains temporal data from the solar photosphere vector magnetogram from the Spaceweather HMI active area patch series. The PSM dataset contains 25-dimensional monitoring information from eBay Server Machines. The SMAP dataset displays soil samples and telemetry data collected by the Mars rover. The SMD dataset stores resource utilization tracking records from 28 servers over five weeks. The SWaT dataset consists of 51-dimensional data collected by multiple sensors in a public water treatment infrastructure. In summary, these seven datasets cover fields such as aviation, commerce, water treatment, and telemetry.

[0138] 2) Univariate datasets. The ECG dataset includes ECG data with anomalous premature ventricular contractions (PVCs). The MITDB dataset contains 48.5 hours of dual-channel dynamic ECG recordings. The SVDB dataset stores 78 half-hour ECG recordings with supraventricular arrhythmias. The UCR dataset is a life-sensing time series with a single anomalous arrhythmia. The UCR-AUG dataset is augmented data from the original UCR dataset. The Occupancy dataset is a binary classification dataset based on temperature, humidity, light intensity, and CO2 levels. In summary, these six datasets cover fields such as nature, environment, and medicine.

[0139] Table 1 Experimental Dataset

[0140]

[0141] (2) Comparison Models and Evaluation Indicators

[0142] 1) Comparison Models: To verify the effectiveness of the LFTSAD model in this embodiment, nine representative anomaly detection models were selected as competitors based on three aspects: model structure, scoring scheme, and MLP network usage, as shown in Table 2. From a network structure perspective, DCdetector, PatchAD, AnomalyTrans, DTAAD, TranAD, USAD, MSCRED, and MTGFLOW are all deep time-series anomaly detection models with multi-layered structures and a large number of parameters. In contrast, IFOrest and LFTSAD are time-series anomaly detection models with fewer parameters and shallower structures. Regarding anomaly scoring schemes, LFTSAD, AnomalyTrans, DCdetector, and PatchAD utilize anomaly scoring schemes based on correlation differences. Other models rely on scoring schemes based on reconstruction error or prediction error. In terms of whether an MLP network is used, both LFTSAD and PatchAD use MLP networks as their backbone, with LFTSAD using a lightweight network and PatchAD using a deep network.

[0143] 2) Evaluation Metrics: To comprehensively evaluate the experiment, four widely used evaluation metrics were selected: Accuracy (ACC), Precision, Recall, and F-Score. Specifically, ACC represents the proportion of correctly predicted samples to the total number of samples. Precision indicates how many of the predicted normal timestamps are actually normal. Recall describes how many correctly predicted timestamps are actually normal. The F-Score combines Precision and Recall, providing a more comprehensive evaluation. All four evaluation metrics range from 0 to 1; higher values ​​indicate better performance in time series anomaly detection.

[0144] Table 2 Comparison Models

[0145]

[0146] (3) Accuracy Experiment

[0147] Experiments were conducted on 10 models on 6 univariate datasets and 7 multivariate datasets, with the results of each experiment being the average of 5 identical experiments.

[0148] 1) Performance on univariate datasets: Table 3 lists the anomaly detection accuracy of 10 models on 6 univariate datasets. The following observations can be easily made from the table.

[0149] I. Focus on overall performance: LFTSAD performs best, followed by DTAAD, PatchAD, AnomalyTrans, and DCdetector, while TranAD, IFOrest, USAD, MSCRED, and MTGFLOW perform worst. For example, on the MITDB dataset, LFTSAD has an ACC of 0.9963 and an F-Score of 0.9314, while TranAD only has an ACC of 0.8370 and an F-Score of 0.8850.

[0150] II. Focus on Stability: LFTSAD showed the most stability across the four evaluation metrics, followed by DTAAD, PatchAD, and AnomalyTrans. For example, on the ECG and UCR datasets, LFTSAD exhibited a stable ranking of three first-place and one second-place results.

[0151] III. Consider anomaly scoring schemes: Time series anomaly detection models based on correlation differences, such as LFTSAD, DCdetector, AnomalyTrans, and PatchAD, outperform models based on other scoring schemes. For example, on the ECG dataset, DCdetector and AnomalyTrans outperform DTAAD (accuracy 0.4814) and USAD (accuracy 0.4830).

[0152] IV. Comparing Model Structures: Deep models (PatchAD, AnomalyTrans, DCdetector, TranAD, USAD, MSCRED, MTGFLOW) generally outperform shallow models (IForest). However, the shallow model LFTSAD surpasses several deep models. For example, on the SVDB dataset, PatchAD has an accuracy of 0.9245, LFTSAD has an accuracy of 0.6905, DTAAD has an accuracy of 0.6809, while IFOreest only has an accuracy of 0.4409.

[0153] Table 3 shows the performance of multiple models on univariate datasets.

[0154]

[0155] 2) Performance on multivariate datasets: Table 4 shows the anomaly detection performance of the 10 models on 7 multivariate datasets. Several similar results can be obtained from the table.

[0156] I. Focus on overall performance: LFTSAD consistently demonstrates a high level of performance, followed by DCdetector, AnomalyTrans, and MSCRED. For example, on the MSL dataset, LFTSAD ranks first with an ACC of 0.9907 and an F-score of 0.9658, while IFOrest only has an ACC of 0.3764 and an F-score of 0.4314.

[0157] II. Focus on Stability: LFTSAD, DCdetector, and MSCRED exhibited relatively stable performance across four metrics. For example, on the SMAP dataset, LFTSAD showed an accuracy of 0.9590 and an ACC of 0.9000, while USAD and IFOrest showed very large fluctuations, especially IFOrest, whose ACC varied significantly.

[0158] III. Considering anomaly scoring schemes: LFTSAD, DCdetector, and AnomalyTrans outperform models based on other schemes. For example, on the SMAP dataset, LFTSAD has an ACC of 0.9000, DTAAD has an ACC of 0.8874, and USAD has an ACC of 0.8134.

[0159] IV. Comparison of Model Structures: The shallow LFTSAD model outperforms several deep models. For example, on the PSM dataset, LFTSAD has an accuracy of 0.9785 and an F-score of 0.9848, which is better than deep models such as AnomalyTrans (accuracy of 0.9545, F-score of 0.9825) and TranAD (accuracy of 0.9784, F-score of 0.9737).

[0160] Table IV: Performance of multiple models on multivariate datasets

[0161]

[0162] 3) Visualization of performance: The anomaly detection results of all models at a given timestamp were visualized on two real-world datasets, MSL and MITDB, as shown below. Figure 4 As shown in the figure. In the graph, the black line represents the actual data curve, and the red or blue line represents the corresponding 0 / 1 label, where a value of 1 indicates an anomaly and a value of 0 indicates normal operation. Figure 4 In (a) and (b), the ground-truth represents the real data and the true labels, while the other subplots represent the predicted labels from different models. From the figures, two conclusions can be drawn:

[0163] I. Focus on prediction accuracy: LFTSAD, DCdetector, AnomalyTrans, and DTAAD are closer to the true labels than other models. For example, on the MSL dataset, these models can identify anomalies well in the 550–750 timestamp range, while other models produce a large number of errors.

[0164] II. Considering the issue of false positives: LFTSAD, DCdetector, AnomalyTrans, and DTAAD produce fewer false positives than other models. For example, on the MITDB dataset, DTAAD produced one false positive, LFTSAD produced two false positives, DCdetector produced three false positives, while other models produced a large number of false positives.

[0165] 4) Summary: Experimental results demonstrate that the LFTSAD algorithm has excellent anomaly detection performance. Specifically, the LFTSAD model consistently ranks first or second in four metrics across multiple datasets, outperforming several deep models.

[0166] (4) Timeliness Experiment

[0167] The effectiveness of 10 models was compared on the MSL dataset from two aspects: the number of model parameters and the processing time per epoch. Generally, the timeliness and resource consumption of neural network-based anomaly detection models mainly depend on the number of training parameters and the processing time per epoch with the same batch size. Although the total number of training epochs also affects the model's speed, it is excluded because the number of epochs in different models is arbitrarily set and cannot accurately reflect the model's true processing time. Specifically, the larger the number of training parameters, the greater the computational (CPU or GPU) and storage resource consumption, the longer the parameter update time, and the longer the training time. Similarly, if the processing time per epoch is long, the model's training time is longer, the detection time is also longer, and the model's timeliness is worse.

[0168] Table 5 Timeliness Analysis

[0169]

[0170] Table 5 lists the timeliness of the 10 models.

[0171] I. Focus on Parameter Count: LFTSAD has the fewest parameters among all evaluation models, with only 12.6K parameters, significantly fewer than other models. For example, the AnomalyTrans model has 4741K parameters, MSCRED has 1237.3K parameters, and PatchAD has 1477.9K parameters. Therefore, LFTSAD requires less CPU / GPU computing and storage resources and is very fast. These superior characteristics enable it to perform well even in resource-constrained industrial IoT scenarios.

[0172] II. Considering the processing time per epoch: LFTSAD boasts excellent timeliness, requiring only 26.5 seconds per epoch. In contrast, deep learning models require significantly more time per epoch, ranging from several times to tens of times longer than LFTSAD. For example, MTGFLOW takes 105.2 seconds, AnomalyTrans takes 201.1 seconds, and PatchAD takes 905.8 seconds. Notably, IFOrest, a zero-parameter time series anomaly detection model based on machine learning, requires only 19.7 seconds per epoch.

[0173] In summary, LFTSAD has fewer parameters and the fastest training speed, and its timeliness surpasses most time series anomaly detection models.

[0174] (5) Deployability Experiment

[0175] This embodiment demonstrates the deployment of ten models on a resource-constrained Raspberry Pi 4b edge device. The Raspberry Pi 4b is equipped with a 1.5GHz ARM Cortex-A72 processor and 2GB of RAM. In this experiment, all models were first trained in the cloud before being deployed to the edge device. Figure 7 The deployment results on a Raspberry Pi 4b are shown. Two results can be observed from the graph.

[0176] I. Deployability: Eight models were successfully deployed. The PatchAD and AnomalyTrans models could not be deployed due to their excessive memory requirements. In contrast, the shallow model deployment process was simpler and easier.

[0177] II. Resource Consumption: In deployment and operation with 100 timestamps, the shallow model consumes significantly less processing time, CPU utilization, and RAM utilization than the deep model, ensuring that it can run smoothly and seamlessly on edge devices with limited resources.

[0178] III. Convenience: In an Industrial Internet of Things (IIoT) environment, production processes change frequently, causing dynamic fluctuations in time signals. Faced with these dynamically changing IIoT time signals, all models must be periodically fine-tuned in the cloud before being updated to edge devices. In comparison, shallow model updates are easier and consume fewer resources, while deep model updates are cumbersome and consume more resources.

[0179] The above embodiments are merely preferred examples of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A lightweight time series anomaly detection method that combines point-level and sequence-level reconstruction-related, characterized in that, The anomaly detection method specifically comprises the following contents: Step S1, point-level reconstruction association learning: a lightweight first double-branch reconstruction network composed of two two-layer MLPs is constructed to learn the local association and global association of each timestamp from a point-level perspective and generate point-level reconstruction association; Step S2, sequence-level reconstruction association learning: another lightweight double-branch reconstruction network of pure MLP is constructed to form a complement with the first double-branch reconstruction network, learn the local association and global association of each timestamp from a sequence-level perspective, and generate sequence-level reconstruction association; Step S3, anomaly scoring based on reconstruction association: the anomaly score of each timestamp is generated by combining the point-level and sequence-level reconstruction associations, and it is determined whether the current timestamp is abnormal; In the step S3, specifically: S3-1, collect one timestamp in the test set Parallel learning is performed by simultaneously inputting into point-level and sequence-level reconstruction associated learning; S3-2, in the point-level reconstruction association learning and sequence-level reconstruction association learning, the M variables of the tth timestamp are calculated in parallel; After the calculation, M reconstruction association values of the tth timestamp are generated respectively; S3-3, for each reconstruction association value in the reconstruction association learning, calculate the average value thereof, to generate a point-level reconstruction association of time stamp t and a sequence-level reconstruction association ; S3-4, the anomaly score of the tth timestamp is generated by combining the point-level reconstruction association and the sequence-level reconstruction association, which is: ； wherein, is a preset parameter, ranging from [0~1]; when only sequence-level reconstruction association is considered; when only point-level reconstruction association is considered; represents point-level reconstruction association on the mth variable, represents sequence-level reconstruction association on the mth variable; S3-5, after obtaining the anomaly score, whether the tth timestamp is abnormal is determined by the following formula: wherein, is a preset threshold value ranging from 0 to 1; if , it indicates that the tth time stamp is abnormal; otherwise, the tth time stamp is normal.

2. The joint point-level and sequence-level reconstruction related light-weight time series anomaly detection method according to claim 1, characterized in that, In the step S1, specifically comprising the following steps: S1-1, Point-level sampling: First, multiple sequences with length WIN_SIZE are randomly drawn from the time series in parallel; second, each sequence is further divided into non-overlapping segments, each segment contains time stamps, and the product of which equals WIN_SIZE; then, the segments are stacked to form a 2D matrix with size × ; finally, the local neighbors and global neighbors of each time stamp are quickly generated from the 2D matrix; for a current time stamp in the i-th row and j-th column, the local neighbors consist of other time stamps in the i-th row, and the global neighbors are other time stamps in the j-th column; S1-2, local association learning: a two-layer MLP network is used as the first branch of the lightweight double-branch reconstruction network to learn the association of each timestamp with its local neighbors; S1-3, global association learning: another two-layer MLP is used as the second branch of the lightweight double-branch reconstruction network to learn the association of each timestamp with global neighbors; S1-4, loss function: the local association learning process and the global association learning process constitute two contrast branches, and the MSE function is used as the loss function of contrast learning to align the reconstruction values of the two branches.

3. The joint point-level and sequence-level reconstruction related light-weight time series anomaly detection method according to claim 2, characterized in that, In S1-2 and S1-3, the two-layer MLP network includes an input layer with one neuron, a hidden layer with d neurons, where the default value of d is 64 or 128, and an output layer with one neuron; the input layer and the hidden layer form a first fully connected network with trainable parameters; the hidden layer and the output layer form a second fully connected network with trainable parameters; and the hidden layer is used with dropout and ReLU activation functions.

4. The joint point-level and sequence-level reconstruction related light-weight time series anomaly detection method according to claim 2, characterized in that, In the step S2, specifically comprising the following steps: S2-1, sequence-level sampling: first, multiple sequences of length W_SEQ are randomly and parallelly drawn from the time series; then, each sequence is divided into non-overlapping segments; meanwhile, each segment is divided into non-overlapping sub-segments, each containing time stamps, , and the product of which equals W_SEQ; next, the segments are stacked to form a 2D matrix of size ; finally, the sequence-level local neighbors and global neighbors of each time stamp are generated from the 2D matrix, for a certain current time stamp located at the i-th row and j-th column, using the sub-segments of its belonging sequence as surrogate sequences, other sub-segments in the row i as local neighbors, and sub-segments in the column j corresponding to other time stamps as global neighbors; S2-2, local association learning: a two-layer MLP network is used as the first reconstruction branch to learn the association of each timestamp with local neighbors from a sequence perspective; S2-3, global association learning: another two-layer MLP network is used as the second reconstruction branch to learn the association of each timestamp with global neighbors from a sequence perspective; S2-4, loss function: the local association learning process and the global association learning process constitute two contrast branches, and the MSE function is used as the contrast learning loss function.

5. The joint point-level and sequence-level reconstruction related light-weight time series anomaly detection method according to claim 4, characterized in that, In S2-2 and S2-3, the two-layer MLP network includes a network with... The system consists of an input layer with d neurons and a hidden layer with d neurons, the hidden layer having a default value of 64 or 128, and a layer with... The output layer contains 1 neuron; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer form a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer form a second fully connected network; the hidden layer uses dropout and ReLU activation functions, and the two-layer MLP network contains 1 neuron output layer; the input layer and hidden layer constitute a first fully connected network; the hidden layer and output layer constitute ... There are 10 training parameters.

Images