Multi-stage anomaly detection system based on multivariable time series data

Abstract

The invention relates to the field of data anomaly detection, and particularly discloses a multi-stage anomaly detection system based on multivariable time sequence data, which comprises an input data acquisition module used for acquiring time sequence multivariable data; the MTS-Mixer spatio-temporal feature extraction module is used for extracting multi-level time sequence correlation features through a feature mixing technology and generating low-rank compression prediction features; the time sequence anomaly detection module is used for realizing anomaly positioning and classification by adopting alternate stacking of an Angle-Attention layer and a feed-forward network, expanding anomaly differences in combination with prior constraints and adversarial learning and fusing reconstruction errors and association differences to generate anomaly scores; and the attention mechanism soft thresholding module is used for suppressing noise, enhancing key signals and outputting high signal-to-noise ratio representation. According to the method, multiple technologies are integrated, and real-time anomaly detection and interpretable positioning of the high-dimensional time series data are realized through multi-stage collaborative optimization.

Description

Technical Field

[0001] The present application relates to the field of data anomaly detection, and more specifically, to a multi-stage anomaly detection system based on multivariate time series data. Background Art

[0002] Anomaly detection analyzes massive amounts of data to identify unexpected, anomalous behaviors that often pose a threat to the system. It is widely used in complex systems such as smart energy, data centers, and manufacturing. These scenarios deploy numerous sensors and actuators, generating complex data with time-series characteristics. As processes and business logic become more complex, data patterns become more subtle, making the identification of anomalous behavior even more crucial.

[0003] Industrial production accumulates a vast amount of multivariate time series data. These data are subject to noise, volatility, and imbalance, making them difficult to effectively address with traditional anomaly detection methods. For example, clustering methods such as K-Means and DBSCAN have difficulty determining parameters and lack the ability to leverage temporal order information. The local outlier factor (LOF) method has shortcomings in processing high-dimensional data and interpreting results. Ensemble learning-based methods are prone to overfitting and poor generalization. In contrast, neural networks can capture temporal and spatial correlations, and deep learning methods such as autoencoders, LSTMs, GANs, and VAEs have demonstrated excellent performance in anomaly detection.

[0004] In recent years, the Transformer architecture has emerged as a leader in multivariate time series anomaly detection thanks to its self-attention mechanism. It efficiently processes sequential data, captures long-range dependencies, and overcomes the gradient issues of traditional RNNs. In multivariate time series analysis, the Transformer can simultaneously focus on information from multiple variables at different points in time, improving the accuracy of anomaly detection. For example, in smart grids, the Transformer can effectively capture the inherent connections between variables such as voltage, current, and power.

[0005] However, current Transformer-based anomaly detection methods still face challenges: first, the computational efficiency is low when processing high-dimensional complex data, making it difficult to meet real-time requirements; second, the ability to mine, locate, and interpret abnormal patterns is insufficient, making it difficult to accurately point out abnormal variables and time points, and difficult to explain the causes of abnormalities, limiting their practical application and promotion.

[0006] Therefore, a multi-stage anomaly detection system based on multivariate time series data is provided. Summary of the Invention

[0007] In order to solve the above technical problems, this application is proposed.

[0008] Specifically, according to one aspect of the present application, a multi-stage anomaly detection system based on multivariate time series data is provided, which includes:

[0009] An input data acquisition module is used to acquire multivariate data based on time series as input data;

[0010] The MTS-Mixer spatiotemporal feature extraction module is used to extract multi-level temporal correlation features based on the MTS-Mixer feature mixing technology, and generate low-rank compressed prediction features in combination with the linear projection layer;

[0011] The time series anomaly detection module dynamically models multi-level time series associations by alternating between stacked anomaly-attention layers and feedforward networks. It combines prior association constraints with minimax association learning to amplify anomaly differences and fuses reconstruction errors with association differences to generate point-by-point anomaly scores for anomaly location and classification.

[0012] The attention mechanism soft thresholding module is used to suppress low-correlation noise through dynamic soft thresholding operations, enhance key signals, and output attention representations with high signal-to-noise ratio.

[0013] Preferably, the MTS-Mixer spatiotemporal feature extraction module includes: using a time filter to capture the dependency of the time dimension; using a channel filter to capture the channel dependency; and outputting the final prediction result through a linear projection layer Linear(·). The specific formula is expressed as follows:

[0014]

[0015] in, is the final prediction result; It is the time feature representation; is the channel dependency after denoising.

[0016] Among them, the time filter is used to capture the dependencies of the time dimension, including:

[0017] The input time series data is normalized and downsampled using the norm(·) function and sample(·) function respectively to obtain multiple subsequences χ h,1 ,…,χ h,s , the specific formula is expressed as:

[0018] χ h,1,…, χ h,s =sample(norm(χ h ))

[0019] in, Represents the input time series data, n represents the data length, and c represents the data dimension size;

[0020] Extract the temporal dependencies of each subsequence χ through the Temporal(·) method, obtaining the temporal feature representation h,i Specifically, the formula is as follows:

[0021]

[0022]

[0023] Merge the temporal features extracted from all subsequences through the merge(·) method[[ID=!16]]

[0024]

[0025]

[0026]

[0027] [[ID=2#7]]

[0028] Perform channel redundancy processing on the input time series data through matrix factorization method. Specifically, the formula is as follows:<00001!\1>

[0027] [[ID=!33]]

[0028] where N ∈ R n×c represents noise; represents the channel correlation after denoising; U ∈ R n×m , V ∈ R m×c (m < c) represents the decomposed channel interaction;

[0029] Calculate the channel dependencies after denoising through factorization method Specifically, the formula is as follows:

[0030]

[0031] where W1 ∈ R m×c and W2 ∈ R c×m represent weight matrices; σ represents the activation function; b1 and b2 represent bias terms (BiasTerms).

[0032] Preferably, in the temporal anomaly detection module, dynamically model multi-level temporal associations by alternately stacking Anomaly-Attention layers and feed-forward networks, including: *

[0033] Calculate the prior association weights between time points through a learnable Gaussian kernel function, combined with the dynamic scale parameter σ, generating a prior association matrix P in the form of a discrete distribution It should be noted that there are some tags in the original text that seem to be incorrect or have special formatting requirements that are not entirely clear. I have tried my best to translate according to the rules. If there are any further issues or need for clarification, please let me know.l , the specific formula is expressed as:

[0034]

[0035] Among them, ‖ji‖ 2 Indicates time distance, represents the scaling factor, N represents the sequence length, represents the σ parameter matrix in the lth layer, d model Represents the model dimension;

[0036] Calculate the sequence correlation matrix S based on the self-attention mechanism l , dynamic time dependency is captured by Softmax normalization, and the specific formula is expressed as:

[0037] Initialization: Q, K, V,

[0038] Sequence correlation matrix:

[0039] in, Represent the query, key, and value in the self-attention mechanism respectively. are the parameter matrices of Q, K, and V in the lth layer respectively.

[0040] reconstruction:

[0041] in, It is the hidden representation after being processed by the abnormal attention mechanism of the lth layer.

[0042] Preferably, in the time series anomaly detection module, combining a priori correlation constraints with minimax correlation learning to expand anomaly differences includes:

[0043] For the prior correlation matrix P l With the sequence correlation matrix S l Perform symmetrized KL divergence calculation and average the association differences of multiple layers to generate the association difference metric AssDis. The specific formula is:

[0044]

[0045] Where AssDis(P, S; χ) represents the point-wise association difference between the prior association P and the serial association S, Represents data χ at the i-th time point P l and S l The KL divergence calculated between;

[0046] Through the mini-maximum association learning method, additional loss is used to expand the difference between normal and abnormal time points. The specific loss calculation formula is:

[0047]

[0048] in, represents the reconstruction result of data χ, and ‖·‖ F and‖·‖ k represents the Min-Frobenius norm and K-norm; λ is used to weigh the two loss terms.

[0049] Preferably, in the time series anomaly detection module, the reconstruction error and the correlation difference are fused to generate a point-by-point anomaly score to achieve anomaly location and classification, including:

[0050] The weighted Softmax results of the fusion reconstruction error and the associated difference are used to generate the point-by-point anomaly score AnomalyScore to locate and classify anomalies. The specific formula is expressed as follows:

[0051]

[0052] Here, AnomalyScore(χ) represents the point-by-point anomaly determination criterion of χ, and ⊙ represents element-wise multiplication.

[0053] Preferably, the attention mechanism soft thresholding module includes:

[0054] Receive the attention representation from the time series anomaly detection module 130, that is, the weight distribution after feature extraction, recorded as input value x;

[0055] Apply a soft thresholding operation to the input value x, and remove low-correlation weights by setting the threshold β. The specific formula of the soft thresholding operation is expressed as:

[0056] y=sign(x)·max(0,|x|-β)

[0057] Among them, x is the input value, sign(x) is the sign function, β is the set threshold, y is the output value after soft thresholding, and max(0,|x|-β) is the ReLU activation function.

[0058] The sign(x) function is used to restore the sign of the input value x, and its specific operation formula is expressed as:

[0059]

[0060] Compared with the existing technology, the multi-stage anomaly detection system based on multivariate time series data provided by this application has the following significant effects:

[0061] (1) The MTS-Mixer framework is used to separate time and channel dimension feature extraction, combined with linear projection compression, which significantly reduces the computational complexity of the Transformer and solves the problem of insufficient real-time performance of high-dimensional data.

[0062] (2) Through the multi-level dynamic modeling capability of the Anomaly-Attention layer and the mini-max correlation learning mechanism, the difference representation between normal and abnormal modes is enhanced, and the anomaly location accuracy and classification interpretability are improved through prior constraints.

[0063] (3) By using soft thresholding of the attention mechanism to suppress noise interference, not only the feature signal-to-noise ratio is optimized, but also the abnormal time window and key variables can be accurately located.

[0064] (4) While maintaining the Transformer's advantage in capturing long-range dependencies, the overall system effectively solves the three core problems of low computational efficiency of high-dimensional data, poor interpretability of abnormal patterns, and insufficient positioning accuracy through hierarchical feature decoupling and association constraint mechanisms, achieving a balance between high-precision real-time detection and explainable abnormality diagnosis in complex industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0066] Figure 1 The figure shows an algorithm flow chart according to an embodiment of the present application.

[0067] Figure 2 The figure shows a system block diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] Below, the embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0069] Example:

[0070] Figure 1 FIG1 illustrates an algorithm flow chart according to an embodiment of the present application, Figure 2 The system block diagram according to the embodiment of the present application is shown in FIG. Figure 1As shown in Figure 1, the system is an end-to-end multi-stage anomaly detection framework for multivariate time series data. Its core goal is to solve the problem of efficient anomaly detection in complex multivariate time series data in industrial scenarios through hierarchical feature extraction, dynamic association modeling and noise suppression. The system consists of three core modules (i.e. Figure 2 The MTS-Mixer spatiotemporal feature extraction module 120, the temporal anomaly detection module 130, and the attention mechanism soft thresholding module 140 shown in the figure are connected in series to form a collaborative process of "feature extraction → anomaly location → signal enhancement", which is highly robust, interpretable, and real-time.

[0071] Specifically, if Figure 2 As shown, according to an embodiment of the present application, a multi-stage anomaly detection system 100 based on multivariate time series data includes: an input data acquisition module 110 for acquiring multivariate data based on time series as input data; an MTS-Mixer spatiotemporal feature extraction module 120 for extracting multi-level time series correlation features based on the MTS-Mixer feature mixing technology, and generating low-rank compressed prediction features in combination with the linear projection layer; a time series anomaly detection module 130 for dynamically modeling multi-level time series associations by alternately stacking Anomaly-Attention layers and feedforward networks, combining prior association constraints and adversarial learning to expand anomaly differences, and fusing reconstruction errors and association differences to generate point-by-point anomaly scores to achieve anomaly location and classification; an attention mechanism soft thresholding module 140 for suppressing low-correlation noise, enhancing key signals, and outputting attention representations with a high signal-to-noise ratio through dynamic soft thresholding operations.

[0072] In an embodiment of the present application, the input data acquisition module 110 is used to obtain multivariate data based on time series as input data. It should be understood that the setting of the input data acquisition module 110 is the basis for the operation of the system, because the multivariate time series data is the original record of the dynamic interaction between sensors and actuators in the industrial scene, and directly carries the complete state information of the system operation (such as the time series changes of multi-dimensional variables such as voltage, current, and temperature). This step provides the subsequent modules with structured input containing time dependence, complex associations between variables, and potential abnormal patterns by collecting multivariate time series data, so that the system can be modeled based on the dynamic characteristics of the real scene. If this step is skipped, the subsequent spatiotemporal feature extraction, abnormality location, and signal optimization will not be able to capture real abnormal behavior due to the lack of original data support, especially the subtle abnormal patterns under the noise mask that traditional methods are difficult to handle. In addition, the imbalance and high-dimensional characteristics of multivariate time series data need to be targeted by subsequent modules (such as MTS-Mixer and attention mechanism), and the input module provides the necessary data foundation for such complex processing, ensuring that the system can mine deep abnormal associations from the original signal. In addition, data from sensors, logs, and device status in industrial systems comes from a variety of sources, with widely varying formats and sampling frequencies. Therefore, in the embodiments of this application, it is also possible to unify data formats and align timestamps to form a standardized time-series data stream, providing consistent input for subsequent models.

[0073] In an embodiment of the present application, the MTS-Mixer spatiotemporal feature extraction module 120 is used to extract multi-level time series correlation features based on the MTS-Mixer feature mixing technology, and generate low-rank compressed prediction features in combination with the linear projection layer. It should be understood that industrial multivariate time series data often causes key features to be submerged due to noise, imbalance and complex interactions between variables, and traditional methods are difficult to effectively decouple time dependence and cross-channel associations. The MTS-Mixer module uses a hierarchical extraction mechanism of time filters (capturing local time series patterns, such as periodicity and trends) and channel filters (modeling global interactions between variables, such as sensor collaborative anomalies) to retain the time series dynamics of the original data and compress redundant information through low-rank linear projection, thereby reducing the computational complexity of subsequent models. This design solves the dual problems of feature redundancy and low computational efficiency in high-dimensional data, while enhancing the model's sensitivity to subtle abnormal patterns (such as sudden disturbances and implicit correlation failures), providing a high-quality feature base for accurate positioning of anomalies in subsequent stages. Therefore, in an embodiment of the present application, the MTS-Mixer framework extracts multi-level temporal correlation features of the input data and combines it with the linear projection layer to generate low-rank compressed prediction features.

[0074] Specifically, the MTS-Mixer spatiotemporal feature extraction module 120 includes the following implementation steps:

[0075] (1) Use the norm(·) function and the sample(·) function to perform normalization and downsampling on the input time series data respectively to obtain multiple subsequences χ h,1 ,…,χ h,s . The specific formula is expressed as:

[0076] χ h,1,…, χ h,s =sample(norm(χ h ))

[0077] where represents the input time series data, n represents the data length, and c represents the data dimension size.

[0078] (2) Extract the time dependencies of each subsequence χ h,i through the Temporal(·) method to obtain the time feature representation . The specific formula is expressed as:

[0079]

[0080] where Temporal(·) is a multi-layer perceptron (MLP) or an attention mechanism used to capture the dependency patterns at different time scales.

[0081] (3) Merge the time features extracted from all subsequences through the merge(·) method to obtain the final time feature representation . The specific formula is expressed as:

[0082]

[0083] (4) Perform channel redundancy processing on the input time series data through matrix factorization method. The specific formula is expressed as:

[0084]

[0085] where N∈R n×c represents noise; represents the channel correlation after denoising; U∈R n×m ,V∈R m×c (m < c) represents the channel interaction after decomposition, that is, the information of the channel dimension is simplified through low-rank decomposition to reduce the redundancy between channels.

[0086] (5) Calculate the channel dependencies after denoising through factorization method . The specific formula is expressed as:

[0087]

[0088] Where W1∈R m×c , W2∈R c×m Represents the weight matrix; σ represents the activation function; b1 and b2 represent bias terms, which are used to translate and adjust the features after linear transformation to enhance the expressiveness of the model. This process transforms the original data that combines time features and noise through matrix operations and activation functions, extracts the dependency relationship in the channel dimension, and uses factorization to reduce the amount of calculation and the number of parameters.

[0089] (6) The final prediction result is output through the linear projection layer Linear(·). The specific formula is expressed as:

[0090]

[0091] in, Indicates the final prediction result.

[0092] In an embodiment of the present application, the time series anomaly detection module 130 is used to dynamically model multi-level time series associations (local dependencies and global patterns) by alternately stacking anomaly-attention layers and feedforward networks, combining prior association constraints and adversarial learning to expand anomaly differences, and fusing reconstruction errors and association differences to generate point-by-point anomaly scores to achieve anomaly location and classification. It should be understood that abnormal patterns in industrial multivariate time series data often manifest as a subtle combination of multi-level features (such as local mutations and cross-variable collaborative failures). Traditional single attention mechanisms are difficult to simultaneously capture complex anomalies of time series dynamics and cross-channel interactions. Considering that by alternately stacking anomaly-attention layers and feedforward networks, it is possible to achieve a two-way enhancement of local time series sensitivity and global association modeling. The anomaly-attention layer focuses on abnormally sensitive time points and variable interactions through dynamic weight allocation, while the feedforward network performs nonlinear reconstruction of features to suppress noise interference; combined with association difference calculation (such as symmetric KL divergence) to quantify the multi-level feature distribution offset, the maximum and minimum strategies are used to enhance the model's ability to discriminate rare abnormal patterns. It not only solves the problem of low computational efficiency under high-dimensional data (reduces redundant computation by hierarchical attention), but also improves positioning accuracy and interpretability through difference-driven anomaly representation, and ultimately achieves highly robust anomaly detection and classification in complex industrial scenarios. Therefore, in an embodiment of the present application, by alternately stacking Anomaly-Attention layers and feedforward networks, combined with associated difference calculation and maximum and minimum strategies, abnormality-sensitive representations are mined from multi-level time series features to achieve highly robust time series anomaly positioning and classification. Among them, the Anomaly Transformer as a whole is an alternating stacking of Anomaly-Attention and feedforward layers, which is beneficial for the model to learn potential time series associations from multi-level features. It is mainly composed of Anomaly Transformer architecture, associated difference calculation, and maximum and minimum strategies.

[0093] Specifically, assume that the model contains l layers and the input is a time series χ∈R of length N N×d The timing anomaly detection module 130 is specifically implemented as follows:

[0094] (1) Constructing the prior correlation matrix P l :Through the learnable Gaussian kernel function and the dynamic scale parameter σ, the prior association weights between time points are calculated to generate the prior association matrix P in the form of discrete distribution l , the specific formula is expressed as:

[0095]

[0096] Among them, ‖ji‖ 2 Indicates time distance, represents the scaling factor, N represents the sequence length, represents the σ parameter matrix in the lth layer, d model Represents the model dimension.

[0097] In particular, the prior incidence matrix P l ∈R N×N It is generated based on the learned scaling factor σ. For the i-th time point, its associated weight with the j-th time point is obtained by the Gaussian kernel function Calculate the degree of association between two points. Then, through the Rescale(·) function, the association weight of each row is divided by the sum of the row, thus converting it into a priori association matrix P in the form of discrete distribution. l Through the above approach, an effective model is provided for time series anomaly detection.

[0098] (2) Constructing the sequence correlation matrix S l :Based on the self-attention mechanism, the sequence correlation matrix S is calculated by Softmax normalization of the correlation matrix (parameter matrix of Q, K, V) l , the specific formula is expressed as:

[0099] Initialization: Q, K, V,

[0100] Sequence correlation matrix:

[0101] in, Represent the query, key, and value in the self-attention mechanism respectively. are the parameter matrices of Q, K, and V in the lth layer respectively.

[0102] reconstruction:

[0103] in, It is the hidden representation after being processed by the abnormal attention mechanism of the lth layer.

[0104] That is, during the calculation of the self-attention mechanism, the Softmax(·) function is applied to the correlation matrix, and the attention map is normalized along the last dimension to obtain the sequence correlation matrix, thereby highlighting the important correlation degree between different time points.

[0105] (3) Calculate the correlation difference: the prior correlation matrix P l With the sequence correlation matrix S lPerform symmetrized KL divergence calculation and average the association differences of multiple layers to generate the association difference metric AssDis. The specific formula is:

[0106]

[0107] Where AssDis(P, S; χ) represents the point-wise association difference between the prior association P and the serial association S, Represents data χ at the i-th time point P l and S l This step calculates the KL divergence between the prior correlation and the sequence correlation by calculating the symmetric KL divergence. This can effectively capture the two-way difference in the distribution pattern of the two, identifying both global abnormal deviations (such as sudden disturbances) and locating local mutations (such as periodic failures). At the same time, averaging the multi-level differences can fuse low-level details with high-level semantic features, eliminating the limitations of a single perspective and forming a more discriminative abnormality-sensitive representation. Ultimately, through noise suppression and multi-scale complementarity, the model's robust recognition ability for complex time series anomalies is significantly improved.

[0108] (4) Calculate Minimax Association Learning: Use the Minimax Association Learning method to expand the difference between normal and abnormal time points. Specifically, it mainly uses additional loss to expand the association difference, so as to more effectively identify abnormal points. The specific loss calculation formula is:

[0109]

[0110] Benius norm and K norm; λ is used to weigh the two loss terms. When λ>0, the correlation difference can be increased.

[0111] (5) Anomaly determination: The weighted Softmax results of the fusion reconstruction error and the correlation difference are used to generate the point-by-point anomaly score AnomalyScore to locate and classify anomalies. The specific formula is expressed as follows:

[0112]

[0113]

[0114] Here, AnomalyScore(χ) represents the point-by-point anomaly determination criterion of χ, and ⊙ represents element-wise multiplication.

[0115] In the embodiment of the present application, the attention mechanism soft thresholding module 140 is used to suppress low-correlation noise, enhance key signals, and output attention representations with a high signal-to-noise ratio through dynamic soft thresholding operations. It should be understood that although the time series anomaly detection module 130 captures abnormality-sensitive features and generates point-by-point anomaly scores through the Anomaly-Attention layer, its output may still contain false positives caused by noise interference (such as fluctuations in irrelevant variables or transient environmental disturbances), and the granularity of anomaly positioning is relatively coarse (for example, only marking "anomaly exists in a certain time period" without being able to clearly define the contribution of specific variables). Based on this, in order to improve the signal-to-noise ratio and positioning accuracy of the results, in the embodiment of the present application, a dynamic soft thresholding operation is further used to adaptively suppress low-correlation noise (such as eliminating isolated sensor noise or redundant variables) and enhance high-correlation key anomaly features (such as the coordinated mutation of voltage surges and current dips). In addition, by quantizing feature weights, the abstract attention weights are converted into interpretable signals, which can solve the pain point of traditional methods of "detecting anomalies but not being able to explain the causes."

[0116] Specifically, the attention mechanism soft thresholding module 140 includes:

[0117] 1) Receive the attention representation (i.e., the weight distribution after feature extraction) from the time series anomaly detection module 130, recorded as input value x, where each element represents the attention weight on the time step or feature dimension.

[0118] 2) Apply soft thresholding to the input value x and remove low-correlation weights by setting the threshold β. The specific formula of the soft thresholding operation is expressed as:

[0119] y=sign(x)·max(0,|x|-β)

[0120] Where x is the input value, sign(x) is the sign function, β is the set threshold, y is the output value after soft thresholding, max(0,|x|-β) is the ReLU activation function, and the size of the difference determines whether the input value x will be retained or set to zero.

[0121] In particular, considering that the sign information of the input value is lost when calculating the absolute value, it is necessary to restore it through the sign function. Here, the sign of the input value x is restored through the sign(x) sign function. The specific operation formula is expressed as:

[0122]

[0123] That is, the attention mechanism soft thresholding module 140 is a post-processing of the output of the timing anomaly detection module 130, which not only eliminates uncertainty interference, but also provides a more reliable and traceable attention representation for the final decision, forming a complete closed loop of "detection → optimization → explanation".

[0124] In addition, in the embodiment of the present application, an experimental design module is also included. The specific design process is as follows:

[0125] (1) Experimental environment setup

[0126] The experimental code was designed using the PyTorch framework. The experimental environment is based on Python version 3.10.14, the PyTorch core library uses torch version 1.12.1, and the main dependent libraries include: numpy 1.23.2, pandas 2.0.3, Pillow 10.4.0, and scikit-learn 1.3.0. The experimental hardware environment uses a single Nvidia GeForce RTX 3090.

[0127] (2) Dataset

[0128] The proposed method is evaluated on three public datasets: SMD, MSL, and PSM. Detailed information about the data is shown in Table 1.

[0129] Table 1 Dataset information

[0130]

[0131] The SMD dataset, derived from server machine operation monitoring, collects data from 28 servers for five consecutive weeks, with a one-minute interval between each set of data. It contains multi-dimensional features related to server operation, such as CPU utilization, memory usage, and network traffic, presented in a time series format.

[0132] The MSL dataset originates from a spacecraft monitoring system. It collects data from multiple sensors to identify telemetry system anomalies. The dataset contains data for 55 entities, each consisting of 25 variables.

[0133] PSM collects internal data from multiple eBay application server nodes. The data collected in the dataset has 26 dimensions.

[0134] (3) Experimental parameter setting

[0135] During the experiment, we primarily used a non-overlapping sliding window method to extract a set of subseries. This method involves sliding windows of a certain size across the data sequence, without overlapping each time. This method extracts a set of subseries from the original data sequence, which can then be used for further analysis and processing. The specific parameter settings are shown in Table 2.

[0136] Table 2 Parameter Description

[0137]

[0138]

[0139] (4) Comparison of algorithms

[0140] The proposed method model is compared with 16 other baseline models, including InterFusion, Beat-GAN, OmniAnomaly, Long Short-Term Memory-Variational Autoencoder (LSTM-VAE), Deep Autoregressive Gaussian Mixture Model (DAGMM), Multiscale Principal Component Analysis and Clustering Anomaly Detection (MPPCACD), Local Outlier Factor (LOF), Integrated Transformation-Based Anomaly Detection (ITAD), Temporal Hierarchical Optimized Clustering-Based Anomaly Detection (THOC), Deep Support Vector Data Description (Deep-SVDD), Multiscale Principal Component Analysis with Contrastive Learning (CL-MPPCCA), Long Short-Term Memory Network (LSTM), Vector Autoregressive Model (VAR), One-Class Support Vector Machine (OC-SVM), Isolation Forest (IsolationForest), and Anomaly Transformer. The algorithm details are shown in Table 3.

[0141] Table 3 Algorithm details

[0142]

[0143]

[0144] (5) Evaluation indicators

[0145] We select precision, recall, and F1 score as evaluation indicators. The higher the value, the better the detection performance of the model. The specific calculation formula is as follows:

[0146]

[0147] Then you can obtain the experimental results and analyze them, including the following:

[0148] (1) Experimental results

[0149] The experiments primarily target different time series datasets for testing and evaluation. Six mainstream machine learning algorithms and seven deep learning models for time series anomaly detection are evaluated. The evaluation metrics primarily use precision (P), recall (R), and F1 score. The experimental results are shown in Table 4.

[0150] Table 4 Performance comparison of state-of-the-art detection methods

[0151]

[0152]

[0153] (2) Analysis of overall experimental results

[0154] Analysis of the machine learning model testing results shows that on the SMD dataset, the VAR model has a high accuracy of 78.35%. The CL-MPPCA model has an accuracy of 99.93% on the PSM dataset, but its performance on the SMD and MSL datasets varies greatly. Overall, the performance of machine learning models on different datasets is unstable. From the perspective of deep learning, Deep-SVDD performs well on the MSL dataset, with an F1 value of 90.73%; InterFusion has an accuracy and F1 value of 83.61% and 83.52% on the PSM dataset, respectively. Deep learning models also have certain fluctuations on different datasets, but some models have obvious advantages on individual datasets. From the overall results, the method proposed in this patent is significantly higher than other models in all indicators on the three datasets, and has excellent performance in precision, recall, and F1 value, indicating that its overall performance is better.

[0155] For the SMD dataset, the method proposed in this patent ranked first with a precision of 90.48%, nearly 11 percentage points higher than the second-place THOC with 79.76%. Other models such as Deep SVDD (78.54%) and LSTM (78.55%) also performed well, but there is still a gap with the model proposed in this patent. The recall rate of the model proposed in this patent is as high as 97.55%, far exceeding other models. For example, the recall rate of THOC, which ranks second, is 90.95%, while the recall rates of models such as LOF (39.86%) and IsolationForest (73.29%) are relatively low. The F1 value of the method proposed in this patent is 93.6%, which is the best performance among all models. This shows that the method proposed in this patent can strike a good balance between precision and recall in the SMD data scenario, accurately identifying abnormal data while avoiding excessive false positives.

[0156] For the MSL dataset, the method proposed in this patent has an accuracy of 93.3%, ranking first, and has an advantage of nearly 5 percentage points compared to the second-ranked THOC (88.45%). Although other models such as DAGMM (89.60%) and BeatGAN (89.75%) also perform well, there is still a significant gap with the method model proposed in this patent. The recall rate of the method proposed in this patent is as high as 97.95%, which is much higher than other models. For example, the recall rate of THOC, which ranks second, is 90.97%, while the recall rates of models such as OCSVM (86.87%) and IsolationForest (86.54%) are relatively low. The F1 value of the method proposed in this patent is 95.57%, which performs best among all models. This shows that the method proposed in this patent can accurately identify abnormal data in the MSL data scenario while maintaining a low false alarm rate, and its performance advantage is obvious.

[0157] On the PSM dataset, the proposed method leads the pack with a precision of 98.09%, surpassing the second-placed BeatGAN (90.30%) by over 8 percentage points. Other models, such as Deep SVDD (95.41%) and LSTM VAE (73.62%), lag significantly behind the proposed method in terms of precision. The proposed method achieves a near-perfect recall of 98.75%. Other models, such as ITAD (64.02%) and CLMPPCA (99.93%), exhibit significantly different recall rates. While CLMPPCA also achieves a high recall, its precision (56.02%) is lower, resulting in inferior overall performance compared to the proposed method. The proposed method achieves an F1 score of 98.42%. This demonstrates that the proposed model can extremely accurately identify anomalies in PSM data scenarios, with virtually no false positives or negatives, resulting in exceptional performance.

[0158] Through comparative analysis, it can be seen that the method proposed in this patent has the best performance in precision, recall and F1 value in the three data scenarios of SMD, MSL and PSM. Whether in terms of the accuracy of identifying abnormal data (precision), or in the ability to avoid missing abnormal data (recall), and in terms of comprehensively considering the balance between precision and recall (F1 value), the model of this patent is significantly better than other models. This fully proves that the method proposed in this patent has a strong performance advantage in processing unsupervised time series data, especially for the rarity of anomalies. Anomaly Transformer soft thresholding can effectively identify abnormal patterns in data. Through the soft threshold processing mechanism, it can more accurately capture points that deviate from the normal data distribution and improve the accuracy of anomaly detection.

[0159] (3) Analysis of ablation experiment results

[0160] This ablation experiment tested the performance of "removing the Mixer module", "removing the soft thresholding module of the attention mechanism", and "removing both" in three data scenarios: SMD, MSL, and PSM. The evaluation indicators included precision (P), recall (R), and F1 value.

[0161] Table 5 shows the results of ablation experiments on the SMD dataset. After removing both the MTS-Mixer module and the attention mechanism soft thresholding module, the model achieved an accuracy of 89.01%. Removing only the MTS-Mixer module achieved an accuracy of 90.97%, and removing only the attention mechanism soft thresholding module achieved an accuracy of 91.63%. The proposed method achieved an accuracy of 90.48%. This suggests that the MTS-Mixer module and the attention mechanism soft thresholding module may have introduced some interference. However, removing both modules resulted in a lower accuracy than removing either module individually or the complete model, indicating that removing both modules simultaneously had a negative impact on the model. The proposed method achieved the highest recall. Removing either the MTS-Mixer module or the attention mechanism soft thresholding module decreased the recall, with the decrease being more pronounced after removing both modules simultaneously. This suggests that these two modules have a positive effect on capturing anomalous samples and help improve recall. When a single module is removed, the F1 value is close to that of the method proposed in this patent, indicating that the overall performance of the model is not greatly affected; but after removing two modules, the F1 value is lower than that of the method proposed in this patent, further indicating that these two modules play a certain role in maintaining the overall performance of the model.

[0162] Table 5 Ablation experiment results of SMD dataset

[0163]

[0164] The ablation results for the MSL dataset are shown in Table 6. In terms of precision, the model with the attention mechanism soft thresholding module removed achieved the highest precision (93.84%), surpassing the model with the MTS-Mixer module removed (92.10%), the model with both modules removed (92.06%), and the model proposed in this patent (93.30%). This suggests that the attention mechanism soft thresholding module may, in some cases, reduce the model's accuracy in distinguishing positive samples, and removing it can help reduce false positives. In terms of recall, the proposed model performed best (97.95%), significantly higher than other ablation configurations, followed by the model with both the MTS-Mixer module removed (96.56%) and the attention mechanism soft thresholding module removed (96.45%). The model with both modules removed simultaneously achieved the lowest recall (95.20%). This clearly demonstrates that the Mixer module and the attention mechanism soft thresholding module play a key role in capturing true anomalies, and their collaborative work maximizes the model's ability to identify anomalies. In terms of F1 value, the model proposed in this patent leads with 0.9557%, followed by removing the soft thresholding module of the attention mechanism (95.12%), removing the MTS-Mixer module (94.28%), and removing both modules at the same time (93.60%). This shows that the existence of the two modules is crucial to maintaining the balance of the overall performance of the model. Although the removal of a single module will affect the performance of the model, it can still maintain a relatively good overall performance, while removing both modules will lead to a significant decline in the overall performance. In summary, both the MTS-Mixer module and the attention mechanism soft thresholding module have a positive effect on performance improvement in this model. Although there are differences in their contributions to precision, their role in recall and overall performance is indispensable, providing strong support for the model's anomaly detection capabilities.

[0165] Table 6 Ablation experiment results on MSL dataset

[0166]

[0167] Table 7 shows the results of ablation experiments on the PSM dataset. In terms of precision, our proposed model (Ours) achieved 98.09%, exceeding both the cases of removing a single module and removing both modules simultaneously. Specifically, removing the Mixer module achieved 97.80%, removing the Attention Mechanism Soft Thresholding Module achieved 97.78%, and removing both modules resulted in a precision of 97.00%. This demonstrates that the presence of these two modules significantly improves the model's ability to accurately identify positive samples. The superior precision of our proposed model reflects the value of module synergy. In terms of recall, our proposed model achieved 98.75%, slightly higher than other configurations. Removing the Mixer module achieved 98.64%, removing the Attention Mechanism Soft Thresholding Module achieved 98.49%, and removing both modules achieved 98.46%. While the differences in recall between configurations were small, the complete model still achieved a slight advantage, indicating that both modules contribute to the model's ability to capture true anomalies to some extent, and their combined effectiveness is even greater. In terms of F1 score, our proposed model leads with a score of 98.42%. Removing the MTS-Mixer module brings the score to 98.22%, removing the attention mechanism soft thresholding module brings the score to 98.13%, and removing both modules brings the score to 97.73%. This demonstrates that the MTS-Mixer module and the attention mechanism soft thresholding module play a key role in maintaining the overall performance of the model. The combination of these modules in the complete model achieves a better balance between accurate recognition and anomaly detection. Overall, on the PSM dataset, both the MTS-Mixer module and the attention mechanism soft thresholding module contribute positively to model performance. Working together, they help improve the model's precision, recall, and overall performance.

[0168] Table 7 Ablation experiment results of PSM dataset

[0169]

[0170] Through the analysis of three ablation experiments, it can be seen that the mixer module has a certain positive contribution to the precision rate in the SMD and MSL datasets, and also has a certain help to the precision rate and comprehensive performance (F1 value) in the PSM dataset, but has little effect on the recall rate in the MSL dataset. The attention soft thresholding module plays a key role in the recall rate in the SMD and MSL datasets, and has a great impact on the comprehensive performance (F1 value) in the MSL dataset, while it has little impact on the precision rate in the SMD dataset, and may even cause certain interference. It has little impact on the recall rate and comprehensive performance (F1 value) in the PSM dataset. The model proposed in this patent integrates modules such as the mixer and attention soft thresholding, and has achieved the best or near-optimal performance in the three data scenarios of SMD, MSL, and PSM, indicating that the combination of these modules can effectively improve the overall performance of the model, verifying the rationality and superiority of the model design.

[0171] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit of the technical solutions of the present invention.

Claims

1. A multi-stage anomaly detection system based on multivariate time series data, characterized in that: include: An input data acquisition module is used to acquire multivariate data based on time series as input data; The MTS-Mixer spatiotemporal feature extraction module is used to extract multi-level temporal correlation features based on the MTS-Mixer feature mixing technology, and generate low-rank compressed prediction features in combination with the linear projection layer; The time series anomaly detection module dynamically models multi-level time series associations by alternating between stacked anomaly-attention layers and feedforward networks. It combines prior association constraints with minimax association learning to amplify anomaly differences and fuses reconstruction errors with association differences to generate point-by-point anomaly scores for anomaly location and classification. The attention mechanism soft thresholding module is used to suppress low-correlation noise through dynamic soft thresholding operations, enhance key signals, and output attention representations with high signal-to-noise ratio.

2. The multi-stage anomaly detection system based on multivariate time series data according to claim 1, characterized in that: The MTS-Mixer spatiotemporal feature extraction module includes: Use time filters to capture dependencies in the time dimension; Use channel filters to capture channel dependencies; The final prediction result is output through the linear projection layer Linear(·). The specific formula is expressed as: in, is the final prediction result; It is the time feature representation; is the channel dependency after denoising.

3. The multi-stage anomaly detection system based on multivariate time series data according to claim 2, characterized in that: Use time filters to capture dependencies in the time dimension, including: The input time series data is normalized and downsampled using the norm(·) function and sample(·) function respectively to obtain multiple subsequences χ h,1 ,…,χ h,s , the specific formula is expressed as: x h,1 ,…,x h,s =sample(norm(χ h )) in, Represents the input time series data, n represents the data length, and c represents the data dimension size; Extract each subsequence χ by Temporal(·) method h,i The temporal dependency of The specific formula is: Temporal(·) is a multi-layer perceptron (MLP) or attention mechanism, which is used to capture dependency patterns at different time scales. The time features extracted from all subsequences are merged using the merge(·) method Merge to get the final time feature representation The specific formula is:

4. The multi-stage anomaly detection system based on multivariate time series data according to claim 3, characterized in that: Leverage channel filters to capture channel dependencies, including: The input time series data is processed for channel redundancy through matrix decomposition method. The specific formula is expressed as: R n×m , V ∈ R m×c (m < c) represents the decomposed channel interaction; Calculate the channel dependencies after denoising by factorization method The specific formula is: Where W1∈R m×c , W2∈R c×m Represents the weight matrix; σ represents the activation function; b1 and b2 represent bias terms (BiasTerms).

5. The multi-stage anomaly detection system based on multivariate time series data according to claim 4, characterized in that: In the time series anomaly detection module, by alternately stacking anomaly-attention layers and feedforward networks, multi-level time series associations are dynamically modeled, including: Through the learnable Gaussian kernel function, combined with the dynamic scale parameter σ, the prior association weights between time points are calculated to generate the prior association matrix P in the form of discrete distribution. l , the specific formula is expressed as: Among them, ‖ji‖ 2 represents the time distance, σ=χ l-1 W l σ (σ∈R N×1 ) represents the scaling factor, N represents the sequence length, represents the σ parameter matrix in the lth layer, d model Represents the model dimension; Calculate the sequence correlation matrix S based on the self-attention mechanism l , dynamic time dependency is captured by Softmax normalization, and the specific formula is expressed as: Initialization: Q, K, V, χ l-1 W l σ Sequence correlation matrix: in, Represent the query, key, and value in the self-attention mechanism respectively. are the parameter matrices of Q, K, and V in the lth layer respectively. reconstruction: in, It is the hidden representation after being processed by the abnormal attention mechanism of the lth layer.

6. The multi-stage anomaly detection system based on multivariate time series data according to claim 5, characterized in that: In the time series anomaly detection module, prior association constraints are combined with mini-max association learning to expand anomaly differences, including: For the prior correlation matrix P l With the sequence correlation matrix S l Perform symmetrized KL divergence calculation and average the association differences of multiple layers to generate the association difference metric AssDis. The specific formula is: Where AssDis(P, S; χ) represents the point-wise association difference between the prior association P and the serial association S, Represents data χ at the i-th time point P l and S l The KL divergence calculated between; Through the mini-maximum association learning method, additional loss is used to expand the difference between normal and abnormal time points. The specific loss calculation formula is: in, represents the reconstruction result of data χ, and ‖·‖ F and‖·‖ k represents the Min-Frobenius norm and K-norm; λ is used to weigh the two loss terms.

7. The multi-stage anomaly detection system based on multivariate time series data according to claim 6, characterized in that: In the time series anomaly detection module, the reconstruction error and the associated difference are fused to generate a point-by-point anomaly score to achieve anomaly localization and classification, including: The weighted Softmax results of the fusion reconstruction error and the associated difference are used to generate the point-by-point anomaly score AnomalyScore to locate and classify anomalies. The specific formula is expressed as follows: Here, AnomalyScore(χ) represents the point-by-point anomaly determination criterion of χ, and ⊙ represents element-wise multiplication.

8. The multi-stage anomaly detection system based on multivariate time series data according to claim 7, characterized in that: The attention mechanism soft thresholding module includes: Receive the attention representation from the time series anomaly detection module 130, that is, the weight distribution after feature extraction, recorded as input value x; Apply a soft thresholding operation to the input value x, and remove low-correlation weights by setting the threshold β. The specific formula of the soft thresholding operation is expressed as: y=sign(x)·max(0,|x|-β) Among them, x is the input value, sign(x) is the sign function, β is the set threshold, y is the output value after soft thresholding, and max(0,|x|-β) is the ReLU activation function.

9. The multi-stage anomaly detection system based on multivariate time series data according to claim 8, characterized in that: In the attention mechanism soft thresholding module, the sign(x) function is used to restore the sign of the input value x. Its specific operation formula is expressed as:

Images