Multivariate time series anomaly detection method

Abstract

The invention relates to a multivariate time sequence anomaly detection method. An anomaly detection model used in the method comprises a U-Net encoder, a U-Net decoder, a time sequence characteristic memory module and a space interaction perception module. Performing multi-scale time sequence feature extraction on the weight-standardized one-dimensional convolutional layer in the U-Net encoder to obtain multi-level encoding features; in a time sequence feature memory module, enhancing long-term dependence features for the multi-level coding features; inputting the memory features into a space interaction perception module, constructing a graph structure between variables through a graph attention network, and outputting space enhancement features; in a U-Net decoder, splicing the corresponding memory features with the up-sampling features of the upper layer, and obtaining reconstruction features through one-dimensional convolution fusion; and calculating a dimension-by-dimension square error sum between the original time sequence fragment and the reconstructed data to obtain an abnormal score. Compared with the prior art, the method has the advantages of high precision, strong robustness, high efficiency and the like.

Description

Technical Field

[0001] This invention relates to the field of anomaly detection, and in particular to a multivariate time series anomaly detection method. Background Technology

[0002] With the deepening of Industry 5.0, industrial automation and intelligent manufacturing systems increasingly rely on sensor networks for real-time monitoring of equipment operating status. Multivariate time-series data, as a core data format in industrial environments, directly impacts production safety, equipment reliability, and operational efficiency through its anomaly detection capabilities. Currently, time-series-based anomaly detection has become a key technology in industrial data analysis, playing a crucial role, especially in fault early warning and predictive maintenance.

[0003] However, multivariate time series data in industrial environments are typically characterized by high dimensionality, high noise, non-stationarity, and complex spatiotemporal dependencies between variables, posing significant challenges to anomaly detection. Existing anomaly detection methods mainly fall into two categories: predictive models and reconstruction models. Predictive models identify anomalies that deviate from expected patterns by establishing time series prediction models, but they are sensitive to sudden anomalies and extreme noise, and struggle to effectively capture nonlinear relationships between multiple variables. Reconstruction models attempt to identify anomalies by learning and reconstructing normal data distributions. While possessing some robustness, they are still susceptible to noise interference and long-term dependencies, leading to ambiguous or unstable reconstruction results, especially when dealing with high-dimensional multivariate data. Furthermore, existing methods often fail to fully exploit long-term evolutionary patterns and spatial correlations between variables in time series, limiting their applicability and accuracy in complex industrial scenarios. Therefore, how to enable anomaly detection to handle the complex spatiotemporal characteristics of multivariate time series and possess stronger anomaly identification capabilities, thereby improving the accuracy and reliability of monitoring the operating status of industrial equipment, is a technical problem that needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a multivariate time series anomaly detection method. This method processes multivariate time series data obtained from industrial equipment sensor networks and obtains accurate detection results through a U-Net encoder, a U-Net decoder, a time series feature memory module, and a spatial interactive sensing module.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] According to one aspect of the present invention, a multivariate time series anomaly detection method is provided, the method using a pre-trained anomaly detection model, the anomaly detection model being constructed based on a denoising diffusion probability model, including a U-Net encoder, a U-Net decoder, a temporal feature memory module, and a spatial interaction perception module;

[0007] The method includes the following steps:

[0008] S1. Collect multivariate time series data generated by the sensor network of the industrial equipment to be tested; perform data preprocessing on the multivariate time series data to obtain standardized time series segments;

[0009] S2. Input the time series segment into the anomaly detection model; perform multi-scale temporal feature extraction in the weight-normalized one-dimensional convolutional layer of the U-Net encoder to obtain multi-level encoded features; in the temporal feature memory module, enhance the long-term dependency features of the multi-level encoded features through LSTM memory units to obtain memory features;

[0010] S3. Input the memory features into the spatial interaction perception module, construct the graph structure between variables through the graph attention network, calculate the attention weights, weighted aggregate the node features, and output the spatial enhancement features.

[0011] S4. In the U-Net decoder, the corresponding memory features are concatenated with the upsampled features of the previous layer and fused through one-dimensional convolution to obtain the reconstructed features. The reconstructed features are then progressively denoised and reconstructed through a reverse denoising process to output the reconstructed data. The sum of squared errors between the original time series segments and the reconstructed data is calculated to obtain the anomaly score. Based on the anomaly score, it is determined whether an anomaly exists to obtain the anomaly detection result.

[0012] Furthermore, the data preprocessing in S1 specifically includes calculating the normalized data for each sensor channel data set and the data points of that channel, expressed as:

[0013]

[0014] Where x is the standardized data, M is the data set of the sensor channel, m is the data point, and ColMin and ColMax are the minimum and maximum values ​​of the channel, respectively;

[0015] The standardized data is segmented using a sliding window strategy to obtain time series segments.

[0016] Furthermore, the U-Net encoder in S2 performs multi-layer convolution and multiple downsampling operations on the time series segments to extract temporal features at different time scales, obtain multi-level encoded features, and performs linear attention calculation after each convolutional layer to obtain attention-enhanced feature representations.

[0017] Further, the specific steps in each convolutional layer include: extracting multi-scale temporal features from the output data of layer l-1 through a weight-normalized one-dimensional convolutional layer to obtain the primary feature representation of layer l; performing layer normalization on the primary feature representation of layer l to obtain the normalized features of layer l; concatenating the normalized features of layer l with the time-step embedding vector to form the fused features of layer l; sequentially performing weight-normalized one-dimensional convolution, group normalization, and SiLU activation function processing on the fused features to obtain the encoding features of layer l; and performing residual concatenation between the encoding features of layer l and the encoding features processed by one-dimensional convolution to obtain the temporal feature tensor of layer l.

[0018] Furthermore, the specific steps of the linear attention calculation include: performing a linear transformation mapping on the temporal feature tensor to obtain the query matrix, key matrix, and value matrix respectively; and transforming the standard dot product attention into a low-rank approximation form through kernel function mapping, reducing the computational complexity from O(n^2) to O(n^2). 2 The linear attention calculation result is reduced to O(n); the linear attention calculation result is transformed in dimension through the projection layer, and the transformed result is residually connected with the temporal feature tensor. After layer normalization, the attention-enhanced multi-level encoded feature representation is finally output.

[0019] Furthermore, the specific steps for the LSTM memory unit in S2 to process the multi-level encoded features include:

[0020] The multi-level encoded features of the current layer are concatenated with the hidden state of the previous layer. The forget gate controls the degree of memory of the cell state of the previous layer. The input gate controls the update degree of the current input multi-level encoded feature information. Candidate cell states are generated and the current cell state is updated. The output gate controls the output of the current hidden state. The current hidden state is calculated. The memory features output by the LSTM memory unit are passed to the U-Net decoder. In the U-Net decoder, the memory features are fused with the upsampled features of the corresponding layer decoder.

[0021] Furthermore, the input to the spatial interaction perception module in S3 is the attention-enhanced multi-level coding feature F = X ∈ R^(B × D), where B is the batch size and D is the feature dimension. The attention-enhanced multi-level coding feature is mapped to a graph structure G = (V, E), where node V represents sensor variables and edge E represents the dynamic relationship between variables. The interaction weights between variables in different feature subspaces are learned in parallel through a multi-head attention mechanism. Residual connections are used to combine the original features with the learned attention-enhanced feature representations.

[0022] Furthermore, the specific steps performed by the multi-head attention mechanism for each attention head include:

[0023] Calculate the attention coefficient between any two nodes i and j. The expression is:

[0024]

[0025] Among them, a k Let w be the learnable weight vector of the k-th attention head, used to map the concatenated features to a scalar attention coefficient. k Let v be the learnable weight matrix for the k-th attention head, used to perform linear transformations on the node features. i and v j These represent the original feature vectors of node i and node j, respectively.

[0026] The attention coefficients are normalized to obtain the attention weights. The expression is:

[0027]

[0028] in, The set of neighboring nodes of node i;

[0029] Based on attention weights, the features of neighboring nodes are weighted and aggregated to generate the output feature of node i at the k-th attention head. The expression is:

[0030]

[0031] The output features of the K attention heads are concatenated to obtain the final feature representation h′ of each node. i The expression is:

[0032]

[0033] Furthermore, the features processed by graph attention are residually concatenated with the attention-enhanced multi-level encoded features F to obtain intermediate features M:

[0034] M = F + GAT(F),

[0035] Where GAT(F) represents the output of the graph attention network;

[0036] After layer normalization, the intermediate feature M is input into a feedforward neural network for nonlinear transformation and dimensionality adjustment. The output of the feedforward neural network is then residually connected to the intermediate feature M again to output the final spatially enhanced feature representation F. att =M+FFN(LayerNorm(M)).

[0037] Furthermore, the U-Net decoder in S4 receives spatial enhancement features and uses a neighbor interpolation upsampling method to upsample the input features to obtain upsampled features; the upsampled features are concatenated with the memory features of the corresponding level in the time dimension to form fused features; a one-dimensional convolution operation is performed on the fused features to realize feature fusion and dimension adjustment, resulting in fused multi-scale temporal features, i.e., reconstructed features.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] (1) Improved accuracy and robustness of anomaly detection in complex industrial environments: This invention constructs an anomaly detection framework that integrates spatiotemporal representation and diffusion generation, and jointly learns the dynamic evolution characteristics of time series data and the spatial correlation between variables. This effectively overcomes the shortcomings of existing methods in terms of long-term dependence and insufficient modeling of spatial structure. Compared with traditional prediction or reconstruction models, this invention can more accurately identify small anomalies and nonlinear anomalies caused by complex spatiotemporal coupling mechanisms, and significantly reduce false alarm and false negative rates.

[0040] (2) Enhanced the model’s ability to express long-term dependencies and spatial correlations in multivariate time series: The time series feature memory module selectively memorizes the long-term evolution pattern of time series through the gating mechanism of LSTM memory units, effectively capturing the time series dependencies of periodic fluctuations and sudden events. At the same time, the spatial interaction perception module uses graph attention network to dynamically learn the correlation weights between variables and explicitly model the spatial topology of sensor network, thereby discovering potential correlation anomalies between variables and improving the feature representation ability of high-dimensional multivariate data.

[0041] (3) An efficient and unified end-to-end anomaly detection process is achieved: The anomaly detection model has a compact overall structure, integrating data preprocessing, feature extraction, noise injection, reverse denoising, and anomaly determination into an end-to-end training and inference framework. The U-Net encoder-decoder combines weighted normalized one-dimensional convolution with a linear attention mechanism, which significantly reduces the computational complexity of long sequence processing while ensuring multi-scale feature extraction capabilities. This method has a unified structure, is easy to extend, and can be adapted to the dual requirements of real-time performance and accuracy in different industrial application scenarios. Attached Figure Description

[0042] Figure 1 The flowchart is for a multivariate time series anomaly detection method;

[0043] Figure 2 This is a diagram showing the training data flow for a multivariate time series anomaly detection method.

[0044] Figure 3 This is a partial abnormal segment of the SWAT dataset used in this embodiment;

[0045] Figure 4 This is the result of anomaly detection for some abnormal segments of the SWAT dataset in this embodiment. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] Existing traditional industrial multivariate time-series anomaly detection methods are insufficient in fully exploring the spatiotemporal correlations between sensor variables and often struggle to distinguish false anomalies, leading to misjudgments or missed detections. This embodiment proposes a multivariate time-series anomaly detection method that integrates spatiotemporal representation and diffusion generation to address the shortcomings of existing methods in learning long-term temporal dependencies and spatial correlations. Based on a denoised diffusion probability model, Gaussian noise is added to simulate data perturbation, learning the changing patterns of data distribution during the training phase. Then, during the testing phase, the original data distribution is gradually reconstructed through reverse denoising, achieving accurate detection of potential anomalies. Simultaneously, a one-dimensional convolution and linear attention mechanism are combined in the U-Net encoder-decoder network to effectively extract multi-scale temporal features and improve the learning efficiency for long sequences. To address the long-term dependencies in industrial equipment operation, the temporal feature memory module incorporates LSTM memory units in multi-scale skip connections to further enhance the learning ability of temporal evolution patterns. To improve the representation of relationships between multi-source heterogeneous variables, a spatial interaction perception module is constructed, using a graph attention network to learn the dependencies between variables and enhancing the spatial information transmission effect through residual connections. Finally, by analyzing the residual distribution between the original sequence and the denoised reconstructed sequence, outliers are adaptively identified, achieving high-precision unsupervised anomaly detection.

[0048] This embodiment provides a multivariate time series anomaly detection method using a pre-trained anomaly detection model. The anomaly detection model is built upon a denoising diffusion probability model and includes a U-Net encoder, a U-Net decoder, a temporal feature memory module, and a spatial interaction perception module. The U-Net encoder / decoder extracts multi-scale temporal features through one-dimensional convolution and linear attention mechanisms; the temporal feature memory module utilizes LSTM units to enhance the learning ability of long-term dependent features; and the spatial interaction perception module learns the correlations between variables through a graph attention network.

[0049] like Figure 1 As shown, the multivariate time series anomaly detection method in this embodiment includes the following steps:

[0050] S1. Collect multivariate time series data generated by the sensor network of the industrial equipment to be tested; perform data preprocessing on the multivariate time series data to obtain standardized time series segments;

[0051] S2. Input the time series segments into the anomaly detection model; extract multi-scale temporal features in the weight-normalized one-dimensional convolutional layer of the U-Net encoder to obtain multi-level encoded features; in the temporal feature memory module, enhance the long-term dependency features of the multi-level encoded features through LSTM memory units to obtain memory features;

[0052] S3. Input the memory features into the spatial interaction perception module, construct the graph structure between variables through the graph attention network, calculate the attention weights, weighted aggregate the node features, and output the spatial enhancement features.

[0053] S4. In the U-Net decoder, the corresponding memory features are concatenated with the upsampled features of the previous layer and fused through one-dimensional convolution to obtain the reconstructed data; the sum of squared errors in each dimension between the original time series segment and the reconstructed data is calculated to obtain the anomaly score; based on the anomaly score, it is determined whether an anomaly exists to obtain the anomaly detection result.

[0054] This embodiment uses the SWAT dataset, a standard dataset from the real industrial field, to validate the effectiveness of the model. The SWAT dataset was collected by the iTrust Center at Singapore Technological University, originating from a water treatment system. It contains 51-dimensional sensor readings, with 495,000 normal data points in the training set and 449,919 attack data points in the test set, resulting in an anomaly rate of 12.14%. This dataset uses a 7-day normal data set followed by a 4-day attack data set, specifically designed for evaluating anomaly detection in industrial control systems. The hardware environment consisted of an Ubuntu 20.04 server equipped with an AMD Ryzen 95900X CPU and an NVIDIA RTX 3090 GPU. The software environment included CUDA version 11.3 and cuDNN version 8.2.1. The model code was trained and inferred using Python version 3.8 and PyTorch version 1.11.0.

[0055] Raw multivariate time-series data generated by industrial sensor networks were acquired, and anomalous segments were labeled. The SWAT dataset was divided into a training set of 495,000 normal data points and a test set of 449,919 attack data points. The original anomaly labeling information was retained for all datasets for model training and evaluation.

[0056] The data collected in S1 is a dataset of a modern industrial water treatment system. This system includes actual water pumps, pipes, valves, water tanks, chemical dosing devices, sensors, and actuators. The dataset was compiled by installing various sensors and monitoring equipment on a test platform, collecting real-time operational data for up to 11 days. This data includes various physical parameters such as flow rate, liquid level, water quality parameters, pressure, temperature, and data points from actuators, as well as system status and operating commands.

[0057] Data preprocessing specifically includes calculating the normalized data for each sensor channel data set and its corresponding data points, expressed as:

[0058]

[0059] Where x is the standardized data, M is the data set of the sensor channel, m is the data point, and ColMin and ColMax are the minimum and maximum values ​​of the channel, respectively;

[0060] The normalized time-series data is segmented using a sliding window strategy, with a window length of L = 256 and a step size of S = 16. This applies to D-dimensional channel data.

[0061] The U-Net encoder in S2 performs multi-layer convolution and multiple downsampling operations on time series segments to extract temporal features at different time scales, resulting in multi-level encoded features. Linear attention is then calculated after each convolutional layer to obtain attention-enhanced feature representations.

[0062] The specific steps in each convolutional layer include: extracting multi-scale temporal features from the output data of layer l-1 through a weight-normalized one-dimensional convolutional layer to obtain the primary feature representation of layer l; performing layer normalization on the primary feature representation of layer l to obtain the normalized features of layer l; concatenating the normalized features of layer l with the time-step embedding vector to form the fused features of layer l; sequentially performing weight-normalized one-dimensional convolution, group normalization, and SiLU activation function processing on the fused features to obtain the encoding features of layer l; and performing residual concatenation between the encoding features of layer l and the encoding features processed by one-dimensional convolution to obtain the temporal feature tensor of layer l.

[0063] The specific steps of linear attention computation include: performing a linear transformation mapping on the temporal feature tensor to obtain the query matrix, key matrix, and value matrix respectively; and transforming the standard dot product attention into a low-rank approximation form through kernel function mapping, reducing the computational complexity from O(n^2) to O(n^2). 2 The linear attention calculation result is reduced to O(n); the linear attention calculation result is transformed in dimension through the projection layer, and the transformed result is residually connected with the temporal feature tensor. After layer normalization, the attention-enhanced multi-level encoded feature representation is finally output.

[0064] Specifically, the U-Net encoder and decoder use weight-normalized one-dimensional convolutions to extract local contextual features from temporal data. The convolutional kernels employ weight normalization to process input features, suppressing gradient explosion and vanishing gradient problems during training. In both the encoder and decoder, the temporal dimension is progressively compressed through multiple convolutional layers and downsampling operations to form a multi-scale temporal context representation. Feature representations are learned using weight-normalized one-dimensional convolutional layers and linear attention layers, and multi-scale temporal features are extracted layer by layer through downsampling and normalization operations. To fully utilize the temporal information during the diffusion process, each convolutional layer is connected to the time-step embedding vector t. emd This time vector is injected into the convolution calculation, forming a temporal modulation convolution mechanism to enhance the prediction ability between different time steps. Multi-level downsampling operations are performed on the features encoded by the U-Net network to obtain the layer-l encoded features X. l X l With time embedding vector t emd The input is connected to the next encoder network, and temporal features at different time scales are extracted through multiple downsampling operations and feature extraction networks.

[0065]

[0066] The output features are obtained after using residual connection.

[0067]

[0068] The U-Net encoder and decoder employs a linear attention mechanism to enhance the model's ability to represent long-term temporal dependencies. This mechanism replaces the traditional self-attention mechanism, significantly reducing computational complexity while maintaining feature extraction capabilities. The standard dot-product attention formula is transformed into a low-rank approximation, reducing the computational complexity of attention from O(n^2) to O(n^2). 2 The computational complexity is reduced to O(n) by using kernel function mapping to minimize the linear impact of sequence length on computational complexity. This transformation is achieved through kernel function mapping operations, mapping each input feature vector separately and then performing a linear combination, avoiding the large-scale matrix multiplication problem inherent in traditional attention methods. This applies to the temporal feature tensor output by the one-dimensional convolutional layer. A linear transformation is performed to obtain the projection matrix of query Q, key K, and value V, and linear attention is calculated using the following formula:

[0069]

[0070] During the U-Net network encoding and decoding process, for the time dimension, the encoder uses one-dimensional convolution to extract local temporal features and then uses a linear attention mechanism to focus on feature changes. After residual connections and normalization, the output feature vector is completed.

[0071]

[0072] The specific steps of the LSTM memory unit in S2 for processing multi-level encoded features include: concatenating the multi-level encoded features of the current layer with the hidden state of the previous layer, controlling the degree of memory of the cell state of the previous layer through the forget gate; controlling the update degree of the current input multi-level encoded feature information through the input gate; generating candidate cell states and updating the current cell state; controlling the output of the current hidden state through the output gate; calculating the current hidden state; passing the memory features output by the LSTM memory unit to the U-Net decoder; and in the U-Net decoder, fusing the memory features with the upsampled features of the corresponding layer decoder.

[0073] The temporal feature memory module enhances the model's ability to learn and represent long-term dependent features in multivariate time series through LSTM memory units. It continuously updates the memory state and hidden state through the gating mechanism of a Long Short-Term Memory (LSTM) network, thereby learning to represent the changing trends and periodic patterns of the time series. The overall state transition expression is:

[0074]

[0075] h t =o t ⊙tanh(c t )

[0076] in, For candidate cell state, o t For output gate, i t For the input gate, f t For the Gate of Oblivion, x t For the multi-level coding features of the previous layer, h t-1 For the hidden state of the previous layer, c t-1 For the previous cell state, c t This represents the current cell state. Through this state transition formula, the LSTM memory unit can continuously learn long-term dependencies across different time scales and filter out irrelevant interference, thereby effectively mitigating the vanishing or exploding gradient problem.

[0077] The input to the spatial interaction perception module in S3 is the attention-enhanced multi-level encoded feature F = X ∈ R^(B × D), where B is the batch size and D is the feature dimension. The attention-enhanced multi-level encoded feature is mapped to a graph structure G = (V, E), where node V represents sensor variables and edge E represents the dynamic relationship between variables. The interaction weights between variables in different feature subspaces are learned in parallel through a multi-head attention mechanism. Residual connections are used to combine the original features with the learned attention-enhanced feature representations.

[0078] The specific steps performed by the multi-head attention mechanism for each attention head include:

[0079] Calculate the attention coefficient between any two nodes i and j. The expression is:

[0080]

[0081] Among them, a k Let w be the learnable weight vector of the k-th attention head, used to map the concatenated features to a scalar attention coefficient. k Let v be the learnable weight matrix for the k-th attention head, used to perform linear transformations on the node features. i and v j These represent the original feature vectors of node i and node j, respectively.

[0082] The attention coefficients are normalized to obtain the attention weights. The expression is:

[0083]

[0084] in, The set of neighboring nodes of node i;

[0085] Based on attention weights, the features of neighboring nodes are weighted and aggregated to generate the output feature of node i at the k-th attention head. The expression is:

[0086]

[0087] The output features of the K attention heads are concatenated to obtain the final feature representation h′ of each node. i The expression is:

[0088]

[0089] The features processed by graph attention are residually concatenated with the attention-enhanced multi-level encoded features F to obtain intermediate features M:

[0090] M = F + GAT(F),

[0091] Where GAT(F) represents the output of the graph attention network;

[0092] After layer normalization of the intermediate feature M, it is input into a feedforward neural network for nonlinear transformation and dimensionality adjustment. The output of the feedforward neural network is then residually connected to the intermediate feature M again to output the final spatially enhanced feature representation F. att =M+FFN(LayerNorm(M)).

[0093] The U-Net decoder in S4 receives spatial enhancement features and uses a neighbor interpolation upsampling method to upsample the input features to obtain upsampled features. The upsampled features are then concatenated with the memory features of the corresponding level in the time dimension to form fused features. A one-dimensional convolution operation is performed on the fused features to achieve feature fusion and dimension adjustment, resulting in fused multi-scale temporal features, i.e., reconstructed data.

[0094] During the decoding stage, a multi-scale memory feature fusion mechanism based on nearest-neighbor interpolation upsampling is employed to ensure that the temporal memory features retained in the encoder can be effectively transferred to the decoder and participate in the final prediction. In the l-th layer decoding process, the decoder's input features include the memory features S processed by the temporal feature memory module from the l-th layer encoder. l and features sampled from layer l+1 These two inputs are concatenated along the time dimension and then mapped via a one-dimensional convolution to obtain the fused multi-scale temporal features.

[0095]

[0096] The training data flow diagram of the anomaly detection model in the multivariate time series anomaly detection method is shown below. Figure 2 As shown, during the forward diffusion process, a linear scheduling strategy is used to gradually inject Gaussian noise, disrupting the original data distribution step by step over 100 diffusion steps. In the reverse denoising stage, optimization is performed based on the noise prediction target, using the mean squared error loss function to measure the difference between the predicted and actual noise. During training, the focus is on the feature distribution of data segments; by learning the spatiotemporal feature distribution of normal patterns, the model can effectively identify anomalies based on reconstruction bias during the inference phase. The entire training process employs an end-to-end optimization strategy, simultaneously updating the parameters of each module to improve the overall model performance.

[0097] Compared to the implementation process, in the training data generation, the normalized time-series data is segmented according to a sliding window strategy, with the window length set to L = 256 and the step size S = 16. For D-dimensional channel data, the model training input samples X∈R are obtained by dividing the data according to the window size. (D×L)The training set samples are divided using a sliding window, which is used for noise injection during the forward diffusion process and feature learning by the reverse denoising network.

[0098] Furthermore, during training, Gaussian noise of varying levels is injected into each sample X, with the noise intensity determined by the linear scheduling parameter β. t The data X ∈ (0,1) is dynamically determined, where t is the number of diffusion steps. t In the input noise prediction model, the output predicted noise By comparing with real noise ∈ t By comparison, the mean squared error loss is calculated and the parameters are updated. Through repeated iterative training, the model gradually learns the data distribution of the normal pattern of the time series data.

[0099] During the inference phase, the same sliding window partitioning method is used to generate input samples X for the normalized test data. test ∈R D×L The test samples, after forward diffusion, are input into the inverse denoising network, and the reconstructed sample Z is obtained through a T-step inverse denoising process. Anomaly scores are calculated based on the mean squared error; if the anomaly score exceeds an adaptive threshold, the corresponding timestamp is considered anomaly. During the forward diffusion process, a linear scheduling strategy is used to progressively inject Gaussian noise, gradually disrupting the original data distribution over 100 diffusion steps. In the inverse denoising stage, optimization is performed based on the noise prediction target, using the mean squared error loss function to measure the difference between the predicted and actual noise. During training, the focus is on the feature distribution of data segments; by learning the spatiotemporal feature distribution of normal patterns, the model can effectively identify anomalies based on reconstruction bias during the inference phase. The entire training process employs an end-to-end optimization strategy, simultaneously updating the parameters of each module to improve the overall model performance.

[0100] The forward diffusion process progressively adds Gaussian noise ∈ to the normalized time-series data X, forming a noisy data sequence and constructing data representations at different noise levels. After t steps of the forward diffusion process, data X with a certain noise level is obtained. t :

[0101]

[0102] The reverse denoising process utilizes a noise prediction model trained within the diffusion model to progressively reconstruct noise data at different noise levels. This process uses a learned conditional probability distribution to determine the noise data X at the current time step by step. t The data is reconstructed into noise-free data X0 that more closely approximates the true distribution, i.e., Gaussian noise introduced during the forward diffusion process is gradually removed through multi-step denoising. The reconstruction prediction at each step is based on the denoising result of the previous step and the noise estimation function ε. θAfter a T-step reverse denoising process, the reconstructed data Z is output. Given d-dimensional time series data points X. i and its reconstructed value Z i Abnormal score Score(X) i The calculation uses the sum of squared errors along each dimension:

[0103]

[0104] like Figure 3 The image shows a portion of the anomalous segments from the SWAT dataset. The anomaly detection results for this portion of the SWAT dataset are as follows: Figure 4 As shown. It can be seen that, Figure 4 Abnormal score curves in almost all Figure 3 Significant peaks were generated when real abnormal events occurred. Within the normal operating range, i.e. Figure 3 During the time periods not marked as abnormal, the anomaly score curve remained stable without any obvious or erroneous peaks. Therefore, the peak values ​​of the anomaly scores in this embodiment typically closely match the start and end times of the abnormal events, clearly defining the boundaries of anomalies.

[0105] This embodiment constructs a multivariate time series anomaly detection method based on spatiotemporal representation and diffusion generation. By jointly learning the dynamic features of time series data and the spatial relationships between variables, it significantly improves the identification accuracy of minute and nonlinear anomalies in complex industrial environments compared to existing methods, as verified in the detection results shown in the attached figures. Through an improved U-Net structured encoder-decoder network, and by fusing multi-scale information during the feature transformation stage of noise prediction, the model simultaneously perceives local changes and global trends in time series data, enhancing its ability to detect anomaly patterns at different time granularities. For example, in chemical plants, it can simultaneously identify sudden changes in raw material flow and overall temperature shift anomalies, enabling early warning. A time series feature memory module is designed into the network, allowing the model to capture the time dependencies of complex periodic fluctuations and sudden events. Compared to traditional methods relying solely on sliding windows, this embodiment can identify periodic vibration anomalies and sudden load anomalies caused by wind speed changes in wind turbine condition monitoring, thereby more accurately determining the equipment's health status. This embodiment improves anomaly detection capabilities in high-dimensional multivariate time series scenarios by designing a spatial interaction sensing module that learns the spatial interaction relationships between sensors. Compared with existing methods that analyze each channel independently, this embodiment is able to discover potential abnormal patterns of correlation between variables.

[0106] The anomaly detection model has a compact overall design, with data processing, noise injection, reverse denoising, and anomaly detection forming a closed loop. It achieves end-to-end training and inference while maintaining the model's expressive power. The structure is unified and easy to expand, and it can be adapted to different industrial application scenarios.

[0107] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multivariate time series anomaly detection method, characterized in that, The method uses a pre-trained anomaly detection model, which is built based on a denoising diffusion probability model and includes a U-Net encoder, a U-Net decoder, a temporal feature memory module, and a spatial interaction perception module. The method includes the following steps: S1. Collect multivariate time series data generated by the sensor network of the industrial equipment to be tested; perform data preprocessing on the multivariate time series data to obtain standardized time series segments; S2. Input the time series segment into the anomaly detection model; perform multi-scale temporal feature extraction in the weight-normalized one-dimensional convolutional layer of the U-Net encoder to obtain multi-level encoded features; in the temporal feature memory module, enhance the long-term dependency features of the multi-level encoded features through LSTM memory units to obtain memory features; S3. Input the memory features into the spatial interaction perception module, construct the graph structure between variables through the graph attention network, calculate the attention weights, weighted aggregate the node features, and output the spatial enhancement features. S4. In the U-Net decoder, the corresponding memory features are concatenated with the upsampled features of the previous layer and fused through one-dimensional convolution to obtain the reconstructed data; the sum of squared errors in each dimension between the original time series segment and the reconstructed data is calculated to obtain the anomaly score; based on the anomaly score, it is determined whether an anomaly exists to obtain the anomaly detection result.

2. The multivariate time series anomaly detection method according to claim 1, characterized in that, The data preprocessing in S1 specifically includes calculating the normalized data for each sensor channel data set and the data points of that channel, expressed as: Where x is the standardized data, M is the data set of the sensor channel, m is the data point, and ColMin and ColMax are the minimum and maximum values ​​of the channel, respectively; The standardized data is segmented using a sliding window strategy to obtain time series segments.

3. The multivariate time series anomaly detection method according to claim 1, characterized in that, The U-Net encoder in S2 performs multi-layer convolution and multiple downsampling operations on time series segments to extract temporal features at different time scales, obtain multi-level encoded features, and performs linear attention calculation after each convolutional layer to obtain attention-enhanced feature representations.

4. The multivariate time series anomaly detection method according to claim 3, characterized in that, The specific steps in each convolutional layer include: extracting multi-scale temporal features from the output data of layer l-1 through a weight-normalized one-dimensional convolutional layer to obtain the primary feature representation of layer l; performing layer normalization on the primary feature representation of layer l to obtain the normalized features of layer l; concatenating the normalized features of layer l with the time step embedding vector to form the fused features of layer l; sequentially performing weight-normalized one-dimensional convolution, group normalization, and SiLU activation function processing on the fused features to obtain the encoding features of layer l; and performing residual concatenation between the encoding features of layer l and the encoding features processed by one-dimensional convolution to obtain the temporal feature tensor of layer l.

5. The multivariate time series anomaly detection method according to claim 3, characterized in that, The specific steps of the linear attention calculation include: performing a linear transformation mapping on the temporal feature tensor to obtain the query matrix, key matrix, and value matrix respectively; and transforming the standard dot product attention into a low-rank approximation form through kernel function mapping, reducing the computational complexity from O(n^2) to O(n^2). 2 The linear attention calculation result is reduced to O(n); the linear attention calculation result is transformed in dimension through the projection layer, and the transformed result is residually connected with the temporal feature tensor. After layer normalization, the attention-enhanced multi-level encoded feature representation is finally output.

6. The multivariate time series anomaly detection method according to claim 1, characterized in that, The specific steps for the LSTM memory unit in S2 to process the multi-level encoded features include: The multi-level encoded features of the current layer are concatenated with the hidden state of the previous layer. The forget gate controls the degree of memory of the cell state of the previous layer. The input gate controls the update degree of the current input multi-level encoded feature information. Candidate cell states are generated and the current cell state is updated. The output gate controls the output of the current hidden state. The current hidden state is calculated. The memory features output by the LSTM memory unit are passed to the U-Net decoder. In the U-Net decoder, the memory features are fused with the upsampled features of the corresponding layer decoder.

7. The multivariate time series anomaly detection method according to claim 1, characterized in that, The input to the spatial interaction perception module in S3 is the attention-enhanced multi-level coding feature F = X ∈ R^(B × D), where B is the batch size and D is the feature dimension. The attention-enhanced multi-level coding feature is mapped to a graph structure G = (V, E), where node V represents sensor variables and edge E represents the dynamic relationship between variables. The interaction weights between variables in different feature subspaces are learned in parallel through a multi-head attention mechanism. The original features are combined with the learned attention-enhanced feature representation using residual connections.

8. The multivariate time series anomaly detection method according to claim 7, characterized in that, The specific steps performed by the multi-head attention mechanism for each attention head include: Calculate the attention coefficient between any two nodes i and j. The expression is: Among them, a k Let w be the learnable weight vector of the k-th attention head, used to map the concatenated features to a scalar attention coefficient. k Let v be the learnable weight matrix for the k-th attention head, used to perform a linear transformation on the node features. i and v j These represent the original feature vectors of node i and node j, respectively. The attention coefficients are normalized to obtain the attention weights. The expression is: in, The set of neighboring nodes of node i; Based on attention weights, the features of neighboring nodes are weighted and aggregated to generate the output feature of node i at the k-th attention head. The expression is: The output features of the K attention heads are concatenated to obtain the final feature representation h of each node. ′ ′ The expression is:

9. The multivariate time series anomaly detection method according to claim 8, characterized in that, The features processed by graph attention are residually concatenated with the attention-enhanced multi-level encoded features F to obtain intermediate features M: M = F + GAT(F), Where GAT(F) represents the output of the graph attention network; After layer normalization, the intermediate feature M is input into a feedforward neural network for nonlinear transformation and dimensionality adjustment. The output of the feedforward neural network is then residually connected to the intermediate feature M again to output the final spatially enhanced feature representation F. att =M+FFN(LayerNorm(M)).

10. The multivariate time series anomaly detection method according to claim 1, characterized in that, The U-Net decoder in S4 receives spatial enhancement features and uses a neighbor interpolation upsampling method to upsample the input features to obtain upsampled features. The upsampled features are then concatenated with the memory features of the corresponding level in the time dimension to form fused features. A one-dimensional convolution operation is performed on the fused features to achieve feature fusion and dimension adjustment, resulting in fused multi-scale temporal features, i.e., reconstructed data.

Images