Multivariate time series anomaly detection method based on memory-augmented gan

Abstract

This invention relates to the field of equipment monitoring technology, specifically disclosing a multivariate time series anomaly detection method based on memory-enhanced GANs. The method includes: acquiring each input subsequence; performing a one-dimensional convolution operation on the input subsequence to obtain a hidden representation matrix; gating and updating the hidden representation using memory modules with a preset number of memory terms of different dimensions; mapping the memory-enhanced representation back to the original feature space using a linear projection structure decoder to output a reconstructed sequence; employing a lightweight multilayer perceptron discriminator to calculate the reconstruction error at each time step, obtaining soft and pseudo labels through a sigmoid function, and participating in the discriminator's binary classification training; and labeling all time steps in the anomaly interval as anomalies based on the anomaly score. This invention can alleviate the overfitting problem in traditional reconstruction methods where anomalies are also reconstructed too well, thereby improving the separability and detection accuracy of anomaly samples.

Description

Technical Field

[0001] This invention relates to the field of equipment monitoring technology, specifically to a multivariate time series anomaly detection method based on memory-enhanced GANs. Background Technology

[0002] Multivariate time series data is widely used in scenarios such as spacecraft telemetry, industrial process control, power dispatching, and server maintenance to reflect the state evolution of monitored systems across multiple sensor channels. This type of data is typically high-dimensional, has a high sampling frequency, and long runtime, making it crucial for equipment health monitoring, fault early warning, and safety protection. Abnormal fluctuations, structural shifts, or disruptions in correlation patterns in key variables can indicate potential faults, attacks, or operational instability. Failure to detect and address these issues promptly can easily lead to equipment damage, production interruptions, or even safety accidents. With the continuous expansion of sensor deployment and the increasing complexity of processes, time series data generated by modern industrial and infrastructure systems exhibit characteristics such as high dimensionality, strong correlation, non-stationarity, and slow drift in distribution over time. However, genuine anomalies are sparse in both time and space, making manual annotation extremely costly. Common statistical methods, single-class classification methods, and reconstruction-based methods are insufficient for the precise processing of multivariate time series data. Summary of the Invention

[0003] The purpose of this invention is to provide a multivariate time series anomaly detection method based on memory-enhanced GANs to solve the above-mentioned technical problems.

[0004] The objective of this invention can be achieved through the following technical solutions: A multivariate time series anomaly detection method based on memory-enhanced GANs includes the following steps: Transform a multivariate time series of a preset length into a sliding window sequence of a preset step size, and obtain each input subsequence; One-dimensional convolution is performed on the input subsequence. The convolution output is added to the position code and then input to a multi-head self-attention Transformer encoder consisting of multiple cascaded coding layers. The hidden representation matrix is ​​obtained through residual connection and layer normalization. Based on a pre-set number of memory modules with different dimensions, the hidden representation is gated for updates and memory readouts: The memory-enhanced representation is mapped back to the original feature space through a decoder with a linear projection structure, and the reconstructed sequence is output. A discriminator with a lightweight multilayer perceptron structure is used to calculate the reconstruction error at each time step and standardize it to a z-score. Soft labels and pseudo labels are obtained through the Sigmoid function. Only time segments with normal pseudo labels are selected to participate in the binary classification training of the discriminator. The anomaly score for each time step is calculated, and the PA interval expansion mechanism is adopted. When the proportion of predicted anomaly time steps in the actual anomaly interval exceeds the preset threshold, all time steps in that interval are marked as anomalies.

[0005] As a further aspect of the present invention: in the memory enhancement process, based on a preset number of memory modules with different dimensions, the hidden representation is subjected to gating update and memory readout, including a gating update stage and a memory readout stage, wherein the gating update stage specifically includes: Gated update phase: Calculate the similarity scalar between each memory item and the query vector, obtain the attention weight by softmax normalization, aggregate related queries to generate memory update signals, and update the memory items by controlling the degree of fusion between new and old information through gating vectors.

[0006] As a further aspect of the present invention: the memory readout stage specifically includes: Memory readout phase: Calculate the attention weights of the query vector and memory items, sum the weighted values ​​of all memory items to obtain the memory readout vector, and concatenate the original query vector and the memory readout vector along the feature dimension to form a memory-enhanced representation matrix.

[0007] As a further aspect of the present invention, it also includes: adopting a two-stage training paradigm, in which online gating updates are performed on all training windows in the first stage, and K-means clustering is performed after concatenating the window-level feature vector with the current memory item in the second stage, and incremental updates are performed on each memory item using only the window set of its cluster.

[0008] As a further aspect of the present invention: the K-means clustering is initialized using existing memory entries as cluster centers.

[0009] As a further aspect of the present invention: each coding layer of the Transformer encoder includes a multi-head self-attention sub-layer and a feedforward network sub-layer, and residual connections and layer normalization structures are provided outside the sub-layers.

[0010] The beneficial effects of this invention are: This invention innovatively introduces an external memory module into the generative adversarial framework, which explicitly uses a set of learnable memory terms as normal pattern prototypes. Before decoding and reconstruction, the latent representation of the encoder is explicitly aligned and constrained, which effectively weakens the ability of the deep autoencoder to express abnormal patterns and alleviates the overfitting problem in traditional reconstruction methods where abnormalities are also reconstructed too well. This significantly improves the separability and detection accuracy of abnormal samples. This invention also incorporates a selective memory update mechanism based on K-means clustering, performing gated updates only on memory items associated with the current training sample. This makes the internal patterns of the memory bank more stable, avoids mutual interference between different patterns, and improves the robustness and interpretability of the memory representation. Compared to existing multivariate time series anomaly detection methods, this invention has been systematically evaluated on four real-world standard benchmark datasets (MSL, SMAP, SMD, and SWlaT) from aerospace telemetry, server operation monitoring, and industrial control. The overall performance significantly outperforms the current state-of-the-art methods, demonstrating that the proposed memory-enhanced generative adversarial framework has good versatility and engineering application value, making it suitable for widespread deployment in various time series monitoring scenarios. Attached Figure Description

[0011] The invention will now be further described with reference to the accompanying drawings.

[0012] Figure 1 This is a flowchart illustrating the multivariate time series anomaly detection method based on memory-enhanced GAN of this invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] Existing technologies can be mainly divided into three categories: statistical methods (such as ARIMA and VAR), which typically rely on assumptions such as linearity and Gaussian stationarity, and are insufficient in characterizing nonlinear correlations and long-term dependencies among multiple variables; single-class classification methods (such as DeepSVDD and THOC), which require manually designed features and are prone to problems such as threshold sensitivity, weak generalization ability, and difficulty in cross-scene transfer in scenarios with a large number of variables, high sampling frequency, and slow drift of distribution over time; and reconstruction-based methods (such as the early linear model RPCA, deep learning methods such as multi-scale convolutional coding and decoding, and adversarial reconstruction GAN architectures that introduce discriminators) have made progress. For example, RPCA extracts the principal structure through a low-rank non-negative subspace and identifies anomalies by residual or sparse term amplitudes, but is limited by the assumptions of linearity and global subspace, and is prone to residual leakage or over-smoothing in scenarios with strong nonlinearity, long-term dependencies, and cross-channel time-varying coupling. Deep autoencoder-based methods model cross-channel and cross-time using convolution, recursion, and attention.

[0015] However, existing state-of-the-art reconstruction-based deep learning methods still have key drawbacks: this architecture relies on the powerful representation learning capabilities of deep networks, achieving good performance in high-dimensional, multivariate scenarios. However, due to the strong expressive power of the encoder and decoder, while learning normal patterns, they may also reconstruct anomalous patterns appearing in the training data, leading to severe overlap in the distribution of normal and anomalous samples in the reconstruction error space, insufficient anomalous score discrimination, and a tendency to produce false positives and false negatives. Further anomaly detection methods based on GANs distinguish between real and reconstructed samples in the output space through a discriminator, thereby regularizing the generator at the data distribution level. However, these methods mainly focus on the authenticity of the reconstruction results, lacking explicit structural modeling of the encoder's latent representation space. Even if anomalous information is mixed into the latent features, with strong decoder and discriminator capabilities, the model may still reconstruct anomalous samples in a way similar to normal samples, failing to significantly alleviate overfitting to anomalous patterns. Meanwhile, a few existing methods have attempted to introduce memory modules into autoencoders or GANs, but they often adopt a uniform and indiscriminate update strategy for all training windows. That is, each memory item is frequently written to all samples, which can easily cause multiple memory items to compete for the same pattern or mix different patterns, resulting in ambiguous memory semantics. They lack selective memory update mechanisms based on clustering or prototypes, and do not impose constraints on the attention distribution of memory use, making it difficult to make full use of the limited memory capacity, and the interpretability is poor.

[0016] Please see Figure 1 As shown, this invention is a multivariate time series anomaly detection method based on memory-enhanced GAN, including a Transformer-based encoder and weak decoder module, a memory module with a gating update mechanism, and a discriminator, which are used to apply explicit constraints to the encoder, further alleviate the overfitting problem, and improve the accuracy of multivariate time series anomaly detection.

[0017] The encoder adopts a Transformer-based autoencoder structure. First, the input subsequence W... j ∈R w×n (W represents the window length, and n represents the variable dimension) Perform a one-dimensional convolution operation to extract short-term relevant features in the local temporal neighborhood, and add the convolution output to the positional encoding to obtain a sequence representation containing temporal positional information.

[0018] The sequence is then input into a multi-head self-attention Transformer encoder consisting of three cascaded coding layers. Each encoder layer includes a multi-head self-attention sublayer and a feedforward network sublayer, with residual connections and layer normalization structures applied outside the two sublayers, respectively. After three layers are stacked, the hidden representation matrix is ​​obtained. , where d model This represents the model dimension of the encoder.

[0019] The memory module is used to enhance the memory of the hidden representation. This module is pre-set to E={e k There are (k=1,...,K) memory entries, each memory entry is d. model A dimensional vector is used to display the prototype features storing different normal patterns. The hidden representation q... j Each row vector in the vector is used as a query vector. For ease of representation, we flatten it into: , where N is the number of queries involved in the update.

[0020] During the gating update phase, the Kth memory item e is first calculated. k With query vector q i The similarity between them is used to obtain the similarity scalar. k,i =e k q i Using this as input, softmax normalization is performed on the query dimension to obtain the memory term e. k Attention weights for each query .

[0021] Based on these weights, it will be related to the memory item e. k Related queries are aggregated into memory update signals Then through the gate vector Controlling the degree of integration between new and old information, memory items are accessed through... Update.

[0022] During the query update phase, for each query vector output by the encoder, the attention weights between it and the memory terms are first calculated. Then, based on this attention, a weighted sum is performed on all memory items to obtain the memory readout vector. Finally, the original query vector and the memory read vector are concatenated along the feature dimension to form a memory-enhanced representation matrix. .

[0023] K-means clustering-driven memory update. MemGAN employs a two-stage memory enhancement training paradigm for complete training. The first stage starts with randomly initialized memory items and uses a gating update strategy to update all training windows online, thus obtaining a preliminary memory representation aligned with the normal pattern. In the second stage, the memory module no longer performs indiscriminate online updates across all training windows, but instead uses an intra-cluster memory update strategy for fine-tuning. Specifically, for a batch in the second stage training, the query vectors output by the encoder are first aggregated along the time dimension to obtain each time window W. j Window-level feature vectors These window representations are then concatenated with the current memory entry, and a K-means clustering is performed in the feature space. The existing memory entries are used as initialization for the cluster centers, ensuring that the clustering process revolves around existing normal prototypes rather than starting from random centers. Each memory entry is incrementally updated using only the set of windows assigned to its cluster, through attention and gating mechanisms.

[0024] The decoder employs a linear projection structure to enhance the memory representation. Output the reconstructed sequence

[0025] The discriminator employs a lightweight multilayer perceptron structure to perform time-step discrimination on the representation of time series in the feature space.

[0026] Calculate the reconstruction error at each time step The error is standardized along the sequence dimension to obtain a standardized error score (z-score) for each time step. Then, a sigmoid function transformation combined with random sampling is used to map the error to soft labels and corresponding pseudo-labels between 0 and 1. During adversarial training, only time segments judged as normal by the pseudo-labels are selected for discriminator training (where the original sequence segments serve as "real" sample inputs with a corresponding class label of 1, and the reconstructed sequence segments serve as "generated" sample inputs with a corresponding class label of 0), and the discriminator performs binary classification learning.

[0027] Loss functions include: Weighted reconstruction loss: ; in The adaptive weights are dynamically calculated based on the statistical characteristics of the reconstruction error and the training process. Combat losses: ; Entropy loss: ; Total loss: .

[0028] Abnormal scores are calculated as follows If the score exceeds the threshold, the time step is considered an anomaly. To improve consistency across the time dimension and mitigate isolated misjudgments, this invention further employs a PA interval expansion mechanism. Given an anomaly coverage threshold K, the prediction results are post-processed at the interval level, using true anomaly intervals as units. When the proportion of time steps predicted as anomalies within a true anomaly interval exceeds K% of the interval length, all time steps within that interval are uniformly marked as anomalies for final anomaly evaluation and judgment.

[0029] The training process alternately optimizes the encoder-decoder and discriminator: the encoder-decoder learns normal data patterns, and the discriminator distinguishes between real and generated data, providing feedback to improve reconstruction quality.

[0030] The overall implementation process is as follows: given a multivariate time series of length T The goal of anomaly detection is for each observation x t Each label y t y t ∈0,1 (0 represents normal, 1 represents abnormal). To learn the correlation between observations and surrounding observations, the original sequence is typically transformed into a sliding window sequence with a step size of s. The reconstruction-based anomaly detection method first obtains the sequence representation H through an encoder. j Then the decoder generates the corresponding reconstruction. Then, the difference between the input and the reconstructed data is calculated as the reconstruction error, which will be used as the anomaly score of the input data. Finally, if the anomaly score d corresponding to the observation is... w If the value exceeds a given threshold, the observation is considered an outlier.

[0031] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A multivariate time series anomaly detection method based on memory-enhanced GANs, characterized in that, Includes the following steps: Transform a multivariate time series of a preset length into a sliding window sequence of a preset step size, and obtain each input subsequence; One-dimensional convolution is performed on the input subsequence. The convolution output is added to the position code and then input to a multi-head self-attention Transformer encoder consisting of multiple cascaded coding layers. The hidden representation matrix is ​​obtained through residual connection and layer normalization. Based on a pre-set number of memory items of different dimensions, the hidden representation is subjected to gating updates and memory readouts: The memory-enhanced representation is mapped back to the original feature space through a decoder with a linear projection structure, and the reconstructed sequence is output. A discriminator with a lightweight multilayer perceptron structure is used to calculate the reconstruction error at each time step and standardize it to a z-score. Soft labels and pseudo labels are obtained through the Sigmoid function. Only time segments with normal pseudo labels are selected to participate in the binary classification training of the discriminator. Calculate the anomaly score for each time step and use the PA interval expansion mechanism. When the proportion of predicted anomaly time steps in the actual anomaly interval exceeds a preset threshold, mark all time steps in that interval as anomalies.

2. The multivariate time series anomaly detection method based on memory-enhanced GAN according to claim 1, characterized in that, In the memory enhancement process, the memory modules based on a preset number of memory items of different dimensions perform gating updates and memory readouts on the hidden representation, including a gating update stage and a memory readout stage. The gating update stage specifically includes: Gated update phase: Calculate the similarity scalar between each memory item and the query vector, obtain the attention weight by softmax normalization, aggregate related queries to generate memory update signals, and update the memory items by controlling the degree of fusion between new and old information through gating vectors.

3. The multivariate time series anomaly detection method based on memory-enhanced GAN according to claim 2, characterized in that, The memory retrieval stage specifically includes: Memory readout phase: Calculate the attention weights of the query vector and memory items, sum the weighted values ​​of all memory items to obtain the memory readout vector, and concatenate the original query vector and the memory readout vector along the feature dimension to form a memory-enhanced representation matrix.

4. The multivariate time series anomaly detection method based on memory-enhanced GAN according to claim 1, characterized in that, It also includes: adopting a two-stage training paradigm. In the first stage, online gating updates are performed on all training windows. In the second stage, K-means clustering is performed after concatenating the window-level feature vector with the current memory item. For each memory item, incremental updates are performed using only the window set of its own cluster.

5. The multivariate time series anomaly detection method based on memory-enhanced GAN according to claim 4, characterized in that, The K-means clustering is initialized using existing memory entries as cluster centers.

6. The multivariate time series anomaly detection method based on memory-enhanced GAN according to claim 1, characterized in that, Each coding layer of the Transformer encoder includes a multi-head self-attention sublayer and a feedforward network sublayer, and residual connections and layer normalization structures are set outside the sublayers.

Images