Multi-dimensional time sequence unsupervised anomaly detection method based on recurrent neural network
By using stacked adversarial variational recurrent neural networks (SaVRNN) for multi-layer dynamic probability modeling, the problem of limited representation ability in multi-dimensional time series is solved, achieving more efficient anomaly detection and improving detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2026-01-21
- Publication Date
- 2026-05-08
AI Technical Summary
Existing unsupervised anomaly detection methods suffer from limited representation capabilities and simple hidden layer distribution in multidimensional time series, making them unsuitable for complex data distribution scenarios.
Stacked adversarial variational recurrent neural networks (SaVRNN) are used for multi-layer dynamic probability modeling. The latent variables of multivariate time series are captured through unsupervised learning, and adversarial optimization is performed by combining a discriminant network to calculate anomaly scores to identify data anomalies.
It significantly improves the accuracy and robustness of multidimensional time series anomaly detection, enabling a more comprehensive characterization of anomalies and enhancing detection performance.
Smart Images
Figure CN121997047A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of anomaly detection technology, specifically relating to a multidimensional time series unsupervised anomaly detection method based on recurrent neural networks. Background Technology
[0002] In recent years, with the rapid development of deep learning technology, significant progress has been made in the field of time series anomaly detection. Unsupervised deep learning network models, with their powerful adaptive capabilities, have shown obvious advantages in handling long-short-term dependencies and nonlinear relationships. Compared with traditional statistical models and machine learning methods, deep learning methods can be directly applied to raw time series data, extracting key patterns without complex feature engineering.
[0003] With the improvement of computing power and the widespread application of graphics processing units (GPUs), the training efficiency of unsupervised deep learning models in big data environments has significantly improved, and they have been widely applied to time series anomaly detection tasks in various fields such as finance, energy, and transportation. Unsupervised deep learning-based methods have the following significant advantages: 1. Ability to handle complex nonlinear relationships: Unsupervised deep learning models can capture complex patterns and long-term and short-term dependencies in data, thereby generating high-precision anomaly detection results.
[0004] 2. High efficiency in processing large-scale data: Unsupervised deep learning models can not only process massive amounts of data quickly, but also effectively capture the complex relationship between input and output through continuous iterative learning.
[0005] 3. Flexibility and adaptability: Unsupervised deep learning models can adjust their architecture, parameters and training methods according to specific needs, and can be combined with other statistical or machine learning models to form hybrid models to adapt to detection tasks in different fields.
[0006] Learning normal patterns in MTS (Multivariate Times Series) data is a crucial step in anomaly detection and has received increasing attention in engineering and applied sciences in recent years. In light of this, several machine learning-based methods have been proposed. As representative studies, EncDec-AD uses an LSTM (Long Short-Term Memory)-based encoder-decoder to capture normal patterns in the temporal dependencies of MTS data and identifies anomalies based on reconstruction errors. Telemanom utilizes LSTM to predict values in spacecraft telemetry channels and detects anomalies based on the residuals between predicted and observed values. Furthermore, MSCRED introduces a multi-scale convolutional recurrent encoder-decoder to learn spatial correlations and temporal characteristics in MTS data and then uses the residual feature matrix to detect anomalies. MAD-GAN uses LSTM-RNN as the basic framework of a Generative Adversarial Network (GAN) model to capture normal spatiotemporal patterns, while OmniAnormaly introduces a Stochastic Recurrent Neural Network (SRNN) to help learn more robust representations. SDFVAE introduces a framework based on static and dynamic factorized variational autoencoders to explicitly learn time-invariant and time-varying properties. Both OmniAnormaly and SDFVAE utilize reconstructed likelihood for anomaly detection, as lower likelihood correlates with a higher probability of anomaly. However, it's noteworthy that these unsupervised works employ shallow RNN-based structures to capture temporal dependencies in time series data points (MTS). Furthermore, some unsupervised anomaly detection methods neglect the temporal dependencies between different moments in the MTS. For example, the unsupervised method DOMI combines a Gaussian mixture variational autoencoder (GmVAE) with one-dimensional convolutions to detect anomalous machine instances in large data centers. DAGMM uses autoencoders to learn representations and a Gaussian Mixture Model (GMM) for distribution estimation. USAD designs an autoencoder based on adversarial training for unsupervised anomaly detection. To model non-stationarity in time series, GmSVRNN introduces mixture models and switching mechanisms into VRNNs. To further consider the correlation between channels in MTS, some unsupervised anomaly detection models based on graph neural networks have been proposed. Among them, the Deep Variational Graph Convolutional Recurrent Network (DVGCRN) creatively integrates graph network structure into a dynamic probabilistic generative network and achieves excellent performance.
[0007] Although existing unsupervised anomaly detection methods are very effective in some practical scenarios, on the one hand, these methods are mostly single-layer structures, which limits their representation capabilities; on the other hand, although deep unsupervised probabilistic dynamic modeling methods have been developed to model the temporal and structural features in MTS, their hidden layers are often limited to simple distribution forms, which cannot adapt to complex data distribution scenarios. Summary of the Invention
[0008] To address the aforementioned problems in the existing technology, this invention provides a multidimensional time series unsupervised anomaly detection method based on recurrent neural networks.
[0009] The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a multidimensional time series unsupervised anomaly detection method based on a recurrent neural network, the method comprising: The multivariate time series to be detected is input into the trained unsupervised multilayer dynamic probability model to obtain the latent variables corresponding to the multivariate time series to be detected; wherein, the unsupervised multilayer dynamic probability model is a stacked adversarial variational recurrent neural network; the unsupervised multilayer dynamic probability model is trained by maximizing ELBO after constructing an optimization objective according to the VAE model training strategy and the discriminant network; An anomaly score is calculated based on the latent variables corresponding to the multivariate time series to be detected and the multivariate time series to be detected. When the anomaly score is lower than a preset threshold, the multivariate time series to be detected is determined to be abnormal data; When the anomaly score is higher than or equal to the preset threshold, the multivariate time series to be detected is determined to be normal data.
[0010] Optionally, the training process of the unsupervised multilayer dynamic probability model includes: Input the sample time series into the unsupervised multilayer dynamic probability model to obtain the sample latent variables and reconstruct the sample time series; The sample latent variables, the sample time series, and the sample latent state of the previous time step are input into the stacked recurrent neural network to obtain the sample latent state of the current time step. The variational distribution of the latent variables of the current time step is obtained based on the random noise, the sample time series, the sample latent variables, and the sample latent state of the previous time step. The inference network variational distribution is constructed based on the stacked adversarial variational recurrent neural network, the reconstructed sample time series, and the sample hidden state at the current time step; The initial optimization objective of the unsupervised multilayer dynamic probability model is constructed according to the VAE model training strategy, and the initial optimization objective is adversarially optimized using a discriminant network to obtain the optimization objective. The optimization objective is subjected to ELBO maximization to train the unsupervised multilayer dynamic probability model, resulting in the trained unsupervised multilayer dynamic probability model.
[0011] Optionally, the latent variables of the samples are represented as follows: ; in, express The time series of time samples at time point The time series data at the _th ... The latent variables of the samples in the layer follow a Gaussian distribution. , This represents the total number of layers in an unsupervised multilayer dynamic probabilistic model. for The mean, for variance This indicates taking the diagonal elements.
[0012] Optionally, the reconstructed sample time series is represented as follows: ; in, express The time series of time samples at time point Reconstructed sample time series corresponding to each time series data point, which follows a Gaussian distribution. for The mean, for The variance.
[0013] Optionally, the hidden state of the sample at the current time step is represented as follows: ; in, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer, Indicates the first Long Short-Term Memory Network of Layers Indicates a fully connected network. Indicates will and To splice, express The first time in the time series of the sample time. A time series data, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer.
[0014] Optionally, the variational distribution of the inference network is represented as follows: ; in, This represents the variational distribution of the inference network. Represents random noise The probability distribution obtained by entering the inference network.
[0015] Optionally, the optimization objective is expressed as follows: ; in, This represents the optimization objective. This represents the number of time points, indicating the total number of time series data in the sample time series. This indicates that we are calculating the expectation of the logarithm of the likelihood. express The time series of time samples at time point The time series data at the _th ... Random noise in the layer, The parameters represent the decoding network of an unsupervised multilayer dynamic probabilistic model. This represents the encoding network parameters of an unsupervised multilayer dynamic probabilistic model. To adjust the first Hyperparameters of layer adversarial loss For the completed training The optimal discriminator for the layer, Indicate The time series of time samples at time point The time series data at the _th ... Standard Gaussian noise of the layer.
[0016] Secondly, the present invention provides a multidimensional time series unsupervised anomaly detection device based on a recurrent neural network, the device comprising: The latent variable acquisition module is used to input the multivariate time series to be detected into the trained unsupervised multilayer dynamic probability model to obtain the latent variables corresponding to the multivariate time series to be detected; wherein, the unsupervised multilayer dynamic probability model is a stacked adversarial variational recurrent neural network; the unsupervised multilayer dynamic probability model is trained by maximizing ELBO after constructing an optimization objective according to the VAE model training strategy and the discriminant network; An anomaly scoring module is used to calculate an anomaly score based on the latent variables corresponding to the multivariate time series to be detected and the multivariate time series to be detected. The first judgment module is used to determine that the multivariate time series to be detected is abnormal data when the anomaly score is lower than a preset threshold. The second judgment module is used to determine that the multivariate time series to be detected is normal data when the abnormal score is higher than or equal to the preset threshold.
[0017] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: In the above technical solution, the present invention captures latent variables in multivariate time series through an unsupervised multilayer dynamic probability model, combines an advanced probability generation framework with a novel anomaly detection strategy, and significantly improves the performance of the MTS anomaly detection task; it also achieves effective training of the unsupervised multilayer dynamic probability model by combining anti-training; and it calculates anomaly scores by latent variables and multivariate time series, which can more comprehensively characterize the anomalies of multivariate time series, and significantly improve the accuracy and robustness of detection.
[0018] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0019] Figure 1 This is a flowchart of a multidimensional time series unsupervised anomaly detection method based on recurrent neural networks provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the hierarchical generation process of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a stacked loop process of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of upward-downward inference of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a multi-layer adversarial model of an unsupervised multi-layer dynamic probability model provided in an embodiment of the present invention; Figure 6This is a schematic diagram illustrating the performance change of an unsupervised multilayer dynamic probability model on the SMD dataset provided in an embodiment of the present invention. Figure 7a This is a schematic diagram of the abnormal score distribution of a 7-layer unsupervised multilayer dynamic probability model provided in an embodiment of the present invention; Figure 7b This is a schematic diagram of the abnormal score distribution of an unsupervised multilayer dynamic probability model with a single-layer structure provided in an embodiment of the present invention; Figure 7c This is a schematic diagram of the abnormal score distribution of an unsupervised multilayer dynamic probability model with a single-layer structure provided in an embodiment of the present invention; Figure 7d This is a schematic diagram of the abnormal score distribution of an unsupervised multilayer dynamic probability model with a single-layer structure provided in an embodiment of the present invention; Figure 7e This is a schematic diagram of anomaly score distribution in an SVRNN provided by an embodiment of the present invention; Figure 7f This is a schematic diagram of the abnormal score distribution of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention; Figure 8 This is a block diagram of a multidimensional time series unsupervised anomaly detection device based on a recurrent neural network provided in an embodiment of the present invention. Detailed Implementation
[0020] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.
[0021] Figure 1 This is a flowchart of a multidimensional time series unsupervised anomaly detection method based on recurrent neural networks provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps: S101. Input the multivariate time series to be detected into the trained unsupervised multilayer dynamic probability model to obtain the latent variables corresponding to the multivariate time series to be detected; wherein, the unsupervised multilayer dynamic probability model is a stacked adversarial variational recurrent neural network; the unsupervised multilayer dynamic probability model is trained by maximizing ELBO after constructing the optimization objective according to the VAE model training strategy and the discriminant network.
[0022] Optionally, the training process of an unsupervised multilayer dynamic probabilistic model includes: Input the sample time series into an unsupervised multilayer dynamic probability model to obtain the sample latent variables and reconstruct the sample time series; The sample latent variables, sample time series, and sample latent state of the previous time step are input into the stacked recurrent neural network in the unsupervised multilayer dynamic probability model to obtain the sample latent state of the current time step. The variational distribution of the latent variables of the sample at the current time step is obtained based on random noise, sample time series, sample latent variables, and the sample latent state of the previous time step. The variational distribution of the inference network is constructed based on the stacked adversarial variational recurrent neural network, the reconstructed sample time series, and the sample hidden state at the current time step. The initial optimization objective of the unsupervised multilayer dynamic probability model is constructed based on the VAE model training strategy, and the initial optimization objective is adversarially optimized using a discriminant network to obtain the optimization objective. The optimization objective is maximized using ELBO to train the unsupervised multilayer dynamic probability model, resulting in a trained unsupervised multilayer dynamic probability model.
[0023] Understandably, we can first define the anomaly detection problem for sample time series: to detect anomalies in the sample time series... Each time series data is defined as ,here and It refers to the amount of time-series data collected. yes Duration, The observation vector at time is , This represents the dimension of the vector at a certain moment, therefore .
[0024] This invention proposes an unsupervised multilayer dynamic probabilistic model that not only captures long-range temporal correlations but also considers the structural features of multiple layers. It inherits the advantages of deep probabilistic networks, and the implicitly distributed latent variables and adversarial training method ensure the representational power of the hidden layers. It is equipped with a powerful probabilistic progressive generative model, a deep dynamic model, and an autoencoder inference model. Figure 2 This is a schematic diagram illustrating the hierarchical generation process of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the top-down probability generation process of SaVRNN (Stacked adversarial variational recurrent neural network) can be represented as: ; in, express The time series of time samples at time point The time series data at the _th ... The latent variables of the samples in the layer follow a Gaussian distribution. , This represents the total number of layers in an unsupervised multilayer dynamic probabilistic model. for The mean, for variance This indicates taking the diagonal elements. Indicates the first The deterministic hidden state of samples in the layer. Represents a deterministic nonlinear function. express The time series of time samples at time point Reconstructed sample time series corresponding to each time series data point, which follows a Gaussian distribution. for The mean, for variance yes Layer factor loading matrix, This indicates that the source is a latent variable in the sample. This indicates the value used to generate the mean of the current layer. This represents the variance generated for the current layer. Indicates the first Layer connection sample hidden state and sample latent variables The transition matrix. This unsupervised multilayer dynamic probabilistic model integrates several advanced techniques within a well-structured Bayesian framework: it employs stacked recurrent neural networks to effectively model multi-layered, long-range temporal dependencies; it introduces hierarchical variational autoencoders to enhance the expressive power of latent representations through hierarchical prior mechanisms; and it combines adversarial training strategies to drive the model to learn richer latent representations that more closely resemble the distribution of real data. The model's generation distribution. Defined as mean and variance respectively and The distribution is a Gaussian distribution that depends on the probabilistic hidden state. and cyclic hidden state . and These are the learnable parameters in this unsupervised model.
[0025] Figure 3 This is a schematic diagram of a stacked cyclic process of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention, as shown below. Figure 3 As shown, in order to capture latent structural features at multiple scales more deeply, stacked recurrent neural networks are used to process the latent states of samples. Updates. For example... Figure 3 As shown, the following formula is used to update the hidden state of a sample: ; in, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer, Indicates the first Long Short-Term Memory Network of Layers Indicates a fully connected network. Indicates will and The concatenation process combines probabilistic hidden representations with dynamic hidden state variables, introducing probabilistic hidden representation information with uncertain and hierarchical structural characteristics into the stacked recurrent neural network. express The first time in the time series of the sample time. A time series data, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer.
[0026] Figure 4 This is a schematic diagram of upward-downward inference of an unsupervised multilayer dynamic probabilistic model provided in an embodiment of the present invention, as shown below. Figure 4 As shown, based on the modeling idea of variational autoencoder, in order to effectively estimate the posterior distribution of latent variables, the sample latent variables are first constructed. The variational distribution is used to approximate the true prior distribution, and it is decomposed into a bottom-up structure for hierarchical modeling: ; Among them, the most basic sample latent variables Directly dependent on input That is, inferred from observational data; latent variables in high-level samples. This depends on the latent variables of the previous layer. This constitutes a hierarchical reasoning approach, giving the entire variational distribution a hierarchical structure. Compared to traditional variational reasoning methods that directly assign... Explicit representations can make it difficult for hierarchical probability generation models to accurately depict the true posterior distribution, especially for such models. In contrast, implicitly distributed variational latent variables can be introduced. Specifically, The definition is as follows: ; in, Random noise with a standard normal distribution The hyphen (-) represents learnable parameters, and the hyphen (-) indicates that specific conditions need further refinement. To address the prior distribution collapse problem that may be caused by high-level random variables in hierarchical Bayesian structures, this invention proposes an improved scheme: First, a deterministic bottom-up information transmission path is constructed, connecting the hidden states of the recurrent neural network samples with the hidden variables of each layer (from...). arrive Establish connections; secondly, introduce input... The mapping relationship to the parameters of each layer. The final result is the first... The variational distribution of latent variables in the layer samples is represented as follows: ; in, express The time series of time samples at time point The reconstructed sample time series corresponding to each time series data point follows a Gaussian distribution; Indicates the previous time step The hidden states of the samples are passed down. Then, the stacked adversarial variational recurrent neural network constructs a top-level ( From the bottom layer ( Variational posterior of the hidden state of the sample: ; Representing the inference network for latent variables Approximate posterior distribution estimation, based on the structure of the inference network, such as Figure 3 Stacked adversarial variational recurrent neural networks estimate the posterior probability of latent variables in samples by integrating bottom-up likelihood information and top-down prior information from the generative distribution. The inference variational distribution is shown below: ; in, Represents the variational distribution of the inference network. Represents random noise The probability distribution obtained by entering the inference network. As shown in the above equation, the stacked adversarial variational recurrent neural network directly depends on the input by making the latent variables of the random samples in the higher layers directly dependent on the input. First, to ensure that the latent variables of the samples can fully capture the effective information in the input, thereby avoiding the posterior failure problem. Second, during inference, the model constructs a richer latent representation by integrating the latent states of the current layer and the dynamic information of the latent variables of higher layers. Furthermore, by analyzing the latent variables of the samples... By adopting an implicit variational distribution, the expressive power of the model is further enhanced, enabling it to more flexibly integrate information from multiple sources.
[0027] Figure 5 This is a schematic diagram of a multi-layer adversarial model of an unsupervised multi-layer dynamic probabilistic model provided in an embodiment of the present invention, as shown below. Figure 5 As shown, based on the training strategy of the VAE model, the optimization objective of SaVRNN can be maximized by maximizing ELBO (Evidence Lower Bound), as expressed below: ; in, This represents the expected value of the likelihood logarithm, used to ensure that the model can better reconstruct the input data. Denotes the KL divergence, which constrains the variational distribution and the prior distribution. The similarity. Indicates the parameters of the decoding network; This represents the parameters of the encoding network. The above equation can be optimized using the logarithmic difference form of the KL divergence as follows: ; in, This represents the expected value of the time series distribution of the sample. The expression means sampling on the sample time series. Then, under this condition, the latent variables are sampled and averaged.
[0028] Inspired by adversarial variational networks, to optimize the hidden states of the implicit variational distribution, a discriminative network can be introduced at each layer. Measuring the difference between two distributions: ; The optimal discriminant for each layer is trained as follows: ; Based on the parameter re-parameter technique, the initial optimization objective can be expressed as follows: ; By optimizing the adversarial approach at each layer, inference about the latent variables of the implicit distribution variation can be achieved. The optimization objective can be achieved by maximizing the ELBO (Evidence Lower Bound), which is expressed as: ; in, Indicates the optimization objective. This represents the number of time points, indicating the total number of time series data in the sample time series. This indicates that we are calculating the expectation of the logarithm of the likelihood. express The time series of time samples at time point The time series data at the _th ... Random noise in the layer, The parameters represent the decoding network of an unsupervised multilayer dynamic probabilistic model. This represents the encoding network parameters of an unsupervised multilayer dynamic probabilistic model. To adjust the first Hyperparameters of layer adversarial loss For the completed training The optimal discriminator for the layer, Indicate The time series of time samples at time point The time series data at the _th ... The layer contains standard Gaussian noise; the first term is the expectation of the log-likelihood, ensuring the model maintains excellent reconstruction performance; the second term is the adversarial loss term, adjusted by... This causes the variational distribution to approximate the prior distribution. It is worth noting that, as an unsupervised deep hierarchical dynamic probabilistic generative model, SaVRNN still faces the typical challenge of "posterior failure." To address this, a "warm-up" training strategy can be introduced to stabilize training. "Warm-up" initially trains parameters using only reconstruction errors, then progressively introduces adversarial loss during the first training cycle, dynamically adjusting hyperparameters from 0 to 1. This training mechanism effectively stabilizes the model training process, significantly alleviates the posterior distribution collapse problem, and provides a more reliable training scheme for hierarchical probabilistic generative models.
[0029] S102. Calculate the anomaly score based on the latent variables corresponding to the multivariate time series to be detected and the multivariate time series to be detected. S103. When the anomaly score is lower than the preset threshold, the multivariate time series to be detected is determined to be abnormal data; S104. When the abnormal score is higher than or equal to the preset threshold, the multivariate time series to be detected is judged to be normal data.
[0030] Understandably, unsupervised multilevel dynamic probabilistic models employ unsupervised learning strategies to model normal patterns in multivariate time series and utilize reconstruction probabilities as the basis for anomaly detection. Specifically, given... Anomaly scores are calculated using the following formula: ; A higher score indicates better reconstruction and a more consistent detection pattern. A threshold is then dynamically set based on the anomaly score; if the anomaly score... Data below a preset threshold is considered anomalous. In unsupervised anomaly detection tasks, accurately modeling the feature representation of normal data patterns plays a crucial role in improving the fitting performance of unsupervised models. As shown in the previous analysis, stacked adversarial variational recurrent neural networks, through their unique hierarchical network architecture, can effectively capture dynamic patterns with bidirectional long-distance dependencies in the data, thus forming multi-level feature representations. To further enhance the model's robustness to input data and fully utilize these hierarchical representations, the anomaly scoring function can be redefined as a joint conditional probability based on multiple layers of latent variables, with the following mathematical expression: ; Regarding threshold selection, this invention employs the POT (Peaks-Over-Threshold) method to achieve adaptive anomaly detection. The specific implementation process is as follows: First, the multivariate time series to be detected is input into the trained unsupervised multilayer dynamic probability model to reconstruct the multivariate time series, obtaining the reconstructed data. Then, a real-time anomaly score is calculated based on the above formula.
[0031] In one implementation, experiments are conducted to compare the unsupervised multilayer dynamic probabilistic model with current mainstream online anomaly detection methods for multivariate time series. The experimental setup includes two benchmark methods: the first is non-dynamic modeling methods, represented by variational autoencoders, serving as the foundational unsupervised probabilistic generation model; the second is dynamic modeling methods, primarily comparing variational recurrent neural networks, which can effectively model the probabilistic dynamic characteristics of high-dimensional time series. By comparing these two benchmark methods, the performance advantages of the unsupervised multilayer dynamic probabilistic model in time series modeling and anomaly detection can be comprehensively evaluated. This invention uses two publicly available multivariate time series datasets for experimental verification: key performance indicator (KPI) data from the Distributed Network Dataset (DND) and the Server Machine Dataset (SMD). Table 1 details the statistical characteristics of the two datasets, including data size (N, the number of multivariate time series), dimension (V, the number of channels in each time series), training / testing set size (T, the time interval), sampling granularity (the time interval between adjacent time intervals), and anomaly ratio (the percentage of anomalous time intervals).
[0032] The DND KPI dataset was collected from a distributed network operated by a well-known Chinese ISP, containing 36 KPI monitoring indicators for 12 different types of websites (such as Video on Demand (VoD), live video, etc.). Data sampling intervals were 60 seconds, lasting approximately 45 days. The experiment used the first 50% of the data as the training set and the latter 50% as the test set, with anomaly labels in the test set verified by experts. The SMD dataset consists of real server monitoring data, containing 38 KPI indicators for 28 servers, with each server monitored for approximately 5 weeks. Both datasets exhibit multivariate time-series characteristics typical of real-world business scenarios, providing reliable benchmarks for algorithm evaluation. Table 1 shows the basic statistical information of the data.
[0033] Table 1
[0034] Regarding experimental parameter settings, this invention employs a rigorous controlled experimental design. All benchmark methods are implemented based on the performance parameters published in their original papers or the official open-source code. The unsupervised multilayer dynamic probabilistic model is implemented using the PyTorch framework, and its network architecture includes the following key components: For datasets such as DND, an encoder-decoder structure with three one-dimensional convolutional layers is used, with the kernel and stride configured as (3,3), (2,2), and (2,2), respectively. The optimization process uses the Adam optimizer, with a base learning rate of 0.0002, a mini-batch size of 256, and a total training iteration count of 50 epochs.
[0035] Specifically, to stabilize model training, a warm-up strategy was employed during the first 10 training epochs, linearly increasing the temperature coefficient in the optimization objective from 0 to 1. Based on extensive experimental validation, the probability parameter of the POT method for initial threshold selection was set to 0.004. All hyperparameter settings were optimized through cross-validation to ensure the reliability and reproducibility of the experimental results. All experiments in this invention were performed on a high-performance computing platform equipped with an Intel(R) Xeon(R) CPU E5-2667v3 @ 3.20GHz processor and dual NVIDIA RTX8000 graphics cards (48GB VRAM each). To comprehensively evaluate model performance, we used precision, recall, and F1 score as core evaluation metrics. Among them, F1 score is considered the most important comprehensive performance metric because it balances precision and recall.
[0036] 2. Quantitative comparison: Parameter sensitivity: This invention systematically evaluates the impact of key model parameters on anomaly detection performance. The experiment mainly examines three core parameters: number of model layers. Latent variable dimensions at each layer and the length of the time window Based on the baseline parameter settings (window length T=20), the analysis first focused on... and The mechanism by which the parameters affect detection performance. After determining the optimal network structure as [15,10,5], we further investigated the impact of the time window length T on model performance. As a key parameter determining the model's temporal perception range, the window length T directly affects the model's ability to capture temporal correlations. Through controlled variable experiments, we were able to accurately quantify the impact of changes in each parameter on key indicators such as the model's F1-score, thus providing a scientific basis for parameter tuning in practical applications. The experimental design ensured the systematic nature and reliability of the parameter sensitivity analysis, and all results were repeatedly verified.
[0037] Figure 6 This is a schematic diagram illustrating the performance change of an unsupervised multilayer dynamic probability model on the SMD dataset provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the F1-score performance of the unsupervised multilayer dynamic probabilistic model on the SMD dataset exhibits the following characteristics as window length and network size change: the performance of the unsupervised multilayer dynamic probabilistic model is significantly affected by network structure parameters. Increasing the number of network layers can effectively enhance feature representation and fitting capabilities, thereby improving anomaly detection accuracy. Regarding the time window length, detection performance shows a trend of first improving and then stabilizing as the window length increases: initially, increasing the window length can provide richer temporal correlation information, but the improvement is limited after exceeding the model's perception range, with deeper models performing better due to their stronger ability to capture long-range dependencies. Regarding the hidden layer width, a larger dimension is not necessarily better; for example, the 15-10-5 structure performs better than the 20-15-8 structure, indicating that an excessively large hidden state dimension can actually reduce model performance due to increased computational complexity under limited training resources. Experimental results show that the model can maintain stable detection performance under different parameter configurations, demonstrating good robustness. All evaluation metrics were calculated based on strict standard procedures to ensure the comparability and reliability of the results.
[0038] Anomaly Detection Performance: In comparing anomaly detection performance, this invention systematically compared the proposed unsupervised multilayer dynamic probabilistic model with all baseline unsupervised models. The average results from five independent runs, as shown in Table 2, indicate that: First, compared to non-dynamic models, the model with the dynamic structure exhibits superior F1 score performance, verifying the crucial role of temporal feature capture in multivariate time series modeling; second, the multilayer structure model generally outperforms the corresponding single-layer structure model, confirming the effectiveness of hierarchical representation enhancement. In particular, the proposed multilayer dynamic variational model, with its hierarchical probability generation and stacked recurrent structure design, achieved the best performance among all compared methods. Experimental results show that SaVRNN with three hidden layers achieves optimal performance in online anomaly detection tasks, fully demonstrating the advantages of multilayer adversarial training mechanisms in enhancing latent feature representation capabilities. Notably, by comparing the performance differences of variant models such as VRNN (an unsupervised multilayer dynamic probabilistic model with and without multilayer structure and adversarial training), SVRNN (an unsupervised multilayer dynamic probabilistic model without adversarial training), and a single-layer unsupervised multilayer dynamic probabilistic model (with a network structure of 20), this study empirically verifies the synergistic advantages of multilayer structure design, variational autoencoder framework, and adversarial training optimization strategy. Furthermore, the experimental results marked "M" (using the improved anomaly score calculation method) further confirm the innovative effectiveness of this invention in anomaly score calculation.
[0039] Table 2
[0040] 3. Qualitative Analysis: This invention provides in-depth verification of the effectiveness of the hierarchical structure and adversarial training mechanism through systematic qualitative analysis. Figure 7a This is a schematic diagram of the abnormal score distribution of a 7-layer unsupervised multilayer dynamic probability model provided in an embodiment of the present invention. Figure 7b This is a schematic diagram of the abnormal score distribution of an unsupervised multilayer dynamic probability model with a single-layer structure provided in an embodiment of the present invention. Figure 7c This is a schematic diagram of the abnormal score distribution of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention. Figure 7d This is a schematic diagram of the abnormal score distribution of another unsupervised multilayer dynamic probability model provided in this embodiment of the invention. Figure 7e This is a schematic diagram of anomaly score distribution in an SVRNN provided by an embodiment of the present invention. Figure 7f This is a schematic diagram of the anomaly score distribution of an unsupervised multilayer dynamic probability model provided in an embodiment of the present invention. The visualization results of the anomaly scores shown in the figure indicate that the unsupervised multilayer dynamic probability model with a 7-layer structure ( Figure 7a Compared to a single-layer model ( Figure 7bThe data exhibits a more stable distribution of outlier scores and more significant peak characteristics in outlier regions, fully demonstrating the superiority of multilayer structures in modeling complex MTS data. Further visualization analysis of the hidden state log-likelihood (…) Figures 7c-7d This reveals an important phenomenon: the hidden states at normal time steps remain stable, while those at abnormal time steps exhibit drastic fluctuations. This characteristic makes the improved anomaly score more stable; the models are respectively... and On the DND dataset, outlier scores differ in that the former only models the generative relationships of Level 1 latent variables under Level 2 latent variable conditions, while the latter only models the hierarchical conditional relationships between mid-to-high-level latent variables. Compared to the SaVRNN variant, based on The model focuses more on capturing long-term dynamic structural changes represented by high-level latent variables, and is more robust to short-term noise, but its response to sudden local anomalies is relatively lagging. Specifically, by comparing the anomaly scoring visualization results of SVRNN and unsupervised multilevel dynamic probabilistic models (…),… Figures 7e-7f It can be clearly observed that the unsupervised multilayer dynamic probability model has a more prominent peak response in anomalous regions (such as the time period marked by the blue box). This finding directly confirms the significant advantage of the adversarial variational training strategy in anomaly pattern detection. These qualitative analysis results corroborate the aforementioned quantitative indicators, verifying the effectiveness of the hierarchical probability framework, stacked recursive structure, and adversarial training method proposed in this invention from multiple perspectives.
[0041] Figure 8 This is a block diagram of a multidimensional time series unsupervised anomaly detection device based on a recurrent neural network provided in an embodiment of the present invention. The device 800 includes: The latent variable acquisition module 801 is used to input the multivariate time series to be detected into the trained unsupervised multilayer dynamic probability model to obtain the latent variables corresponding to the multivariate time series to be detected; wherein, the unsupervised multilayer dynamic probability model is a stacked adversarial variational recurrent neural network; the unsupervised multilayer dynamic probability model is trained by maximizing ELBO after constructing the optimization objective according to the VAE model training strategy and the discriminant network; Anomaly scoring calculation module 802 is used to calculate anomaly scores based on the latent variables corresponding to the multivariate time series to be detected and the multivariate time series to be detected. The first judgment module 803 is used to judge the multivariate time series to be detected as abnormal data when the abnormality score is lower than a preset threshold. The second judgment module 804 is used to determine that the multivariate time series to be detected is normal data when the abnormal score is higher than or equal to a preset threshold.
[0042] In the above technical solution, the present invention captures latent variables in multivariate time series through an unsupervised multilayer dynamic probability model, combines an advanced probability generation framework with a novel anomaly detection strategy, and significantly improves the performance of the MTS anomaly detection task; it also achieves effective training of the unsupervised multilayer dynamic probability model by combining anti-training; and it calculates anomaly scores by latent variables and multivariate time series, which can more comprehensively characterize the anomalies of multivariate time series, and significantly improve the accuracy and robustness of detection.
[0043] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.
[0044] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.
[0045] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. A multidimensional time series unsupervised anomaly detection method based on recurrent neural networks, characterized in that, The method includes: The multivariate time series to be detected is input into the trained unsupervised multilayer dynamic probability model to obtain the latent variables corresponding to the multivariate time series to be detected; wherein, the unsupervised multilayer dynamic probability model is a stacked adversarial variational recurrent neural network; the unsupervised multilayer dynamic probability model is trained by maximizing ELBO after constructing an optimization objective according to the VAE model training strategy and the discriminant network; An anomaly score is calculated based on the latent variables corresponding to the multivariate time series to be detected and the multivariate time series to be detected. When the anomaly score is lower than a preset threshold, the multivariate time series to be detected is determined to be abnormal data; When the anomaly score is higher than or equal to the preset threshold, the multivariate time series to be detected is determined to be normal data.
2. The multidimensional time series unsupervised anomaly detection method based on recurrent neural networks according to claim 1, characterized in that, The training process of the unsupervised multilayer dynamic probability model includes: Input the sample time series into the unsupervised multilayer dynamic probability model to obtain the sample latent variables and reconstruct the sample time series; The sample latent variables, the sample time series, and the sample latent state of the previous time step are input into the stacked recurrent neural network in the unsupervised multilayer dynamic probability model to obtain the sample latent state of the current time step. The variational distribution of the latent variables of the current time step is obtained based on the random noise, the sample time series, the sample latent variables, and the sample latent state of the previous time step. The inference network variational distribution is constructed based on the stacked adversarial variational recurrent neural network, the reconstructed sample time series, and the sample hidden state at the current time step; The initial optimization objective of the unsupervised multilayer dynamic probability model is constructed according to the VAE model training strategy, and the initial optimization objective is adversarially optimized using a discriminant network to obtain the optimization objective. The optimization objective is subjected to ELBO maximization to train the unsupervised multilayer dynamic probability model, resulting in the trained unsupervised multilayer dynamic probability model.
3. The multidimensional time series unsupervised anomaly detection method based on recurrent neural networks according to claim 2, characterized in that, The latent variables of the sample are represented as follows: ； in, express The time series of time samples at time point The time series data at the _th ... The latent variables of the samples in the layer follow a Gaussian distribution. , This represents the total number of layers in an unsupervised multilayer dynamic probabilistic model. for The mean, for variance This indicates taking the diagonal elements.
4. The multidimensional time series unsupervised anomaly detection method based on recurrent neural networks according to claim 3, characterized in that, The reconstructed sample time series is represented as follows: ； in, express The time series of time samples at time point Reconstructed sample time series corresponding to each time series data point, which follows a Gaussian distribution. for The mean, for The variance.
5. The multidimensional time series unsupervised anomaly detection method based on recurrent neural networks according to claim 4, characterized in that, The hidden state of the sample at the current time step is represented as follows: ； in, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer, Indicates the first Long Short-Term Memory Network of Layers Indicates a fully connected network. Indicates will and To splice, express The first time in the time series of the sample time. A time series data, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer, express The time series of time samples at time point The time series data at the _th ... The hidden state of the sample in the layer.
6. According to claim 5 The multidimensional time series unsupervised anomaly detection method based on recurrent neural networks described herein is characterized by, The variational distribution of the inference network is represented as follows: ； in, This represents the variational distribution of the inference network. This represents the probability distribution obtained when random noise enters the inference network.
7. The multidimensional time series unsupervised anomaly detection method based on recurrent neural networks according to claim 5, characterized in that, The optimization objective is expressed as follows: ； in, This represents the optimization objective. This represents the number of time points, indicating the total number of time series data in the sample time series. This indicates that we are calculating the expectation of the logarithm of the likelihood. express The time series of time samples at time point The time series data at the _th ... Random noise in the layer, The parameters represent the decoding network of an unsupervised multilayer dynamic probabilistic model. This represents the encoding network parameters of an unsupervised multilayer dynamic probabilistic model. To adjust the first Hyperparameters of layer adversarial loss For the completed training The optimal discriminator for the layer, Indicate The time series of time samples at time point The time series data at the _th ... Standard Gaussian noise of the layer.
8. A multidimensional time series unsupervised anomaly detection device based on recurrent neural networks, characterized in that, The device includes: The latent variable acquisition module is used to input the multivariate time series to be detected into the trained unsupervised multilayer dynamic probability model to obtain the latent variables corresponding to the multivariate time series to be detected; wherein, the unsupervised multilayer dynamic probability model is a stacked adversarial variational recurrent neural network; the unsupervised multilayer dynamic probability model is trained by maximizing ELBO after constructing an optimization objective according to the VAE model training strategy and the discriminant network; An anomaly scoring module is used to calculate an anomaly score based on the latent variables corresponding to the multivariate time series to be detected and the multivariate time series to be detected. The first judgment module is used to determine that the multivariate time series to be detected is abnormal data when the anomaly score is lower than a preset threshold. The second judgment module is used to determine that the multivariate time series to be detected is normal data when the abnormal score is higher than or equal to the preset threshold.