Time series data anomaly detection method and device
Through the composite model of self-attention mechanism and variational autoencoder, the robustness and accuracy problems of time series data anomaly detection are solved, accurate detection of highly volatile data is achieved, and the risk of noise interference and overfitting is reduced.
Patent Information
- Application Number
- CN202410298448.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-15
- Publication Date
- 2025-09-16
AI Technical Summary
The existing anomaly detection methods for time series data have problems of poor robustness and low detection accuracy, which are mainly caused by the high volatility of time series data and the separate execution of feature extraction and detection.
A composite model of self-attention mechanism and variational autoencoder is used to process multivariate time series. Sequence information is extracted through the representation network, and machine learning is performed in the memory network. A loss function is constructed to optimize the model, and anomaly detection is performed using maximum mean difference (MMD) and reconstruction error.
It improves the accuracy of anomaly detection in time series data, can effectively reduce noise interference, reduce the risk of overfitting, adapt to highly volatile data, and achieve accurate identification of abnormal data.
Smart Images
Figure CN120654122A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of Internet data analysis, and in particular to a method and device for detecting anomalies in time series data. Background Art
[0002] With the development of digitalization in various applications, a large amount of high-dimensional sensor data and multivariate time series data for monitoring these sensor data have been generated. Anomaly detection in time series data has become a key research direction.
[0003] In the current time series anomaly detection methods, since time series data tends to show high volatility in a short time interval, and the feature extraction and detection of the data need to be performed separately, this leads to poor robustness of the model and low accuracy of anomaly detection for time series data. Summary of the Invention
[0004] The purpose of the present invention is to solve the above-mentioned problems existing in the prior art and provide a method and device for detecting anomalies in time series data.
[0005] The time series data anomaly detection method provided by the present invention is achieved through the following technical solutions: after processing the multivariate time series, the multivariate time series is input into the representation network, and sequence information is obtained in the representation network based on the attention mechanism; the sequence information is input into the memory network, and machine learning is performed based on the variational autoencoder in the memory network to obtain reconstructed data; a loss function is constructed based on the sequence information and the reconstructed data, and the loss function includes the maximum mean difference (MMD) between the sequence information and the reconstructed data; the representation network and the memory network are optimized and trained according to the loss function to obtain a time series data anomaly detection model; and anomaly detection is performed on the time series data based on the time series data anomaly detection model.
[0006] Furthermore, the processing of the multivariate time series and then inputting the processed time series into the representation network includes: adding position coding to the multivariate time series and then inputting the processed time series into the representation network, wherein the position coding method includes: absolute position coding, relative position coding, or fixed position coding.
[0007] Furthermore, obtaining sequence information based on the attention mechanism in the representation network includes: obtaining sequence information based on the attention mechanism, residual addition and feedforward network in the representation network, and the attention mechanism is a multi-head attention mechanism or a single-head attention mechanism.
[0008] Furthermore, the variational autoencoder in the memory network is used to perform machine learning to obtain reconstructed data, including: based on the variational autoencoder in the memory network, mapping the sequence information to the corresponding mean vector and the corresponding standard deviation vector through a fully connected layer, and performing standard Gaussian distribution sampling based on the standard Gaussian distribution of the mean vector and the standard deviation vector to obtain the reconstructed data.
[0009] Furthermore, the loss function also includes: the reconstruction error between the sequence information and the reconstructed data, the potential distribution learned by the variational autoencoder, or the relative entropy KL divergence of the predefined spatial distribution; the anomaly detection of the time series data based on the time series data anomaly detection model includes: setting a judgment threshold based on the reconstruction error between the sequence information and the reconstructed data; when the detection exceeds the judgment threshold, judging that the time series data is abnormal data; or, when the detection does not exceed the judgment threshold, judging that the time series data is normal data.
[0010] The time series data anomaly detection device provided by the present invention is realized through the following technical solutions: a representation module, which is used to process the multivariate time series and input it into the representation network, and obtain sequence information in the representation network based on the attention mechanism; a reconstruction module, which is used to input the sequence information into the memory network, and perform machine learning based on the variational autoencoder in the memory network to obtain reconstructed data; a loss construction module, which is used to construct a loss function based on the sequence information and the reconstructed data, and the loss function includes the maximum mean difference (MMD) between the sequence information and the reconstructed data; a training module, which is used to optimize the training of the representation network and the memory network according to the loss function to obtain a time series data anomaly detection model; a detection module, which is used to perform anomaly detection on time series data based on the time series data anomaly detection model.
[0011] Furthermore, the representation module is further configured to add position coding to the multivariate time series and then input the result into the representation network. The position coding method includes absolute position coding, relative position coding, or fixed position coding.
[0012] Furthermore, the representation module is also used to obtain sequence information based on the attention mechanism, residual addition and feedforward network in the representation network, and the attention mechanism is a multi-head attention mechanism or a single-head attention mechanism.
[0013] Furthermore, the reconstruction module is also used to map the sequence information to the corresponding mean vector and the corresponding standard deviation vector through a fully connected layer based on the variational autoencoder in the memory network, and perform standard Gaussian distribution sampling based on the standard Gaussian distribution of the mean vector and the standard deviation vector to obtain the reconstructed data.
[0014] Furthermore, the loss function constructed by the loss construction module also includes: the reconstruction error between the sequence information and the reconstructed data, the potential distribution learned by the variational autoencoder, or the relative entropy KL divergence of the predefined spatial distribution; the detection module includes a threshold submodule and a judgment submodule; the threshold submodule is used to set a judgment threshold based on the reconstruction error between the sequence information and the reconstructed data; the judgment submodule is used to judge that the time series data is abnormal data when the detection exceeds the judgment threshold; or, when the detection does not exceed the judgment threshold, judge that the time series data is normal data.
[0015] Compared with the prior art, the present invention has the following beneficial effects:
[0016] 1. The present invention uses a composite model based on the self-attention mechanism and the variational autoencoder to simultaneously perform reconstruction and predictive analysis to characterize complex spatiotemporal patterns, thereby improving the accuracy of anomaly detection in time series data.
[0017] 2. The present invention uses the attention mechanism to extract features from time series data, so that the model can process data at different positions of the input sequence separately, and can process sequences of different lengths, which can reduce the interference of noise on time series data, thereby improving the accuracy of time series data anomaly detection.
[0018] 3. This invention uses a variational autoencoder for data reconstruction, enabling the model to capture the complex distribution of time series data. During the model training phase, the variational autoencoder optimizes model parameters by maximizing the likelihood of the observed data and achieves regularization by minimizing the KL divergence of the relative entropy of the latent space, ensuring a sufficiently compact latent representation. During the anomaly detection phase, anomalous patterns can be accurately identified by comparing the reconstruction error of the input data with the degree of anomaly in the latent representation. This further improves the accuracy of anomaly detection in time series data.
[0019] 4. By minimizing the maximum mean difference (MMD) between sequence information and reconstructed data, this method can better distinguish between noisy data, normal data, and abnormal data, reducing the impact of noise on detection results and lowering the risk of overfitting. Even for highly volatile time series data, it can accurately detect abnormal data, thereby improving the model's anomaly detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a flow chart of the time series data anomaly detection method in this specific implementation;
[0021] Figure 2 is another flowchart of the time series data anomaly detection method in this specific embodiment;
[0022] Figure 3 is a schematic diagram of a time series data anomaly detection model in this specific embodiment;
[0023] Figure 4 Schematic diagram of the structure of the time series data anomaly detection device in this specific embodiment;
[0024] Figure 5 It is a structural diagram of the detection module in this specific implementation. DETAILED DESCRIPTION
[0025] The present invention is further described in detail below with reference to the accompanying drawings:
[0026] refer to Figure 1 As shown, the time series data anomaly detection method in this specific embodiment includes:
[0027] 101. After processing the multivariate time series, it is input into the representation network, and the sequence information is obtained in the representation network based on the attention mechanism.
[0028] In this embodiment of the present invention, an attention mechanism is used to extract features from time series data. This allows the model to process data at different positions in the input sequence separately and handle sequences of different lengths, reducing the interference of noise on the time series data and thus improving the accuracy of anomaly detection in time series data. Furthermore, this embodiment of the present invention uses a variational autoencoder for data reconstruction, enabling the model to capture the complex distribution of time series data.
[0029] 102. Input the sequence information into the memory network, perform machine learning based on the variational autoencoder in the memory network, and obtain reconstructed data.
[0030] In this embodiment of the present invention, during the model training phase, the variational autoencoder optimizes model parameters by maximizing the likelihood of the observed data and achieves regularization by minimizing the KL divergence of the relative entropy of the latent space, ensuring that the latent representation is sufficiently compact. During the anomaly detection phase, abnormal patterns can be accurately identified by comparing the reconstruction error of the input data with the degree of anomaly in the latent representation. This further improves the accuracy of anomaly detection in time series data.
[0031] 103. Construct a loss function based on the sequence information and the reconstructed data. The loss function includes the maximum mean difference (MMD) between the sequence information and the reconstructed data.
[0032] By minimizing the maximum mean difference (MMD) between sequence information and reconstructed data, this embodiment of the present invention can better distinguish between noisy data, normal data, and abnormal data, reducing the impact of noise on detection results and lowering the risk of overfitting. Even for highly volatile time series data, it can accurately detect abnormal data, thereby improving the model's anomaly detection accuracy.
[0033] 104. Optimize and train the representation network and memory network according to the loss function to obtain a time series data anomaly detection model.
[0034] 105. Perform anomaly detection on time series data based on the time series data anomaly detection model.
[0035] Compared to existing technologies, this embodiment of the present invention uses a composite model based on a self-attention mechanism and a variational autoencoder to simultaneously perform reconstruction and predictive analysis to characterize complex spatiotemporal patterns. In the reconstruction analysis, the self-attention mechanism is used to extract sequence information from time series data; in the predictive analysis, a variational autoencoder-based framework is constructed to capture complex temporal dependencies. This avoids the problem of separate feature extraction and detection in existing technologies, thereby improving the accuracy of anomaly detection in time series data.
[0036] refer to Figure 2 As shown, another time series data anomaly detection method in this specific embodiment includes:
[0037] 201. Add position encoding to the multivariate time series and input it into the representation network.
[0038] The position coding method includes absolute position coding, relative position coding, or fixed position coding. The embodiment of the present invention does not specifically limit the position coding method. Any position coding method that can realize multivariate time series is within the protection scope of the embodiment of the present invention.
[0039] 202. In the representation network, sequence information is obtained based on the attention mechanism, residual addition and feedforward network.
[0040] Among them, the attention mechanism can be a multi-head attention mechanism or a single-head attention mechanism.
[0041] 203. Input the sequence information into the memory network. Based on the variational autoencoder in the memory network, map the sequence information to the corresponding mean vector and the corresponding standard deviation vector through a fully connected layer. Based on the standard Gaussian distribution of the mean vector and the standard deviation vector, perform standard Gaussian distribution sampling to obtain reconstructed data.
[0042] 204. Construct a loss function based on the sequence information and the reconstructed data, where the loss function includes a maximum mean difference (MMD) between the sequence information and the reconstructed data.
[0043] By minimizing the maximum mean difference (MMD) between sequence information and reconstructed data, this embodiment of the present invention can better distinguish between noisy data, normal data, and abnormal data, reducing the impact of noise on detection results and lowering the risk of overfitting. Even for highly volatile time series data, it can accurately detect abnormal data, thereby improving the model's anomaly detection accuracy.
[0044] In an embodiment of the present invention, the loss function further includes: a reconstruction error between the sequence information and the reconstructed data, a potential distribution learned by the variational autoencoder, or a relative entropy KL divergence of a predefined spatial distribution.
[0045] 205. Optimize and train the representation network and memory network according to the loss function to obtain a time series data anomaly detection model.
[0046] 206. Set a judgment threshold based on the sequence information and the reconstruction error between the reconstructed data.
[0047] 207. When the detection exceeds the judgment threshold, the time series data is judged to be abnormal data; or, when the detection does not exceed the judgment threshold, the time series data is judged to be normal data.
[0048] refer to Figure 3 As shown in the figure, the time series data anomaly detection model consists of a representation network and a memory network. Position encoding can be added for time series data, and the attention mechanism in the representation network can be a multi-head attention mechanism. A feedforward network and residual addition can also be added to the representation network. The sequence information output by the representation network is input into the memory network. In the network, the sequence information can be mapped to the mean and standard deviation through the fully connected layer, and the reconstructed data can be obtained through the standard Gaussian distribution. The model is optimized based on the MMD between the sequence information and the reconstructed data.
[0049] Compared to existing technologies, this embodiment of the present invention uses a composite model based on a self-attention mechanism and a variational autoencoder to simultaneously perform reconstruction and predictive analysis to characterize complex spatiotemporal patterns. In the reconstruction analysis, the self-attention mechanism is used to extract sequence information from time series data; in the predictive analysis, a variational autoencoder-based framework is constructed to capture complex temporal dependencies. This avoids the problem of separate feature extraction and detection in existing technologies, thereby improving the accuracy of anomaly detection in time series data.
[0050] refer to Figure 4 As shown, the time series data anomaly detection device of this specific embodiment includes:
[0051] The representation module 41 is used to process the multivariate time series and input it into the representation network, and obtain sequence information in the representation network based on the attention mechanism.
[0052] The reconstruction module 42 is used to input the sequence information into the memory network, perform machine learning based on the variational autoencoder in the memory network, and obtain reconstructed data.
[0053] The loss construction module 43 is configured to construct a loss function based on the sequence information and the reconstructed data, wherein the loss function includes a maximum mean difference (MMD) between the sequence information and the reconstructed data.
[0054] The training module 44 is used to optimize the training of the representation network and the memory network according to the loss function to obtain a time series data anomaly detection model.
[0055] The detection module 45 is used to perform anomaly detection on the time series data based on the time series data anomaly detection model.
[0056] The representation module 41 is further configured to add position coding to the multivariate time series and then input the resultant into the representation network. The position coding method includes absolute position coding, relative position coding, or fixed position coding.
[0057] The representation module 41 is further used to obtain sequence information based on the attention mechanism, residual addition and feedforward network in the representation network, and the attention mechanism is a multi-head attention mechanism or a single-head attention mechanism.
[0058] The reconstruction module 42 is also used to map the sequence information to the corresponding mean vector and the corresponding standard deviation vector through a fully connected layer based on the variational autoencoder in the memory network, and perform standard Gaussian distribution sampling based on the standard Gaussian distribution of the mean vector and the standard deviation vector to obtain the reconstructed data.
[0059] The loss function constructed by the loss construction module 43 also includes: a reconstruction error between the sequence information and the reconstructed data, a potential distribution learned by the variational autoencoder, or a relative entropy KL divergence of a predefined spatial distribution.
[0060] refer to Figure 5 As shown, further, the detection module 45 includes a threshold submodule 4501 and a judgment submodule 4502.
[0061] The threshold submodule 4501 is used to set a judgment threshold based on the reconstruction error between the sequence information and the reconstructed data.
[0062] The judgment submodule 4502 is configured to judge that the time series data is abnormal data when the detection exceeds the judgment threshold; or judge that the time series data is normal data when the detection does not exceed the judgment threshold.
[0063] Compared to existing technologies, this embodiment of the present invention uses a composite model based on a self-attention mechanism and a variational autoencoder to simultaneously perform reconstruction and predictive analysis to characterize complex spatiotemporal patterns. In the reconstruction analysis, the self-attention mechanism is used to extract sequence information from time series data; in the predictive analysis, a variational autoencoder-based framework is constructed to capture complex temporal dependencies. This avoids the problem of separate feature extraction and detection in existing technologies, thereby improving the accuracy of anomaly detection in time series data.
[0064] The time series data anomaly detection device provided in this embodiment can implement the method implementation provided above. For specific function implementation, please refer to the description in the method embodiment, which will not be repeated here.
[0065] This embodiment also provides another computer-readable storage medium, which can be the computer-readable storage medium contained in the memory in the above embodiment; or it can be a computer-readable storage medium that exists independently and is not installed in the terminal. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute Figure 1 、 Figure 2 The specific embodiment shown provides a method for detecting anomalies in time series data.
[0066] The above technical solution is only one embodiment of the present invention. For those skilled in the art, it is easy to make various types of improvements or modifications based on the principles disclosed in the present invention, and it is not limited to the technical solution described in the above specific embodiments of the present invention. Therefore, the above description is only preferred and does not have a restrictive meaning.
Claims
1. A time series data anomaly detection method, characterized in that: include: After processing the multivariate time series, the multivariate time series is input into a representation network, and sequence information is obtained in the representation network based on an attention mechanism; Inputting the sequence information into a memory network, and performing machine learning based on a variational autoencoder in the memory network to obtain reconstructed data; constructing a loss function based on the sequence information and the reconstructed data, wherein the loss function includes a maximum mean difference (MMD) between the sequence information and the reconstructed data; Optimizing and training the representation network and the memory network according to the loss function to obtain a time series data anomaly detection model; Anomaly detection is performed on the time series data based on the time series data anomaly detection model.
2. The time series data anomaly detection method according to claim 1, characterized in that: The multivariate time series is processed and input into the representation network, including: Position coding is added to the multivariate time series and then input into the representation network. The position coding method includes: absolute position coding, relative position coding, or fixed position coding.
3. The time series data anomaly detection method according to claim 1, characterized in that: The obtaining of sequence information based on the attention mechanism in the representation network includes: In the representation network, sequence information is obtained based on the attention mechanism, residual addition and feedforward network, and the attention mechanism is a multi-head attention mechanism or a single-head attention mechanism.
4. The time series data anomaly detection method according to claim 1, characterized in that: The performing of machine learning based on the variational autoencoder in the memory network to obtain reconstructed data includes: Based on the variational autoencoder in the memory network, the sequence information is mapped to the corresponding mean vector and the corresponding standard deviation vector through a fully connected layer, and standard Gaussian distribution sampling is performed based on the standard Gaussian distribution of the mean vector and the standard deviation vector to obtain the reconstructed data.
5. The time series data anomaly detection method according to claim 1, characterized in that: The loss function also includes: a reconstruction error between the sequence information and the reconstructed data, a potential distribution learned by the variational autoencoder, or a relative entropy KL divergence of a predefined spatial distribution; The performing anomaly detection on the time series data based on the time series data anomaly detection model includes: Setting a judgment threshold based on a reconstruction error between the sequence information and the reconstructed data; When the detection exceeds the judgment threshold, the time series data is judged to be abnormal data; or, when the detection does not exceed the judgment threshold, the time series data is judged to be normal data.
6. A time series data anomaly detection device, characterized in that: include: A representation module is used to process the multivariate time series and input it into the representation network, and obtain sequence information in the representation network based on the attention mechanism; A reconstruction module, configured to input the sequence information into a memory network and perform machine learning based on a variational autoencoder in the memory network to obtain reconstructed data; a loss construction module, configured to construct a loss function based on the sequence information and the reconstructed data, wherein the loss function includes a maximum mean difference (MMD) between the sequence information and the reconstructed data; A training module, configured to optimize and train the representation network and the memory network according to the loss function to obtain a time series data anomaly detection model; The detection module is used to perform anomaly detection on the time series data based on the time series data anomaly detection model.
7. The time series data anomaly detection device according to claim 6, wherein: The representation module is further configured to add position coding to the multivariate time series and then input the resultant into the representation network. The position coding method includes absolute position coding, relative position coding, or fixed position coding.
8. The time series data anomaly detection device according to claim 6, wherein: The representation module is also used to obtain sequence information based on the attention mechanism, residual addition and feedforward network in the representation network, and the attention mechanism is a multi-head attention mechanism or a single-head attention mechanism.
9. The time series data anomaly detection device according to claim 6, wherein: The reconstruction module is also used to map the sequence information to the corresponding mean vector and the corresponding standard deviation vector through a fully connected layer based on the variational autoencoder in the memory network, and perform standard Gaussian distribution sampling based on the standard Gaussian distribution of the mean vector and the standard deviation vector to obtain the reconstructed data.
10. The time series data anomaly detection device according to claim 6, wherein: The loss function constructed by the loss construction module further includes: a reconstruction error between the sequence information and the reconstructed data, a potential distribution learned by the variational autoencoder, or a relative entropy KL divergence of a predefined spatial distribution; The detection module includes a threshold submodule and a judgment submodule; The threshold submodule is configured to set a judgment threshold based on a reconstruction error between the sequence information and the reconstructed data; The judgment submodule is configured to judge that the time series data is abnormal data when the detection exceeds the judgment threshold; or judge that the time series data is normal data when the detection does not exceed the judgment threshold.