Multivariate time sequence anomaly detection method and device
Through the fusion of unsupervised learning models and neural networks, combined with adversarial training methods, the problem of abnormal detection in multivariate time series data is solved, stable and flexible detection of massive data is achieved, and the robustness and generalization ability of the model are improved.
Patent Information
- Application Number
- CN202510051278.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-13
- Publication Date
- 2025-04-25
AI Technical Summary
The prior art is difficult to effectively detect abnormalities in massive multivariable time series data, especially when the data samples are small and unbalanced, and traditional supervised learning methods are difficult to cope with this challenge.
Unsupervised learning model is adopted, and the fusion of neural networks, including GRU network, attention network, encoder and decoder network, combined with adversarial training methods, predicted values are generated and abnormal scores are calculated to determine whether the data is abnormal.
It realizes stable and flexible anomaly detection of multivariate time series data, improves the robustness and generalization capabilities of the model, and can adapt to data sets of different sizes and complexities.
Smart Images
Figure CN119884834A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of anomaly detection, and in particular relates to a multivariate time series anomaly detection method and device. Background Art
[0002] Multivariate time series data consists of related variables in multiple dimensions in chronological order, and there are complex correlations between these different dimensions. In the process of transformation to a fully digital society, the task of anomaly detection in multivariate time series faces huge challenges. First, the application of a large number of sensors usually generates massive amounts of multivariate time series data, which are high-dimensional, highly volatile, and dynamic, and show complex temporal and dimensional correlations. As the amount of data grows exponentially, the limitations of manual experience gradually become apparent, making it difficult to accurately capture anomalies in the data. This complexity makes the traditional method of relying on domain experts to pre-set static thresholds unsustainable.
[0003] Inspired by the successful experience in fields such as natural language processing and audio processing, researchers began to explore anomaly detection methods based on advanced deep learning architectures and made significant progress. Since the abnormal patterns in real-world scenarios are complex and changeable, and the abnormal data is far less than the normal data, collecting large-scale abnormal labeled data requires huge human resources. This means that for supervised learning methods that require a large amount of accurately labeled data to support, it becomes particularly difficult to deal with the problem of small and unbalanced data samples. In contrast, semi-supervised and unsupervised learning methods have shown their unique advantages: they do not need to rely on large-scale labeled samples, and can use a small amount of labeled data or rely solely on the inherent laws of the data itself to mine potential information and identify anomalies, opening up an effective new path for anomaly detection in industrial data.
[0004] Therefore, we propose an unsupervised learning model that is stable and adaptable to datasets of different sizes and complexities, aiming to effectively address the above challenges and provide a more flexible and efficient anomaly detection solution to meet the needs of modern industrial systems. Summary of the invention
[0005] Based on the above situation, the main purpose of the present invention is to propose a multivariate time series anomaly detection method, and propose corresponding devices, equipment and computer-readable storage media, aiming to solve the above technical problems.
[0006] In a first aspect, to achieve the above-mentioned object, the present invention provides a multivariate time series anomaly detection method, the method comprising the following steps:
[0007] 1. Through the fusion of neural networks, their advantages are combined to collaboratively realize anomaly detection of multivariate time series data. Among them, the GRU network, attention network, E network, D1 network and D2 network are sub-neural networks used in the anomaly detection system;
[0008] 2. Obtain the hidden state information of the current time series data X through the GRU network;
[0009] 3. Input the state value of the sampling point into the attention network, assign different weights according to the importance of the information, and then extract the time series features;
[0010] 4. Input the state value of the sampling point time series feature into the encoder E network to calculate the low-dimensional mean and variance of the sampling point;
[0011] 5. Store the above results in the latent space, obtain the potential distribution of the data, and sample from it;
[0012] 6. Input the sampling results into the decoder D1 and D2 networks to obtain the corresponding predicted values and , this is the first stage;
[0013] 7. In order to better detect subtle anomalies, adversarial training is used to obtain the predicted values. Repeat the above operation, through the GRU network, attention network, E network and D2 network, to generate new prediction values , this is the second stage;
[0014] 8. Based on the forecast value of two-stage output and The mean square error between the observed value and the anomaly score is calculated , determine whether the target data is abnormal, and those that exceed the set threshold are abnormal.
[0015] 9. The two-stage training is a confrontation training method, and the specific process also includes:
[0016] (1) The neural network adopts a two-stage training method during training, and the corresponding loss function has different training objectives. The first stage aims to train the V1 and V2 networks to have good reconstruction capabilities, and the second stage aims to minimize the mean square error between the predicted value and the observed value of the V1 network, and aims to maximize the mean square error between the predicted value and the observed value of the V2 network;
[0017] (2) The loss function used in training is a composite loss function of two-stage training, which divides the D1 network and the D2 network into two sub-neural networks V1 and V2, sharing the GRU network, the attention network, and the E network. Among them, the training loss function of the V1 network is , the training loss function of the V2 network is ;
[0018] (3) Among them yes The weight of , which controls the generative effect of reconstructing the observations;
[0019] (4) Two KL divergences are introduced into the loss function to ensure a more balanced state between the reconstruction error and the latent space distribution during training.
[0020] in is the number of training rounds. As the number of training rounds increases, the loss weight generated in the second stage gradually increases, and the training process gradually focuses on the adversarial learning of the V1 network and the V2 network.
[0021] 10. The predicted values of the two-stage output are judged to be abnormal, and further include:
[0022] (1) Anomaly score of the neural network described above in anomaly detection ;
[0023] (2) Among them Used to balance the weights of the V1 network and the V2 network to control the sensitivity of anomaly detection and adapt to different needs.
[0024] In a second aspect, to achieve the above-mentioned purpose, the present invention further provides a device for multivariate time series anomaly detection, comprising:
[0025] 1. Feature extraction module, used to obtain the time series data detected in real time, calculate the hidden state vector at the current time point through the GRU network, and predict the hidden state vector at the next time point; the attention network calculates the weight of the hidden state of the sampled data output by the GRU network to extract the time series features;
[0026] 2. The data reconstruction module is used to reconstruct the hidden state information of the sampled data output by the feature extraction module using the encoder network and the decoder network to generate a predicted value.
[0027] 3. Anomaly detection module, which is used to determine whether there is an anomaly by comparing the observed value of the sampling point with the predicted value obtained by the data reconstruction module.
[0028] In a third aspect, to achieve the above-mentioned purpose, the present invention further provides a hardware device for multivariate time series anomaly detection, comprising:
[0029] 1. Memory, used to store code;
[0030] 2. A processor, configured to call the computer-readable code stored in the memory to implement the method described.
[0031] 3. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-readable instructions, which are used to call a processor to execute the method when the computer-readable instructions are executed.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] 1. Through the fusion of neural networks, their advantages are combined to realize the anomaly detection of multivariate time series data. This method can model random time series dependencies, and the learned data is more robust. And it is well reconstructed through the introduction of adversarial training.
[0034] 2. The KL divergence is introduced into the loss function of this method to ensure that the model pays more attention to the balance between reconstruction error and latent space distribution during training, thereby improving the generalization ability of the model while ensuring good reconstruction ability.
[0035] 3. A novel multivariate time series anomaly detection method is proposed. This method has self-adjustment capabilities, and the captured time series features are more robust, helping the model to obtain better stability and improve generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 This is a system framework diagram of the neural network provided by the present invention.
[0037] Figure 2 This is a schematic diagram of the structure of the anomaly detection device provided by the present invention.
[0038] Figure 3 Flow chart of the anomaly detection method provided by the present invention Specific implementation plan
[0039] The present invention is further described in detail below with reference to the accompanying drawings and embodiments. The detailed description of the following embodiments and the accompanying drawings are used to exemplarily illustrate the principles of the present application, but cannot be used to limit the scope of the present application, that is, the present application is not limited to the described embodiments.
[0040] Figure 1 A system framework diagram of a neural network provided by the present invention is shown.
[0041] like Figure 1 As shown in the figure, the neural network here includes 5 sub-networks, namely GRU network, attention network, E network, D1 network and D2 network. Among them, D1 network and D2 network share GRU network, attention network and E network, forming V1 and V2 networks respectively.
[0042] The model training adopts adversarial training method and is divided into two stages.
[0043] Phase 1: Time series data X is input into the GRU network to obtain hidden state information; then the state value is input into the attention network, and different weights are assigned according to the importance of the information, so as to extract the time series features; the time series features are input into the encoder E network to calculate the low-dimensional mean and variance of the sampling points; the above results are stored in the latent space to obtain the potential distribution of the data, and sampling is performed from it; the sampling results are input into the decoder D1 and D2 networks to obtain the corresponding predicted values and ;
[0044] The second stage: The predicted value obtained by the first stage Repeat the above operation, through the GRU network, attention network, E network and D2 network, to generate new prediction values .
[0045] The training process conducts two phases of training simultaneously. is the number of training rounds, Adjust the weights of the two-stage loss, gradually focusing on the adversarial learning of the V1 network and the V2 network. The training loss function of the V1 network is , the training loss function of the V2 network is .
[0046] Finally, the anomaly score of the data during anomaly detection , by comparison Whether the target data exceeds the set threshold is determined to be abnormal.
[0047] Optionally, the neural networks mentioned above are not limited to the specific types described in the article and can be replaced by related extended networks. For example, the GRU network can choose to use the Long-Short Term Memory (LSTM) network, the standard Recurrent Neural Network (RNN) network, etc.; the attention network can choose the self-attention network, the multi-head attention network, etc. It can be optimized and adjusted according to the needs of the specific application scenario to achieve the best performance.
[0048] Figure 2 The schematic diagram of the structure of the abnormality detection device provided by the present invention is shown. The device 200 can execute the abnormality detection method 300 of the present application.
[0049] like Figure 2 As shown, the device includes: a memory 201, a processor 202, a communication interface 203 and a bus 204. The memory 201, the processor 203 and the communication interface 203 are connected to each other through the bus 204.
[0050] The memory 201 can store codes. When the program stored in the memory 201 is executed by the processor 202, the processor 201 and the communication interface 203 are used to perform the various steps of the abnormality detection method of the present invention. Optionally, the memory 201 can be a read-only memory (ROM), a static storage device and a random access memory (RAM).
[0051] The memory 201 includes a program 21 for multivariate time series anomaly detection of the present invention, and the program includes: a feature extraction module 211 , a data reconstruction module 212 and an anomaly detection module 213 .
[0052] The feature extraction module 211 is used to extract features from the time series data detected in real time. The hidden state vector at the current time point is calculated through the GRU network, and the hidden state vector at the next time point is predicted; the attention network calculates the weight of the state value of the sampled data output by the GRU network to extract the time series features.
[0053] The data reconstruction module 212 is used to reconstruct the hidden state information of the sampled data output by the feature extraction module using the encoder network and the decoder network to generate a prediction value in the two-stage training.
[0054] The anomaly detection module 213 is used to determine whether there is an anomaly by comparing the observed value of the sampling point with the predicted value obtained by the data reconstruction module. It is configured to calculate the anomaly score based on the mean square error between the predicted value and the observed value output in the two stages, and determine whether the target data is abnormal. If it exceeds the set threshold, it is abnormal.
[0055] The processor 202 is used to call the computer readable code stored in the memory to implement the functions that need to be performed by the units in the device of the abnormality detection method of the present invention, or to perform the abnormality detection method of the present invention. Optionally, the processor 202 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits.
[0056] The communication interface 203 is not limited to a transceiver or other device to realize the communication between the device 200 and other devices or communication networks. Optionally, the flow data of unknown devices can be obtained through the communication interface 203.
[0057] The bus 204 includes a path for transmitting information between various components in the device 200, such as the memory 201, the processor 202, and the communication interface 203.
[0058] Optionally, the present invention further provides a computer-readable storage medium storing computer-readable instructions, which is used to cause the computer to execute the above-mentioned abnormality detection method when the computer-readable instructions are executed. Optionally, the computer-readable storage medium can be a transient computer-readable storage medium or a non-transient computer-readable storage medium.
[0059] It should be mentioned that embodiments of the present invention may include Figure 2 The above architecture is merely exemplary and is used to explain the method 300 provided by the embodiment of the present invention. In the specific implementation process, those skilled in the art should understand that the device 200 may also include other devices necessary for normal operation and hardware devices for implementing other additional functions.
[0060] Figure 3 A flow chart of the anomaly detection method provided by the present invention is shown.
[0061] like Figure 3 As shown, the method may include the following steps:
[0062] S301: Obtain hidden state information of current time series data X through the GRU network;
[0063] S302: Input the state value of the sampling point into the attention network, assign different weights according to the importance of the information, and then extract the time series features;
[0064] S303: Input the state value of the sampling point time series feature into the encoder E network, and calculate the low-dimensional mean and variance of the sampling point;
[0065] S304: Store the above results in the latent space to obtain the potential distribution of the data, and sample from it, and then input the sampling results into the decoder D1 and D2 networks to obtain the corresponding predicted values and , this is the first stage;
[0066] S305: The predicted value Input the GRU network, attention network, E network and D2 network to generate new prediction values , this is the second stage;
[0067] S306: Predicted value based on two-stage output and The mean square error between the observed value and the anomaly score is calculated ;
[0068] S307: Determine whether the target data is abnormal by comparing whether the abnormality score exceeds a threshold. Optionally, it is set that the data exceeding a set threshold is abnormal.
[0069] Exemplarily, when the model is trained, the above steps S301-S305 are repeated, using a two-stage composite loss function and , and continuously iterate and optimize until the model converges. Determine whether the target data is abnormal.
[0070] Exemplarily, when detecting anomalies in the model, the above steps S301-S307 are executed to calculate the anomaly score.
[0071] It should be mentioned that the execution order of each step in the embodiment of the present invention is not fixed, and can be optionally adjusted according to actual needs. The above system structure is not limited to showing the physical structure or logical structure, and some modules are implemented by a single or multiple physical entities, or optionally by a combination of components in multiple independent physical entities. It should be noted that the specific implementation method can be determined according to engineering practice.
Claims
1. A multivariate time series anomaly detection method, characterized in that: include: By integrating neural networks, their advantages are combined to collaboratively realize anomaly detection of multivariate time series data; Adopt adversarial learning method for training; Obtain the hidden state information of the current time series data X through the GRU network; The state value of the sampling point is input into the attention network, and different weights are assigned according to the importance of the information, so as to extract the temporal features; Input the state value of the sampling point time series feature into the encoder E network to calculate the low-dimensional mean and variance of the sampling point; The above results are stored in the latent space to obtain the potential distribution of the data and sample from it; The sampling results are input into the decoder D1 and D2 networks to obtain the corresponding predicted values. and , this is the first stage; In order to better detect subtle anomalies, adversarial training is used to obtain the predicted values. Repeat the above operation, through the GRU network, attention network, E network and D2 network, to generate new prediction values , this is the second stage; Based on the forecast value of two-stage output and The mean square error between the observed value and the anomaly score is calculated , determine whether the target data is abnormal, and those exceeding the set threshold are abnormal; Among them, the GRU network, attention network, E network, D1 network and D2 network are sub-neural networks used in the anomaly detection system.
2. The method according to claim 1, characterized in that The above-mentioned neural network adopts a two-stage training method during training, and the corresponding loss function has different training objectives. The first stage aims to train the V1 and V2 networks to have good reconstruction capabilities. The second stage aims to minimize the mean square error between the predicted values and the observed values of the V1 network, and to maximize the mean square error between the predicted values and the observed values of the V2 network.
3. The method according to claim 1, characterized in that The loss function used in training is a composite loss function, which divides the D1 network and the D2 network into two sub-neural networks V1 and V2, sharing the GRU network, the attention network, and the E network. Among them, the training loss function of the V1 network is , the training loss function of the V2 network is .
4. The method according to claim 3, wherein yes The weights of , control the generative effect of reconstructing the observations.
5. According to the method of claim 3, two KL divergences are introduced into the loss function to ensure a more balanced state between the reconstruction error and the latent space distribution during the training process.
6. The method according to claim 3, wherein is the number of training rounds. As the number of training rounds increases, the loss weight generated in the second stage gradually increases, and the training process gradually focuses on the adversarial learning of the V1 network and the V2 network.
7. The method according to claim 1, characterized in that Anomaly score of the neural network described above for anomaly detection .
8. The method according to claim 6, Used to balance the weights of the V1 network and the V2 network to control the sensitivity of anomaly detection and adapt to different needs.
9. A device for detecting anomalies in multivariate time series, characterized in that: include: The feature extraction module is used to obtain the time series data detected in real time, calculate the hidden state vector at the current time point through the GRU network, and predict the hidden state vector at the next time point; the attention network calculates the weight of the state value of the sampled data output by the GRU network to extract the time series features; The data reconstruction module is used to reconstruct the hidden state information of the sampled data output by the feature extraction module using the encoder network and the decoder network to generate a predicted value; The anomaly detection module is used to determine whether there is an anomaly by comparing the observed value of the sampling point with the predicted value obtained by the data reconstruction module.
10. A hardware device for multivariate time series anomaly detection, characterized in that: include: A memory for storing codes; A processor, configured to call the computer-readable code stored in the memory to implement the method according to any one of claims 1 to 8; A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions for invoking a processor to execute the method according to any one of claims 1 to 8 when the computer-readable instructions are executed.