Time series anomaly detection method and device based on variational autoencoder time series decomposition

Abstract

The present invention discloses a time series anomaly detection method and device based on variational autoencoder time series decomposition. The method is based on a time series decomposition method, a recurrent neural network, a long short-term memory network, and a variational autoencoder. The recurrent variational autoencoder model is used to reconstruct the decomposed time series, thereby realizing multivariate time series anomaly detection. The present invention has a good time series anomaly detection effect, effectively reduces the data fluctuation caused by non-abnormal factors in the time series, better obtains the time series dependency information in the time series, and thus improves the overall anomaly detection performance.

Description

Technical Field

[0001] The present invention relates to time series anomaly detection, and in particular to a time series anomaly detection method and device based on variational autoencoder time series decomposition. Background Art

[0002] Anomaly detection is the identification of data that is significantly different from most data or does not conform to the normal pattern, while multivariate time series anomaly detection is anomaly detection for multivariate time series data. Time series anomaly detection has many application scenarios in practice. For example, sensors output device status information in real time. These data can constitute time series data. Anomaly detection of time series data can further determine whether the sensor device is abnormal in real time. Time series dependency is an important characteristic of time series data. In anomaly detection, whether the target data is abnormal at a certain moment is related to its time series information, context information and the numerical information of the data itself. The characteristics of time series and the characteristics of the anomaly detection problem itself make the time series anomaly detection problem have certain particularity and difficulty, and these properties guide researchers to propose appropriate model algorithms for time series anomaly detection. Time series contains time series dependency information, so anomaly detection algorithms for discrete points usually cannot effectively identify anomalies in time series data. Neural networks used to process sequence data, such as LSTM and GRU, have been applied by researchers to time series anomaly detection. The serious imbalance between normal data and abnormal data is an important feature of the anomaly detection problem, which distinguishes the anomaly detection problem from conventional classification problems. Although the existing generative models based on neural networks have strong expressive power and can learn the normal patterns of time series through unsupervised learning, they cannot detect abnormal data well when there is insufficient training data. In addition, time series data will produce some unpredictable fluctuations under the influence of non-abnormal factors. For example, some time series will be affected by seasonal and cyclical factors. Therefore, the existing anomaly detection methods based on time series prediction are difficult to achieve satisfactory results. Summary of the invention

[0003] Purpose of the invention: In order to improve the performance of multivariate time series anomaly detection algorithms, the present invention proposes a time series anomaly detection method based on variational autoencoder time series decomposition, which reduces the impact of time series data fluctuations caused by non-abnormal factors (such as seasonal changes in time series data) on anomaly detection, better obtains the time series dependency information of the time series, and improves the anomaly detection performance.

[0004] Technical solution: In order to perform anomaly detection on multivariate time series and improve the performance of anomaly detection, according to a first aspect of the present invention, a time series anomaly detection method based on variational autoencoder time series decomposition is proposed, comprising the following steps:

[0005] S1. Decompose the time series and retain the part of the time series related to anomaly detection;

[0006] S2, dividing the decomposed time series into a set of sub-time series of fixed length;

[0007] S3. Use the training set data to train the anomaly detection model, wherein the anomaly detection model is a cyclic variational autoencoder model, the first neural network processing layers of the encoder part and the decoder part of the model are both LSTM layers, the processing layer for generating latent variable distribution parameters of the model is an RNN layer, and the data at each moment in the training set are all sub-time series of normal data;

[0008] S4. Reconstruct the sub-time series in the training set using the anomaly detection model, calculate the reconstruction error of the last moment data in the sub-time series, and use it as the anomaly score, save the anomaly score corresponding to the training set, and calculate the anomaly threshold using the density-based anomaly threshold selection algorithm;

[0009] S5. Decompose the time series to be detected and divide it into sub-time series, input the anomaly detection model for reconstruction, calculate the anomaly score, and if the anomaly score is greater than the anomaly threshold, it is determined that the last moment data in the currently input sub-time series is abnormal.

[0010] Furthermore, the objective function of the cyclic variational autoencoder model in step S3 is as follows:

[0011]

[0012] Where w is the length of the sub-time series, t represents time t, and mse represents mean square error; represents the t-w+ith data of the time series generated by the variational self-encoder, represents the t-w+ith data of the original time series, β represents the hyperparameter for adjusting the weight, KL represents the calculation of the KL divergence of two Gaussian distributions, and q φ represents the encoder part of the model, z t-w+i represents the t-w+ith data of the latent variable z, Represents known time series data The premise is that the hidden variable z t-w+i The conditional probability distribution of t-w+i ) represents the latent variable z t-w+i The probability distribution of .

[0013] Furthermore, the calculation process of the anomaly score in the anomaly detection in steps S4 and S5 is as follows:

[0014]

[0015] The D-R-VAE represents a recurrent variational autoencoder model. Through this model, the time series is reconstructed, and the time series value at time t of the reconstructed time series represents the calculation of and of the mean squared error, and as t represents the anomaly score at time t.

[0016] Furthermore, the anomaly threshold calculation process is as follows: Store the anomaly scores of the training set data in the array scores, set the number of neighbors k and the multiple m, sort the array scores in reverse order, and then establish the distances array. The calculation process of the elements in the distances array is as follows:

[0017] distances[i] = k * scores[i] - scores[i + 1] - … - scores[i + k]

[0018] Calculate the average value aver_distance of the elements in the distances array, traverse the distances array, and if the condition: distances[i] < aver_distance * m is satisfied, then use scores[i] as the anomaly threshold.

[0019] According to the second aspect of the present invention, a computer device is proposed. The device includes one or more processors; a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to the first aspect of the present invention.

[0020] According to the third aspect of the present invention, a computer-readable storage medium is proposed, storing a computer program. When the program is executed by one or more processors, the one or more processors implement the method according to the first aspect of the present invention.

[0021] Beneficial effects: The present invention proposes a multivariate time series anomaly detection method and device based on recurrent variational autoencoder time series decomposition. Based on the time series decomposition method, recurrent neural network, long short-term memory network, and variational autoencoder, the recurrent variational autoencoder model is used to reconstruct the decomposed time series, realizing multivariate time series anomaly detection. The present invention has good time series anomaly detection effects, effectively weakens the data fluctuations caused by non-anomaly factors in the time series, and can better obtain the time series dependence information through time series decomposition, thereby improving the anomaly detection performance. Description of the Drawings

[0022] Figure 1is a general framework diagram of multivariate time series anomaly detection according to an embodiment of the present invention;

[0023] Figure 2 is an overall flow chart of a multivariate time series anomaly detection method according to an embodiment of the present invention;

[0024] Figure 3 is a structural diagram of a multivariate time series anomaly detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is further described below in conjunction with the accompanying drawings. It should be understood that the embodiments provided below are only intended to disclose the present invention in detail and completely, and to fully convey the technical concept of the present invention to those skilled in the art. The present invention can also be implemented in many different forms and is not limited to the embodiments described herein. The terms in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention.

[0026] In the embodiment of the present invention, a multivariate time series anomaly detection method based on variational autoencoder time series decomposition is used to detect anomalies in time series. Based on the time series decomposition method, recurrent neural network, long short-term memory network, variational autoencoder, the recurrent variational autoencoder model is used to reconstruct the decomposed time series, thereby realizing a multivariate time series anomaly detection algorithm. Figure 1 , Figure 2 , Figure 3 , the method of the present invention comprises the following steps:

[0027] Step S1, time series decomposition. First, it is necessary to determine whether the multivariate time series is periodic. This can be determined based on the properties of the time series itself. For example, continuous traffic flow data is periodic, and the length of the cycle should be 1 day. When the source of the data is unclear, it is necessary to divide the multivariate time series into a group of univariate time series, and use the visualization function to visualize each univariate time series to observe whether it changes periodically.

[0028] For multivariate time series, it is necessary to use the time series decomposition method to decompose each dimension of the univariate time series. For periodic time series, the STL (seasonal trend decomposition) method is used for time series decomposition. The decomposition formula is as follows:

[0029]

[0030] in is the data of the i-th dimension in the numerical vector of the time series at time t. The time series can be decomposed into three components using the STL method: For the trend part, For the seasonal part, The present invention only retains the remaining part and assign it to Construct a new time series for subsequent anomaly detection tasks.

[0031] For time series without periodicity, HP (Hodrick-Prescott) filtering method is used for decomposition. The decomposition formula is as follows:

[0032]

[0033] The HP filter method can be used to decompose the time series into two components: the trend part and the fluctuation part The present invention only retains the trend part and assign it to Used for subsequent anomaly detection tasks.

[0034] Step S2: time series window division. Since time series data has time series dependence, the time series value x at time t of the time series t When performing anomaly detection, judging whether it is abnormal based only on the numerical information at the current moment usually cannot meet the requirements. Therefore, the sequence is divided into subsequences, and the subsequence {x t-w+1 ,x t-w+2 ,…,x t} as the model input, w is the length of the subsequence, and x is determined based on the subsequence t Is it abnormal?

[0035] Time series window division is performed after time series decomposition, so after the decomposition, the time series The division process is as follows:

[0036]

[0037] The length of the subsequence is w. A time series with a length of n can be divided into n-w+1 windows, and one window is a subsequence.

[0038] Step S3, model training. The model used in the present invention is a cyclic variational autoencoder model, and the model structure is as follows: Figure 3 As shown in Figure 2, the overall structure of the model is consistent with that of VAE, where is the encoder part of the model, is the decoder part of the model, such as Figure 3As shown in the figure, the first neural network processing layer of the encoder part and the decoder part of the model are both LSTM (Long Short Term Memory Network) layers, and the processing layer for generating the distribution parameters of the latent variables is RNN (Recurrent Neural Network) layer. Z is the latent variable generated by the encoder and is used for decoding by the decoder. Due to the introduction of LSTM and RNN, the model can obtain the temporal dependency information in the sequence when processing time series data, and process the time series data on this basis. In the model training stage, only the normal data in the training set is used to train the model, that is, the data at each moment in each fixed-length window is normal data.

[0039] The objective function of the model is as follows:

[0040]

[0041] During the model training process, the objective function is maximized. θ is the decoder part of the model, q φ is the encoder part of the model, p(z t ) is the prior distribution of the latent variable and is set to a standard Gaussian distribution, The output of the RNN layer after the LSTM layer in the encoder part of the model is the distribution The parameter μ t and σ t The objective function consists of two terms. The right-hand term calculates the KL divergence of two Gaussian distributions (the KL divergence measures the asymmetric difference between two distributions. The larger the KL divergence, the greater the difference between the distributions). The left-hand term calculates the VAE decoder part, which consists of the latent variable z. t Refactoring The probability of . The term on the left is expressed using the mean square error: The closer the reconstructed vector is to the original vector, the greater the probability that the input vector is reconstructed by the latent variable. In the actual model training, the hyperparameter β is introduced to adjust the weight of the right side of the objective function, and the final objective function is:

[0042]

[0043] Step S4, abnormal threshold selection. First, define an abnormal score for the detected object and judge the abnormality based on the abnormal score. As the detected object, the anomaly score calculation process is as follows:

[0044]

[0045] DR-VAE is a cyclic variational autoencoder model used in this anomaly detection method. The model reconstructs the time series and obtains the time series value at time t. ast Denote the anomaly score. In the present invention, by calculating and the mean square error is used as the anomaly score at time t.

[0046] After the model training is completed, the model is used to reconstruct the time series in the training set to obtain a set of anomaly scores. The anomaly scores are placed in a one-dimensional space, and thus there is a set of points distributed in the one-dimensional space. Anomaly data is data that is significantly different from normal data. The model is trained using normal data. Therefore, when the model reconstructs an anomalous time series, the reconstruction error will be large. Since the training data are all normal data, the scores in the set of anomaly scores corresponding to the training set are concentrated in a certain normal range, and the density of the anomaly scores within the normal range is the largest. However, there will also be a small number of data in the training set whose anomaly scores deviate from the normal range, and the point density of the positions of these anomaly scores in the space is also small. We select a threshold from these anomaly scores. The small point density of the positions of the anomaly score points results in a relatively large distance between the points and their neighboring points. Therefore, the sum of the differences in the distances between the anomaly score points and their neighboring points in the one-dimensional space is used as a measure of the anomaly point density.

[0047] The process of calculating the anomaly threshold is as follows: Store the anomaly scores of the training set data in the array scores, and set the number of neighbors k and the multiple m. Sort the array scores in reverse order, and then establish the distances array. The calculation process of the elements in the distances array is as follows:

[0048] distances[i] = k * scores[i] - scores[i + 1] - … - scores[i + k]

[0049] Calculate the average value aver_distance of the elements in the distances array. Traverse the distances array. If the condition: distances[i] < aver_distance * m is satisfied, then take scores[i] as the anomaly threshold.

[0050] Step S5, anomaly detection. Take the data at time t as the object to be detected. First, use the model to reconstruct the input subsequence calculate the reconstructed vector and the original vector of the mean square error to obtain the anomaly score. Compare the anomaly score with the anomaly threshold. If the anomaly score is greater than the anomaly threshold, then the data at time t is judged as an anomaly.

[0051] It will be appreciated by those skilled in the art that all or part of the steps to implement the above-described embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware to accomplish the steps, and the program may be stored in a computer-readable storage medium. In the context of the present invention, the computer-readable medium may be considered to be tangible and non-temporary. Non-limiting examples of non-temporary tangible computer-readable media include non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital tapes or hard disk drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs), etc.

[0052] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0053] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present invention. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0054] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A time series anomaly detection method based on variational autoencoder time series decomposition, characterized in that: Including the following steps: S1. Decompose the time series and retain the part related to anomaly detection in the time series, including: according to whether the time series has periodicity, use the STL method or the HP filtering method to decompose the time series. For a time series with periodicity, use the STL method to decompose the time series; for a time series without periodicity, use the HP filtering method to decompose the time series. S2. Divide the decomposed time series into a group of fixed-length sub-time series, and the division method is as follows: is the decomposed time series, w is the length of the sub-time series, and the time series with length n is divided into n-w+1 sub-time series; S3. Use the training set data to train the anomaly detection model. The anomaly detection model is a cyclic variational autoencoder model. The first neural network processing layer of the encoder part and the decoder part of the model is an LSTM layer, and the processing layer for generating the latent variable distribution parameters of the model is an RNN layer. The training set is a sub-time series in which the data at each moment is normal data. S4. Use the anomaly detection model to reconstruct the sub-time series in the training set, calculate the reconstruction error of the data at the last moment in the sub-time series, and use it as the anomaly score. Save the anomaly scores corresponding to the training set, and use the density-based anomaly threshold selection algorithm to calculate the anomaly threshold. S5. Decompose and divide the time series to be detected into sub-time series, input them into the anomaly detection model for reconstruction, calculate the anomaly score. If the anomaly score is greater than the anomaly threshold, it is determined that the data at the last moment in the currently input sub-time series is abnormal. Among them, the calculation process of the anomaly score in steps S4 and S5 is as follows: DR-VAE stands for the cyclic variational autoencoder model, which is used to train the time series Reconstruct the time series value at time t Representation calculation and The mean square error, as t represents the anomaly score at time t; The calculation process of the anomaly threshold in step S4 is as follows: Store the anomaly scores of the training set data in the array scores, set the number of neighbors k and the multiple m, sort the array scores in reverse order, and then establish the distances array. The calculation process of the elements in the distances array is as follows: distances[i] = k * scores[i] - scores[i + 1] - … - scores[i + k] Calculate the average value aver_distance of the elements in the distances array. Traverse the distances array. If the condition distances[i] < aver_distance * m is satisfied, then use scores[i] as the anomaly threshold.

2. The time series anomaly detection method based on variational autoencoder time series decomposition according to claim 1 is characterized in that: For a time series with periodicity, the decomposition formula is as follows: where x t is the time series value at time t, τ t is the time series value of the trend part at time t after the time series is decomposed by the STL method, s t is the seasonal part, r t For the remainder, keep the remainder r t and assign it to Construct a new time series for subsequent anomaly detection.

3. The time series anomaly detection method based on variational autoencoder time series decomposition according to claim 1 is characterized in that: For a time series without periodicity, the decomposition formula is as follows: The HP filter method decomposes the time series into two components: the trend part τ t and the fluctuating part r t , retain the trend part τ t and assign it to Construct a new time series for subsequent anomaly detection.

4. The time series anomaly detection method based on variational autoencoder time series decomposition according to claim 1 is characterized in that: The objective function of the cyclic variational autoencoder model in step S3 is as follows: Where w is the length of the sub-time series, t represents time t, and mse represents mean square error; represents the t-w+ith data of the time series generated by the variational self-encoder, represents the t-w+ith data of the original time series, β represents the hyperparameter for adjusting the weight, KL represents the calculation of the KL divergence of two Gaussian distributions, and q φ represents the encoder part of the model, z t-w+i represents the t-w+ith data of the latent variable z, Represents known time series data The premise is that the hidden variable z t-w+i The conditional probability distribution of t-w+i ) represents the latent variable z t-w+i The probability distribution of .

5. A computer device, characterized in that: The device includes: One or more processors; A memory; And one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors. When the program is executed by the processor, it implements the steps of the method described in any one of claims 1-4.

6. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by one or more processors, the one or more processors execute the steps of the method described in any one of claims 1-4.

Images