Time sequence anomaly detection method based on comparative learning and generative adversarial network

Abstract

The invention belongs to the technical field of data anomaly detection, and provides a time series anomaly detection method based on comparative learning and a generative adversarial network, which comprises the following steps: preprocessing a time series data set, performing data enhancement of different domains, and inputting the data into an encoder to obtain a first potential variable; performing comparative learning on the first potential variable to obtain a second potential variable; and performing generative adversarial network training on the second potential variable corresponding to the current batch of time series data through a generator and a discriminator, calculating an abnormal score of the time series data according to an error obtained by training, finding out the time series data of which the abnormal score is higher than a preset threshold, and marking the time series data as abnormal data. Compared with the defect of universality in the prior art, the invention provides a time sequence anomaly detection method suitable for different scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of data anomaly detection, and in particular to a time series anomaly detection method based on contrastive learning and generative adversarial networks. Background Art

[0002] In industrial digital systems, the large-scale deployment of sensors and controllers provides strong technical support for equipment monitoring, production process simulation, control and prediction. Sensors are used to monitor equipment, collect data and send it to the corresponding data receiver to help the system run in an efficient and continuous manner. However, these data are not only large in volume but also time-series, and their storage and calculation processes are complex. Once an abnormality occurs in the system, it will affect the quality of the data and the security and stability of the system. The main task of time series anomaly detection is to identify those abnormal points that obviously deviate from the normal operating mode from the data. Such anomalies may be caused by a variety of reasons such as equipment failure, operational errors or external interference. For example, in the field of network security, anomaly detection technology can help monitor network traffic, timely detect and prevent unauthorized access and potential network attacks, and ensure information security; in the field of medical health, by continuously monitoring the patient's physiological indicators such as heart rate, blood pressure, etc., anomaly detection can help medical personnel quickly identify potential health problems and intervene in time, thereby improving the efficiency and effectiveness of medical treatment. Therefore, whether in industrial production or daily life, time series anomaly detection plays an indispensable role. It not only helps to improve the automation and intelligence level of the system, but also greatly reduces the risks and losses caused by abnormal situations.

[0003] At present, most anomaly detection methods use autoencoders to capture the normal patterns of time series data. For example, in the encoder-decoder model based on the long short-term memory network, the encoder learns the potential representation of the input time series, and the decoder uses the potential representation to reconstruct the time series. The reconstruction error of the future time point is calculated to determine the possibility of anomalies at that point. If there is abnormal data that is relatively close to normal data, a simple encoder-decoder network will often miss the anomaly. In order to improve the model's ability to distinguish anomalies, the existing technology introduces an adversarial network into the encoder-decoder network. However, due to the time dependence of time series data, relying solely on the contextual information of a certain point is not enough to fully describe the characteristics of the entire time series, resulting in the model's performance in different scenarios being unstable and not extensive. Summary of the invention

[0004] In order to overcome the defect of insufficient applicability of the above-mentioned prior art in different scenarios, the present invention provides a time series anomaly detection method based on contrastive learning and generative adversarial network.

[0005] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0006] A time series anomaly detection method based on contrastive learning and generative adversarial network, the steps of which include:

[0007] S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows;

[0008] S2. For any batch of time series data, perform data enhancement in the time domain and frequency domain on each time window, input the enhanced time series data into the encoder to obtain a first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain a second latent variable;

[0009] S3, performing generative adversarial network training on the second latent variable corresponding to the current batch of time series data through the generator and the discriminator, and calculating the abnormal score of each time series data in each window of the current batch according to the error obtained by the training, finding the time series data in the current batch whose abnormal score is higher than the preset threshold and marking it as abnormal data;

[0010] S4. Repeat steps S2 to S3 for the next batch of time series data until all batches of time series data have completed anomaly detection.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] The present invention proposes a time series anomaly detection method based on contrastive learning and generative adversarial networks. In the encoding stage, the model uses the contrastive learning mechanism to enhance the recognition ability of commonalities and differences by comparing different data views of latent variables; in the reconstruction and adversarial stages of the adversarial network, the synergy of the generator and the discriminator effectively improves the accuracy of data reconstruction. By combining the comprehensive effects of contrastive learning loss, reconstruction error, and adversarial error, the model has stronger generalization ability and accuracy of anomaly detection when processing complex time series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flowchart of a time series anomaly detection method based on contrastive learning and generative adversarial network in Example 1. DETAILED DESCRIPTION

[0014] The accompanying drawings are only used for illustrative purposes and are not to be construed as limiting the present invention;

[0015] It is understandable to those skilled in the art that some well-known descriptions may be omitted in the drawings.

[0016] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0017] Example 1

[0018] This embodiment proposes a time series anomaly detection method based on contrastive learning and generative adversarial network. Figure 1 As shown, it is a flow chart of the time series anomaly detection method based on contrastive learning and generative adversarial network of this embodiment.

[0019] The time series anomaly detection method based on contrastive learning and generative adversarial network proposed in this embodiment includes the following steps:

[0020] S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows;

[0021] S2. For any batch of time series data, perform data enhancement in the time domain and frequency domain on each time window, input the enhanced time series data into the encoder to obtain a first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain a second latent variable;

[0022] S3, performing generative adversarial network training on the second latent variable corresponding to the current batch of time series data through the generator and the discriminator, and calculating the abnormal score of each time series data in each window of the current batch according to the error obtained by the training, finding the time series data in the current batch whose abnormal score is higher than the preset threshold and marking it as abnormal data;

[0023] S4. Repeat steps S2 to S3 for the next batch of time series data until all batches of time series data have completed anomaly detection.

[0024] In this embodiment, the time series data set is preprocessed and divided into several batches, and each batch is divided into several time windows. In the encoding stage, the model uses the contrastive learning mechanism to enhance the recognition ability of commonalities and differences by comparing different data views of latent variables, so that the model can better learn the potential features related to anomalies. In addition, in the reconstruction and adversarial stages of the adversarial network, the synergy of the generator and the discriminator not only effectively improves the accuracy of data reconstruction, but also enhances the amplification effect of abnormal data through adversarial loss. Combining the combined effects of contrastive learning loss, reconstruction error and adversarial error, the model has stronger generalization ability and accuracy of anomaly detection when processing complex time series data. This multi-level training framework can effectively identify time windows where the anomaly score is significantly higher than the threshold, greatly improves the model's detection rate of abnormal behavior, and reduces the false alarm rate, which is suitable for anomaly detection tasks of large-scale time series data.

[0025] In an optional embodiment, the step of preprocessing the time series data set includes: performing min-max normalization processing on the time series data, and the expression is as follows:

[0026]

[0027] Where t is time, x t is the unnormalized time series data, min(T) and max(T) are the minimum and maximum values ​​in the time series before normalization, respectively, and x′ t It is the time series data after normalization.

[0028] In this embodiment, normalization can convert various features with different units and magnitudes in the time series data to the same scale, reduce the scale differences between the various features, and help the model learn the normal behavior patterns of the data more stably; in addition, although normalization changes the absolute scale of the data, it can maintain the relative difference between outliers and normal values.

[0029] In an optional embodiment, the step of performing data enhancement in the time domain and the frequency domain on each time window in the time series data set includes:

[0030] In the time domain, Gaussian noise ε is added to the time series data, and its expression is as follows:

[0031] x tim =x t ′+q*ε

[0032] Among them, x tim is the data enhanced in the time domain, q is the scaling factor, the random variable ε has a mean of 0 and a variance of σ 2 Normal distribution of x′ t is the preprocessed time series data;

[0033] In the frequency domain, a two-dimensional discrete Fourier transform is performed on the time series data to obtain the spectrum F(u, v), and the spectrum is expressed as a combination of an amplitude spectrum and a phase spectrum, which is expressed as follows:

[0034]

[0035] Where u = 0, 1, ..., K-1 and v = 0, 1, ..., d-1 represent the frequency indexes, K is the window size, and d is x′ t The dimension of , A(u,v) is the amplitude spectrum, θ(u,v) is the phase spectrum, and k represents the dimension of the time series data;

[0036] Add Gaussian noise ε′ to the amplitude spectrum A(u, v) and phase spectrum θ(u, v) respectively to obtain the disturbed spectrum F′(u, v), which is expressed as follows:

[0037] F′(u,v)=(A(u,v)+ε′)(exp(jθ(u,v))+ε′)

[0038] Among them, the random variable ε′ has a mean of 0 and a variance of σ′ 2 Normal distribution of

[0039] Perform an inverse discrete Fourier transform to convert the frequency domain data back to the time domain data, and take the real part as the enhanced time series data, which is expressed as follows:

[0040]

[0041] x frre =Re(f(t, k))

[0042] Among them, x fre is the time series data after frequency domain enhancement, t = 0, 1, ..., K-1 and k = 0, 1, ..., d-1 represent the time step and dimension of the enhanced data f(t, k), respectively.

[0043] In this embodiment, time domain and frequency domain processing can reveal different characteristics of time series data, where time domain analysis emphasizes the time-varying characteristics of the data, while frequency domain analysis emphasizes the periodicity and frequency components of the data. This dual-view approach enables the model to simultaneously learn these two different but complementary characteristics of the time series; by enhancing in two different domains, the model can better identify abnormal patterns that may not be obvious in one domain. For example, some anomalies may not be easily perceived in the time domain, but may appear as unconventional frequency distributions in the frequency domain, and vice versa. In addition, using information in the time and frequency domains can also increase the diversity of training data, which helps the model learn a wider range of data features, thereby reducing the risk of overfitting.

[0044] In an optional embodiment, the encoder includes an LSTM long short-term memory network, a fully connected layer, and a ReLU rectified linear unit layer connected in sequence.

[0045] In this embodiment, the LSTM network is designed to solve the long-term dependency problem in traditional recurrent neural networks (RNNs). In time series data, the dependencies between events may span a long time interval, and LSTM can effectively learn these dependencies, thereby capturing key patterns in the data, including potential anomalies.

[0046] The fully connected layer can further process the features of the LSTM output and transform them into a form that is more suitable for prediction or classification. The ReLU modified linear unit layer introduces nonlinear characteristics, which helps improve the expressiveness of the model.

[0047] In an optional embodiment, the comparative learning of the first latent variable with all batches of time series data sets includes the following steps:

[0048] The loss function is used to bring the positive sample pairs in the latent variables corresponding to the current time window closer and push the negative sample pairs farther; wherein, the combination of the time domain and frequency domain view data of the same time series data is regarded as a positive sample pair, and the combination with the view data of the remaining time series data is regarded as a negative sample pair; the loss function is as follows:

[0049]

[0050] Among them, S is the number of time windows in the current batch, B is the number of batches into which the preprocessed time series dataset is divided, is the positive sample pair of the t-th time window in the i-th batch, τ is the temperature coefficient, n represents the n-th time window in the batch, I [n≠t] is an indicator function, if n≠t, the value is 1, otherwise the value is 0.

[0051] In this embodiment, contrastive learning strengthens the model's ability to distinguish between normal and abnormal patterns in the data. By minimizing the similarity loss of positive sample pairs and maximizing the similarity loss of negative sample pairs, the feature representation learned by the model can better reflect the intrinsic structure of time series data. The use of cosine similarity and temperature parameters enhances the discriminative power of the model, making similar and dissimilar samples more dispersed in high-dimensional space and improving the sensitivity of anomaly detection. In addition, contrastive learning can effectively reduce the impact of data imbalance and improve the model's recognition ability on minority classes by balancing the learning of positive and negative sample pairs.

[0052] Example 2

[0053] This embodiment makes improvements on the time series anomaly detection method based on contrastive learning and generative adversarial network proposed in Embodiment 1.

[0054] The time series anomaly detection method based on contrastive learning and generative adversarial network proposed in this embodiment includes the following steps:

[0055] S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows;

[0056] S2. For any batch of time series data, perform data enhancement in the time domain and frequency domain on each time window, input the enhanced time series data into the encoder to obtain a first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain a second latent variable;

[0057] S3, performing generative adversarial network training on the second latent variable corresponding to the current batch of time series data through the generator and the discriminator, and calculating the abnormal score of each time series data in each window of the current batch according to the error obtained by the training, finding the time series data in the current batch whose abnormal score is higher than the preset threshold and marking it as abnormal data;

[0058] S4. Repeat steps S2 to S3 for the next batch of time series data until all batches of time series data have completed anomaly detection.

[0059] In an optional embodiment, the training of the second latent variable through the generator and the discriminator to generate a generative adversarial network includes a reconstruction phase and an adversarial phase, and the weight ratio of the loss in the reconstruction phase in the adversarial network training is The weight of the loss in the adversarial stage in adversarial network training is Where n is the number of iterations of the current generative adversarial network training.

[0060] Further optionally, in the reconstruction stage, the second latent variable is respectively input into the generator and the discriminator for reconstruction and the reconstructed time series is output, and the loss function is:

[0061] L G =||O G -W t ||2

[0062] L D =||O D -W t ||2

[0063] Among them, L G and L D are the loss functions of the generator and the discriminator respectively, O G and O D are the outputs of the generator and discriminator respectively, W t It is the time series data in the current time window.

[0064] Further optionally, in the adversarial stage, the reconstructed time series output by the generator in the reconstruction stage is enhanced in the time domain and the frequency domain and then input into the encoder to obtain a first latent variable; and the first latent variable is subjected to comparative learning based on the time domain and the frequency domain to obtain a second latent variable, and the second latent variable is input into the discriminator to distinguish the data from the generator or the real input data, and the generator is trained to deceive the discriminator; and the loss function is:

[0065]

[0066] in, is the output of the discriminator; L G and L D are the loss functions of the generator and the discriminator respectively, W t It is a time series dataset.

[0067] In this embodiment, in the reconstruction phase, the goal of the generator and the discriminator is to reconstruct the second latent variable of the input as much as possible into the original input data. This helps the model learn the basic structure and characteristics of the time series data, and the ability to reproduce normal patterns is enhanced. The quality of the generated data is affected by the optimization of the reconstruction loss, so that the generated time series data is closer to the real data, thereby providing a high-quality foundation for the adversarial phase. In the adversarial phase, the generator attempts to deceive the discriminator so that it determines the reconstructed time series data as real data. This process drives the generator to generate more complex and diverse data, and improves the model's ability to handle abnormal situations. In addition, by dynamically adjusting the loss weight ratio of the reconstruction phase and the adversarial phase, the model can focus on optimizing the reconstruction quality or adversarial performance as needed during the training process; specifically, the reconstruction phase loss is given a higher weight in the early stage of model training to help the model stabilize and quickly learn the basic characteristics of the data. As the training progresses, the loss of the adversarial phase gradually occupies a larger proportion, enhancing the model's ability to identify and amplify anomalies.

[0068] In an optional embodiment, the step of calculating the abnormal score of each data in each window in the batch according to the error obtained by training includes:

[0069] For each window t, the abnormal score of the data in the window is calculated based on the final output of the generator and the discriminator. The calculation formula is as follows:

[0070]

[0071] Among them, s t,d is the anomaly score of the dth data in time window t, is the output of the generator for the dth data in time window t, is the output of the discriminator for the dth data in time window i, Wt,d is the dth data in time window t.

[0072] In this embodiment, by combining the outputs of the two generators and the discriminator, it is possible to capture subtle variations in the data more carefully, thereby enhancing the model's sensitivity to abnormal conditions.

[0073] In an optional embodiment, the preset threshold value is the value of the threshold value when the F1 score is the maximum value;

[0074] The calculation formula of the F1 score is as follows:

[0075]

[0076] Among them, TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives.

[0077] Specifically, F1 Score is used as an indicator to evaluate model performance. The calculation of F1 Score requires the number of true positives (abnormal data is identified as abnormal), false positives (normal data is identified as abnormal), and false negatives (abnormal data is identified as normal). To determine whether the data is abnormal, the obtained anomaly score needs to be compared with the threshold. If it is higher than the threshold, it is marked as abnormal.

[0078] In this embodiment, by adjusting the threshold to optimize the F1 score, the best balance between precision and recall can be achieved, and by maximizing the F1 score, the reliability of model detection can be improved.

[0079] Example 3

[0080] In this embodiment, the time series anomaly detection method based on contrastive learning and generative adversarial network proposed in Embodiment 1 or Embodiment 2 is used to perform time series anomaly detection on the WADI dataset.

[0081] The WADI dataset is a dataset used for network security research in industrial control systems (ICS). The WADI dataset can reflect the data output of various sensors and devices during normal operation of the water distribution system.

[0082] In this embodiment, the WADI data set is first obtained, the WADI data set is normalized and divided into batches, and each batch is divided into multiple time windows. Taking the time window W as an example, the time series data in the window is first enhanced in both the time domain and the frequency domain, and then input into the encoder for encoding to obtain the first latent variable, and the first latent variable is compared and learned to obtain the second latent variable, and then the second latent variable is input into the generator and the discriminator for reconstruction training to output the reconstructed time series, ensuring that both have a certain reconstruction sequence ability, which is the reconstruction stage training; the reconstructed time series output by the generator in the reconstruction stage is input into the discriminator for adversarial stage training to amplify the inconspicuous anomalies. The anomaly score of each data in the window W is calculated based on the reconstruction error and the adversarial error, and compared based on the preset threshold that maximizes the F1 score. If the anomaly score is greater than the anomaly threshold, the time data and the time window where the data is located are marked as abnormal, otherwise the next data is judged until all time series data are detected.

[0083] The terms in the drawings are for illustrative purposes only and should not be construed as limiting this patent;

[0084] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A time series anomaly detection method based on contrastive learning and generative adversarial network, characterized in that: The following steps are involved: S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows; S2. For any batch of time series data, perform data enhancement in the time domain and frequency domain on each time window, input the enhanced time series data into the encoder to obtain a first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain a second latent variable; S3, performing generative adversarial network training on the second latent variable corresponding to the current batch of time series data through the generator and the discriminator, and calculating the abnormal score of each time series data in each window of the current batch according to the error obtained by the training, finding the time series data in the current batch whose abnormal score is higher than the preset threshold and marking it as abnormal data; S4. Repeat steps S2 to S3 for the next batch of time series data until all batches of time series data have completed anomaly detection.

2. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 1 is characterized in that: The step of preprocessing the time series data set includes: performing min-max normalization processing on the time series data, and the expression is as follows: Where t is time, x t is the unnormalized time series data, min(T) and max(T) are the minimum and maximum values ​​in the time series before normalization, respectively. t ′ is the time series data after normalization.

3. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 1 is characterized in that: The step of performing data enhancement in the time domain and the frequency domain on each time window in the time series data set comprises: In the time domain, Gaussian noise ε is added to the time series data, and its expression is as follows: x tim =x t ′+q*ε Among them, x tim is the data enhanced in the time domain, q is the scaling factor, the random variable ε has a mean of 0 and a variance of σ 2 Normal distribution of x t ′ is the preprocessed time series data; In the frequency domain, a two-dimensional discrete Fourier transform is performed on the time series data to obtain the spectrum F(u,v), and the spectrum is expressed as a combination of an amplitude spectrum and a phase spectrum, which is expressed as follows: Where u = 0, 1, ..., K-1 and v = 0, 1, ..., d-1 represent the frequency index, K is the window size, and d is x. t ′ is the dimension of the time series data, A(u,v) is the amplitude spectrum, θ(u,v) is the phase spectrum, and k represents the dimension of the time series data; Add Gaussian noise ε′ to the amplitude spectrum A(u,v) and phase spectrum θ(u,v) respectively to obtain the disturbed spectrum F′(u,v), which is expressed as follows: F′(u,v)=(A(u,v)+ε′)(exp(jθ(u,v))+ε′) Among them, the random variable ε′ has a mean of 0 and a variance of σ ′2 Normal distribution of Perform an inverse discrete Fourier transform to convert the frequency domain data back to the time domain data, and take the real part as the enhanced time series data, which is expressed as follows: x fre =Re(f(t,k)) Among them, x fre is the time series data after frequency domain enhancement, t=0,1,...,K-1 and k=0,1,...,d-1 represent the time step and dimension of the enhanced data f(t,k) respectively.

4. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 1 is characterized in that: The encoder includes an LSTM long short-term memory network, a fully connected layer and a ReLU rectified linear unit layer connected in sequence.

5. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 1 is characterized in that: The comparative learning of the first latent variable based on the time domain and the frequency domain comprises the following steps: The loss function is used to bring the positive sample pairs in the latent variables corresponding to the current time window closer and push the negative sample pairs farther; wherein, the combination of the time domain and frequency domain view data of the same time series data is regarded as a positive sample pair, and the combination with the view data of the remaining time series data is regarded as a negative sample pair; the loss function is as follows: Among them, S is the number of time windows in the current batch, B is the number of batches into which the preprocessed time series dataset is divided, is the positive sample pair of the t-th time window in the i-th batch, τ is the temperature coefficient, n represents the n-th time window in the batch, I [n≠t] is an indicator function, if n≠t, the value is 1, otherwise the value is 0.

6. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 1 is characterized in that: The generative adversarial network training of the second latent variable by the generator and the discriminator includes a reconstruction phase and an adversarial phase, and the weight proportion of the loss in the reconstruction phase in the adversarial network training is The weight of the loss in the adversarial stage in adversarial network training is Where n is the number of iterations of the current generative adversarial network training.

7. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 6 is characterized in that: In the reconstruction stage, the second latent variable is input into the generator and the discriminator respectively for reconstruction and outputting the reconstructed time series, and the loss function is: L G =||O G -W t ||2 L D =||O D -W t ||2 Among them, L G and L D are the loss functions of the generator and the discriminator respectively, O G and O D are the outputs of the generator and discriminator respectively, W t It is the time series data in the current time window.

8. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 7 is characterized in that: In the adversarial stage, the reconstructed time series output by the generator in the reconstruction stage is enhanced in the time domain and the frequency domain and then input into the encoder to obtain a first latent variable; the first latent variable is subjected to comparative learning based on the time domain and the frequency domain to obtain a second latent variable, and the second latent variable is input into the discriminator to distinguish the data from the generator or the real input data, and the generator is trained to deceive the discriminator; its loss function is: in, is the output of the discriminator; L G and L D are the loss functions of the generator and the discriminator respectively, W t It is a time series dataset.

9. The time series anomaly detection method based on contrastive learning and generative adversarial network according to claim 6, characterized in that: The step of calculating the abnormal score of each data in each window in the batch according to the error obtained by training includes: For each window t, the abnormal score of the data in the window is calculated based on the final output of the generator and the discriminator. The calculation formula is as follows: Among them, s t,d is the anomaly score of the dth data in time window t, is the output of the generator for the dth data in time window t, is the output of the discriminator for the dth data in time window i, W t,d is the dth data in time window t.

10. The time series anomaly detection method based on contrastive learning and generative adversarial network according to any one of claims 1 to 9, characterized in that: The preset threshold value is the value of the threshold value when the F1 score is the maximum value; The calculation formula of the F1 score is as follows: Among them, TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives.

Images