A Time Series Anomaly Detection Method Based on Contrastive Learning and Generative Adversarial Networks

Abstract

This invention belongs to the field of data anomaly detection technology, proposing a time series anomaly detection method based on contrastive learning and generative adversarial networks. The steps include: preprocessing the time series dataset, performing data augmentation in different domains, and inputting the data into an encoder to obtain a first latent variable; performing contrastive learning on the first latent variable to obtain a second latent variable; training a generative adversarial network on the second latent variable corresponding to the current batch of time series data using a generator and a discriminator, calculating anomaly scores for the time series data based on the training error, and identifying and marking time series data with anomaly scores higher than a preset threshold as anomalous data. Compared to the lack of versatility in existing technologies, this invention proposes a time series anomaly detection method applicable to different scenarios.

Description

Technical Field

[0001] This invention relates to the field of data anomaly detection technology, specifically to a time series anomaly detection method based on contrastive learning and generative adversarial networks. Background Technology

[0002] In industrial digital systems, the large-scale deployment of sensors and controllers provides strong technical support for equipment monitoring, production process simulation, and control and prediction. Sensors monitor equipment, collect data, and send it to corresponding data receivers to help the system operate efficiently and continuously. However, this data is not only massive in volume but also time-series, and its storage and computation processes are complex. Anomalies in the system can impact data quality and the system's security and stability. The main task of time series anomaly detection is to identify anomalous points in the data that significantly deviate from normal operating patterns. These anomalies may be caused by various reasons, such as equipment failure, operational errors, or external interference. For example, in the field of cybersecurity, anomaly detection technology can help monitor network traffic, promptly detect and prevent unauthorized access and potential cyberattacks, and ensure information security. In the field of healthcare, by continuously monitoring patients' physiological indicators such as heart rate and blood pressure, anomaly detection can help medical personnel quickly identify potential health problems and intervene in a timely manner, thereby improving the efficiency and effectiveness of medical care. Therefore, whether in industrial production or daily life, time series anomaly detection plays an indispensable role. It not only helps improve the automation and intelligence level of systems but also significantly reduces the risks and losses caused by anomalies.

[0003] Currently, most anomaly detection methods use autoencoders to capture normal patterns in time series data. For example, encoder-decoder models based on Long Short-Term Memory (LSTM) networks involve the encoder learning a latent representation of the input time series, and the decoder using this representation to reconstruct the time series. The probability of an anomaly at a future time point is determined by calculating the reconstruction error. However, simple encoder-decoder networks often miss anomalies if they exist that are relatively close to normal data. To improve the model's ability to identify anomalies, existing techniques introduce adversarial networks into the encoder-decoder network. However, due to the temporal dependence of time series data, relying solely on the contextual information of a single point is insufficient to comprehensively describe the features of the entire time series, resulting in unstable model performance across different scenarios and a lack of broad applicability. Summary of the Invention

[0004] To overcome the shortcomings of the prior art in terms of applicability in different scenarios, the present invention provides a time series anomaly detection method based on contrastive learning and generative adversarial networks.

[0005] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows:

[0006] A time series anomaly detection method based on contrastive learning and generative adversarial networks includes the following steps:

[0007] S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows;

[0008] S2. For any batch of time series data, perform time-domain and frequency-domain data augmentation on each time window, input the augmented time series data into the encoder to obtain the first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain the second latent variable.

[0009] S3. Generative adversarial network training is performed on the second latent variable corresponding to the current batch of time series data through generator and discriminator, and the abnormal score of each time series data in each window of the current batch is calculated according to the training error. Time series data with abnormal scores higher than the preset threshold in the current batch are identified and marked as abnormal data.

[0010] S4. Repeat steps S2 to S3 for the next batch of time series data until anomaly detection is completed for all batches of time series data.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0012] This invention proposes a time series anomaly detection method based on contrastive learning and generative adversarial networks (GANs). In the encoding stage, the contrastive learning mechanism enhances the model's ability to identify commonalities and differences by comparing different data views of latent variables. In the reconstruction and adversarial phases of the adversarial network, the synergistic effect of the generator and discriminator effectively improves the accuracy of data reconstruction. By combining the combined effects of contrastive learning loss, reconstruction error, and adversarial error, the model exhibits stronger generalization ability and higher anomaly detection accuracy when processing complex time series data. Attached Figure Description

[0013] Figure 1 This is a flowchart of a time series anomaly detection method based on contrastive learning and generative adversarial networks, as described in Example 1. Detailed Implementation

[0014] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention.

[0015] It will be understood by those skilled in the art that some well-known descriptions may be omitted in the accompanying drawings.

[0016] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Example 1

[0018] This embodiment proposes a time series anomaly detection method based on contrastive learning and generative adversarial networks, such as... Figure 1 The diagram shown is a flowchart of the time series anomaly detection method based on contrastive learning and generative adversarial networks in this embodiment.

[0019] The time series anomaly detection method based on contrastive learning and generative adversarial networks proposed in this embodiment includes the following steps:

[0020] S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows;

[0021] S2. For any batch of time series data, perform time-domain and frequency-domain data augmentation on each time window, input the augmented time series data into the encoder to obtain the first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain the second latent variable.

[0022] S3. Generative adversarial network training is performed on the second latent variable corresponding to the current batch of time series data through generator and discriminator, and the abnormal score of each time series data in each window of the current batch is calculated according to the training error. Time series data with abnormal scores higher than the preset threshold in the current batch are identified and marked as abnormal data.

[0023] S4. Repeat steps S2 to S3 for the next batch of time series data until anomaly detection is completed for all batches of time series data.

[0024] In this embodiment, the time series dataset is preprocessed and divided into several batches, with each batch further divided into several time windows. During the encoding stage, a contrastive learning mechanism is used to enhance the model's ability to identify commonalities and differences by comparing different data views of latent variables, thereby enabling the model to better learn latent features related to anomalies. Furthermore, in the reconstruction and adversarial phases of the adversarial network, the synergistic effect of the generator and discriminator not only effectively improves the accuracy of data reconstruction but also enhances the amplification effect of anomalous data through adversarial loss. Combining the combined effects of contrastive learning loss, reconstruction error, and adversarial error, the model possesses stronger generalization ability and higher accuracy in anomaly detection when processing complex time series data. This multi-layered training framework can effectively identify time windows where anomaly scores are significantly higher than a threshold, greatly improving the model's detection rate of anomalous behavior while reducing the false positive rate, making it suitable for anomaly detection tasks involving large-scale time series data.

[0025] In an optional embodiment, the step of preprocessing the time series dataset includes: performing min-max normalization on the time series data, the expression of which is as follows:

[0026]

[0027] Where t is time, x t For unnormalized time series data, min(T) and max(T) are the minimum and maximum values ​​in the time series before normalization, respectively. x′ t This is the time series data after normalization.

[0028] In this embodiment, normalization can transform various features with different units and magnitudes in time series data to the same scale, reduce the scale difference between features, and help the model learn the normal behavior pattern of the data more stably. In addition, although normalization changes the absolute scale of the data, it can maintain the relative difference between outliers and normal values.

[0029] In an optional embodiment, the step of performing time-domain and frequency-domain data augmentation on each time window within the time-series dataset includes:

[0030] In the time domain, Gaussian noise ε is added to time series data, and its expression is as follows:

[0031] x tim =x t ′+q*ε

[0032] Where, x tim The data is augmented in the time domain, q is the scaling factor, and the random variable ε follows a mean of 0 and a variance of σ. 2 The normal distribution; x′ t This is preprocessed time series data;

[0033] In the frequency domain, a two-dimensional discrete Fourier transform is performed on the time series data to obtain the spectrum F(u, v), which is expressed as a combination of the amplitude spectrum and the phase spectrum, as follows:

[0034]

[0035] Where u = 0, 1, ..., K-1 and v = 0, 1, ..., d-1 represent the frequency indices, K is the window size, and d is the frequency of x′. t The dimension of the time series data is denoted by A(u,v), where A(u,v) is the amplitude spectrum, θ(u,v) is the phase spectrum, and k represents the dimension of the time series data.

[0036] Gaussian noise ε′ is added to the amplitude spectrum A(u, v) and the phase spectrum θ(u, v) respectively to obtain the perturbed spectrum F′(u, v), which is expressed as follows:

[0037] F′(u,v)=(A(u,v)+ε′)(exp(jθ(u,v))+ε′)

[0038] Wherein, the random variable ε′ follows a function with a mean of 0 and a variance of σ′. 2 The normal distribution;

[0039] Performing the inverse discrete Fourier transform converts the frequency domain data back to the time domain data, and taking the real part as the enhanced time series data, its expression is as follows:

[0040]

[0041] x frre =Re(f(t, k))

[0042] Where, x fre For the frequency domain augmented time series data, t = 0, 1, ..., K-1 and k = 0, 1, ..., d-1 represent the time step and dimension of the augmented data f(t, k), respectively.

[0043] In this embodiment, time-domain and frequency-domain processing can reveal different characteristics of time-series data. Time-domain analysis emphasizes the temporal variation characteristics of the data, while frequency-domain analysis emphasizes the periodicity and frequency components. This dual-view approach allows the model to learn these two different but complementary features of the time series simultaneously. By enhancing the data in two different domains, the model can better identify anomalous patterns that might not be obvious in one domain. For example, some anomalies may be difficult to detect in the time domain but may manifest as irregular frequency distributions in the frequency domain, and vice versa. Furthermore, utilizing information from both the time and frequency domains can increase the diversity of the training data, which helps the model learn a wider range of data features, thereby reducing the risk of overfitting.

[0044] In one alternative embodiment, the encoder includes an LSTM long short-term memory network, a fully connected layer, and a ReLU corrected linear unit layer connected in sequence.

[0045] In this embodiment, the LSTM network is designed to address the long-term dependency problem in traditional recurrent neural networks (RNNs). In time series data, dependencies between events may span long time intervals, and LSTM can effectively learn these dependencies, thereby capturing key patterns in the data, including potential anomalies.

[0046] Fully connected layers can further process the features output by the LSTM, transforming them into a form more suitable for prediction or classification. ReLU correction of the non-linearity introduced by the linear unit layers helps improve the model's expressive power.

[0047] In an optional embodiment, the step of comparing the first latent variable with time series datasets from all batches includes the following steps:

[0048] The loss function is used to bring positive sample pairs closer together and push negative sample pairs further apart from the latent variables corresponding to the current time window. Specifically, combinations of both time-domain and frequency-domain views of the same time series data are considered positive sample pairs, while combinations with views of other time series data are considered negative sample pairs. The loss function is as follows:

[0049]

[0050] Where S is the number of time windows within the current batch, and B is the number of batches into which the preprocessed time series dataset is divided. Let I represent the positive sample pairs in the i-th batch and the t-th time window, where τ is the temperature coefficient and n represents the n-th time window within that batch. [n≠t] This is an index function; if n≠t, the value is 1, otherwise the value is 0.

[0051] In this embodiment, contrastive learning enhances the model's ability to distinguish between normal and abnormal patterns in the data. By minimizing the similarity loss of positive sample pairs and maximizing the similarity loss of negative sample pairs, the feature representation learned by the model better reflects the inherent structure of the time series data. Using cosine similarity and temperature parameters enhances the model's discriminative power, making similar and dissimilar samples more dispersed in the high-dimensional space, thus improving the sensitivity of anomaly detection. Furthermore, by balancing the learning of positive and negative sample pairs, contrastive learning can effectively mitigate the impact of data imbalance and improve the model's ability to identify minority classes.

[0052] Example 2

[0053] This embodiment improves upon the time series anomaly detection method based on contrastive learning and generative adversarial networks proposed in Embodiment 1.

[0054] The time series anomaly detection method based on contrastive learning and generative adversarial networks proposed in this embodiment includes the following steps:

[0055] S1. Preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows;

[0056] S2. For any batch of time series data, perform time-domain and frequency-domain data augmentation on each time window, input the augmented time series data into the encoder to obtain the first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain the second latent variable.

[0057] S3. Generative adversarial network training is performed on the second latent variable corresponding to the current batch of time series data through generator and discriminator, and the abnormal score of each time series data in each window of the current batch is calculated according to the training error. Time series data with abnormal scores higher than the preset threshold in the current batch are identified and marked as abnormal data.

[0058] S4. Repeat steps S2 to S3 for the next batch of time series data until anomaly detection is completed for all batches of time series data.

[0059] In an optional embodiment, the generative adversarial network training of the second latent variable using a generator and a discriminator includes a reconstruction phase and an adversarial phase, wherein the weight of the loss in the reconstruction phase in the adversarial network training is [percentage missing]. The weight of the loss in the adversarial phase in the training of the adversarial network is as follows: Where n is the current iteration round of the generative adversarial network training.

[0060] Further, optionally, in the reconstruction stage, the second latent variable is input into the generator and discriminator respectively for reconstruction and the reconstructed time series is output, with the loss function being:

[0061] L G =||O G -W t ||2

[0062] L D =||O D -W t ||2

[0063] Among them, L G and L D The loss functions for the generator and discriminator are O, respectively. G and O D W represents the output of the generator and the discriminator, respectively. t This refers to time series data within the current time window.

[0064] Optionally, in the adversarial phase, the reconstructed time series output by the generator in the reconstruction phase is augmented in both the time and frequency domains and then input into the encoder to obtain a first latent variable; and the first latent variable is subjected to comparative learning based on the time and frequency domains to obtain a second latent variable. The second latent variable is then input into the discriminator to distinguish between data from the generator and real input data, and the generator is trained to deceive the discriminator; its loss function is:

[0065]

[0066] in, The output of the discriminator; L G and L D The loss functions W for the generator and discriminator are respectively. t This is a time series dataset.

[0067] In this embodiment, during the reconstruction phase, the generator and discriminator aim to reconstruct the second latent variable into the original input data as accurately as possible. This helps the model learn the basic structure and features of time-series data, enhancing its ability to reproduce normal patterns. Optimizing the reconstruction loss affects the quality of the generated data, making the generated time-series data closer to real data, thus providing a high-quality foundation for the adversarial phase. In the adversarial phase, the generator attempts to deceive the discriminator into classifying the reconstructed time-series data as real data. This process drives the generator to generate more complex and diverse data, improving the model's ability to handle anomalies. Furthermore, by dynamically adjusting the weight ratio of the losses in the reconstruction and adversarial phases, the model can prioritize optimizing reconstruction quality or adversarial performance during training as needed. Specifically, the reconstruction phase loss is given a higher weight in the early stages of model training to help the model stabilize and quickly learn the basic features of the data. As training progresses, the adversarial phase loss gradually takes on a larger proportion, enhancing the model's ability to identify and amplify anomalies.

[0068] In an optional embodiment, the step of calculating the outlier score for each data point within each window of the batch based on the training error includes:

[0069] For each window t, the anomaly score of the data within that window is calculated based on the final outputs of the generator and discriminator, as follows:

[0070]

[0071] Among them, s t,d The outlier score for the d-th data point within time window t. For the generator's output on the d-th data point within time window t, For the discriminator to output the d-th data point within time window i, Wt,d This represents the d-th data point within the time window t.

[0072] In this embodiment, by combining the outputs of two generators and discriminators, it is possible to capture subtle variations in the data more precisely, thereby enhancing the model's sensitivity to abnormal states.

[0073] In one optional embodiment, the preset threshold is the value of the threshold that maximizes the F1 score;

[0074] The formula for calculating the F1 score is as follows:

[0075]

[0076] Where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives.

[0077] Specifically, the F1 Score is used as an indicator to evaluate the model's performance. The calculation of the F1 Score requires the number of true positives (abnormal data is identified as anomalous), false positives (normal data is identified as anomalous), and false negatives (abnormal data is identified as normal). To determine whether the data is anomalous, the obtained anomalous score is compared with a threshold. If it is higher than the threshold, it is marked as anomalous.

[0078] In this embodiment, by adjusting the threshold to optimize the F1 score, the best balance between precision and recall can be achieved, and by maximizing the F1 score, the reliability of model detection can be improved.

[0079] Example 3

[0080] In this embodiment, the time series anomaly detection method based on contrastive learning and generative adversarial networks proposed in Embodiment 1 or Embodiment 2 is used to perform time series anomaly detection on the WADI dataset.

[0081] The WADI dataset is a dataset used for cybersecurity research in Industrial Control Systems (ICS). The WADI dataset reflects the data output of various sensors and devices in a water distribution system during normal operation.

[0082] In this embodiment, the WADI dataset is first acquired, normalized, and divided into batches. Each batch is further divided into multiple time windows. Taking time window W as an example, the time series data within this window undergoes both time and frequency domain augmentation before being input into the encoder for encoding, yielding a first latent variable. The first latent variable is then subjected to comparative learning to obtain a second latent variable. Subsequently, the second latent variable is input into the generator and discriminator for reconstruction training, outputting a reconstructed time series to ensure that both have a certain ability to reconstruct sequences; this is the reconstruction phase training. The reconstructed time series output by the generator during the reconstruction phase is input into the discriminator for adversarial phase training, amplifying inconspicuous anomalies. Anomaly scores for each data point within window W are calculated based on the reconstruction error and adversarial error. These scores are compared with a preset threshold that maximizes the F1 score. If the anomaly score is greater than the threshold, the data point and its corresponding time window are marked as anomaly; otherwise, the next data point is evaluated until all time series data have been detected.

[0083] The terminology used in the accompanying drawings is for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0084] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A time series anomaly detection method based on contrastive learning and generative adversarial networks, characterized in that, Includes the following steps: S1. Obtain the WADI dataset, which includes data outputs from various sensors and devices during the operation of the water distribution system. Use the WADI dataset as a time series dataset, preprocess the time series dataset and divide it into several batches, and divide each batch into several time windows. S2. For any batch of time series data, perform time-domain and frequency-domain data augmentation on each time window, input the data-augmented time series data into the encoder to obtain the first latent variable; and perform comparative learning on the first latent variable based on the time domain and frequency domain to obtain the second latent variable. S3. Generative adversarial network training is performed on the second latent variable corresponding to the current batch of time series data through generator and discriminator, and the abnormal score of each time series data in each window of the current batch is calculated according to the training error. Time series data with abnormal scores higher than the preset threshold in the current batch are identified and marked as abnormal data. S4. Repeat steps S2 to S3 for the next batch of time series data until anomaly detection is completed for all batches of time series data. The steps of performing time-domain and frequency-domain data augmentation on each time window include: In the time domain, Gaussian noise is added to the time series data. Its expression is as follows: in, For data augmented in the time domain, q Scaling factor, random variable It follows a mean of 0 and a variance of . The normal distribution; This is preprocessed time series data; In the frequency domain, the spectrum is obtained by performing a two-dimensional discrete Fourier transform on the time series data. F ( u,v The spectrum is expressed as a combination of amplitude and phase spectra, and its expression is as follows: in u =0,1,..., K -1 and v =0,1,..., d -1 represents the frequency index, K For window size, d for Dimensions A ( u,v () represents the amplitude spectrum. ( u,v () represents the phase spectrum. k Dimensions of time series data; Give the amplitude spectrum respectively A ( u,v and phase spectrum ( u,v Add Gaussian noise The perturbated spectrum is obtained. ( u,v Its expression is as follows: Among them, random variables It follows a mean of 0 and a variance of . The normal distribution; Performing the inverse discrete Fourier transform converts the frequency domain data back to the time domain data, and taking the real part as the enhanced time series data, its expression is as follows: in, For time series data enhanced in the frequency domain, t =0,1,..., K -1 and k =0,1,..., d -1 represents augmented data. f ( t,k The time step and dimension; The comparative learning of the first latent variable based on the time domain and frequency domain includes the following steps: The loss function brings positive sample pairs closer to each other and pushes negative sample pairs further apart in the latent variables corresponding to the current time window. Specifically, the combination of the time domain and frequency domain view data of the same time series data is regarded as a positive sample pair, and the combination of the view data with the view data of other time series data is regarded as a negative sample pair.

2. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 1, characterized in that, The step of preprocessing the time series dataset includes: performing min-max normalization on the time series data, the expression of which is as follows: in, t For time, For unnormalized time series data, min( T ) and max( T These are the minimum and maximum values ​​of the time series before normalization, respectively. This is the time series data after normalization.

3. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 1, characterized in that, The encoder comprises an LSTM long short-term memory network, a fully connected layer, and a ReLU corrected linear unit layer connected in sequence.

4. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 1, characterized in that, The loss function is as follows: in, S This represents the number of time windows within the current batch. B The preprocessed time series dataset is divided into several batches. For the first i The first batch t Positive sample pairs within a time window For temperature coefficient, n Indicates the first in this batch n A time window, For the index function, if The value is 1 if it is true, otherwise the value is 0.

5. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 1, characterized in that, The generative adversarial network training of the second latent variable using a generator and a discriminator includes a reconstruction phase and an adversarial phase, wherein the weight of the loss in the reconstruction phase in the adversarial network training is [percentage missing]. The weight of the loss in the adversarial phase in the training of the adversarial network is (1- );in, n This is the current iteration of the generative adversarial network training.

6. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 5, characterized in that, In the reconstruction phase, the second latent variable is input into the generator and discriminator respectively for reconstruction, and the reconstructed time series is output. The loss function is: in, and The loss functions for the generator and discriminator are respectively. and The outputs of the generator and discriminator are respectively. This refers to time series data within the current time window.

7. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 6, characterized in that, In the adversarial phase, the reconstructed time series output by the generator in the reconstruction phase is augmented in both the time and frequency domains and then input into the encoder to obtain a first latent variable. Based on the time and frequency domains, the first latent variable is subjected to comparative learning to obtain a second latent variable. This second latent variable is then input into the discriminator to distinguish between data from the generator and real input data, and the generator is trained to deceive the discriminator. The loss function is: in, This is the output of the discriminator; and The loss functions for the generator and discriminator are respectively. This is a time series dataset.

8. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to claim 5, characterized in that, The step of calculating the outlier score for each data point within each window of the batch based on the error obtained from training includes: For each window t The anomaly score of the data within this window is calculated based on the final outputs of the generator and discriminator, using the following formula: in, For time window t Inner d Outlier scores for each data point For the generator in the time window t The above is the first d The output of each data point, For the discriminator in the time window i The above is the first d One data output, For time window t Upper d Data.

9. The time series anomaly detection method based on contrastive learning and generative adversarial networks according to any one of claims 1 to 8, characterized in that, The preset threshold value is the value of the threshold that maximizes the F1 score; The formula for calculating the F1 score is as follows: in, It is the number of true cases. It is the number of false positives. It represents the number of false negatives.

Images