Multi-dimensional time series data anomaly detection method, device and medium based on adversarial training and frequency domain improved self-attention mechanism

Abstract

The method based on the adversarial training and the frequency domain improved self-attention mechanism solves the problems that the robustness of the model is poor, the external interference is easily received, and the accuracy of the detection result is reduced; and the frequency domain features of the time series data are not fully considered, the model precision is limited, and the analysis structure is not accurate. The method is divided into two stages: a training stage and an abnormality detection stage. In the training stage, the model needs to be trained by using the historical collected data, so that the model parameters can fit the corresponding application scene. In the abnormality detection stage, when each latest real data is generated, the model needs to reconstruct the time series data with a fixed time window length ending at the latest time point, to perform residual error calculation with the original real data, and to judge whether the current data is abnormal data according to the residual error size of the real data and the reconstructed data. The application is suitable for industrial equipment fault detection and production line operation state detection and analysis.

Description

Technical Field

[0001] This invention belongs to the field of intelligent fault diagnosis technology, and specifically relates to a method for detecting time-series data anomalies. Background Technology

[0002] Anomaly detection in time-series data is one of the most mature methods in time-series data analysis. Efficient anomaly detection techniques are widely used in various fields of real life. In the context of comprehensive informatization in industrial production, the types of sensors on industrial equipment are more numerous, the sampling frequency is higher, and the data accumulation speed is extremely fast. This further increases the difficulty of anomaly detection when anomalies were originally very rare. Faced with massive amounts of monitoring data and numerous indicators in industrial time-series data, traditional purely manual anomaly detection becomes impractical. Modern industrial anomaly detection has gradually moved towards artificial intelligence. Traditional statistical and machine learning methods mainly use historical data modeling, dimensionality reduction detection of input data, or detection based on classification or clustering methods. However, in complex industrial production processes, there are direct or indirect relationships between data. Traditional machine learning models struggle to extract direct connections between targets from high-dimensional data. The emergence of deep learning technology has alleviated this problem. However, current deep learning models in the field of anomaly detection still generally suffer from two problems:

[0003] 1. The models exhibit poor robustness and are susceptible to noise. Current time-series anomaly detection models suffer from insufficient robustness, which may pose challenges in practical applications. Furthermore, these models are easily affected by external factors such as noise, leading to a decrease in the accuracy of detection results.

[0004] 2. The frequency domain characteristics of time series data are not adequately considered. In time series data analysis, frequency domain characteristics are just as crucial as time domain features. However, current time series data analysis often fails to adequately consider frequency domain features, potentially leading to inaccurate results in complex scenarios. Specifically, frequency domain features typically include signal frequency, amplitude, phase, and power spectral density, providing deeper signal analysis and diagnostic information. However, traditional time series data analysis methods usually only focus on time domain features, such as simple indicators like mean, variance, maximum, and minimum values, without fully utilizing frequency domain features. This can easily overlook potential information, limiting model accuracy. To better utilize the frequency domain characteristics of time series data, advanced analysis techniques such as Fourier transform and wavelet transform can be employed to convert time-domain signals into frequency-domain signals and extract important features. Furthermore, machine learning or deep learning-based methods can be used to train and optimize models in conjunction with frequency domain features. Summary of the Invention

[0005] The purpose of this invention is to address the problem of anomaly detection in time-series data and overcome the shortcomings of existing technologies. By fully utilizing the frequency domain features of time-series data through discrete wavelet decomposition and Fourier transform, and combining the advantages of robustness of adversarial training, a multi-dimensional anomaly detection method for time-series data based on adversarial training and a frequency domain-modified self-attention mechanism is proposed. This method solves the problems of poor model robustness, susceptibility to external interference leading to decreased accuracy of detection results, and insufficient consideration of the frequency domain features of time-series data, resulting in limited model accuracy and inaccurate analysis structure.

[0006] This invention is achieved through the following technical solution: This invention proposes a method, device, and medium for detecting anomalies in multidimensional time-series data based on adversarial training and frequency domain improved self-attention mechanism.

[0007] The method specifically includes the following steps:

[0008] Step 1: Obtain the raw dataset and preprocess it;

[0009] Step 2: Construct an anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism;

[0010] Step 3: Train the anomaly detection model described in Step 2, which is based on adversarial training and frequency domain improved self-attention mechanism;

[0011] Step 4: Input the time series data to be detected into the model trained in Step 3 to perform anomaly detection.

[0012] Further, step 1 includes the following steps:

[0013] Step 1.1: Divide the original dataset into a training set and a test set;

[0014] Step 1.2: Define a sliding window with a sliding step size of 1, and set the window size according to the original dataset;

[0015] The sliding window contains continuous time series data with a fixed length of time points.

[0016] The sliding step size is the distance the sliding window moves on the original dataset each time.

[0017] Step 1.3: Slide the sliding window over the original dataset, dividing the data within each sliding window into a fixed-length subsequence;

[0018] Step 1.4: Store the subsequences obtained in Step 1.3 in a set for use as input for model training;

[0019] Furthermore, step 2 includes the following steps:

[0020] Step 2.1: Construct a positional encoding layer, calculate the positional encoding information of the input subsequence, and append it to the input;

[0021] Step 2.2: Construct the encoder structure, which consists of a frequency-domain enhanced self-attention module based on wavelet decomposition and Fourier transform, residual links, normalization and fully connected neural networks connected in series.

[0022] Step 2.3: Construct two parallel decoder structures. The decoder structure consists of a frequency domain enhanced self-attention module based on wavelet decomposition and Fourier transform, a residual connection and normalization layer, a fully connected layer and a Softmax layer.

[0023] Furthermore, step 3 includes the following steps:

[0024] Step 3.1: Input the subsequences in the training set processed in Step 1 into the multidimensional time series data anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism in Step 2, and obtain the outputs of the two first stages through the two parallel decoder structures respectively;

[0025] Step 3.2: Calculate the residual matrix using the output described in Step 3.1, append it to the input subsequence, and input it again into the encoder structure of the model. The output of the second stage training is obtained through the second decoder.

[0026] Step 3.3: Calculate the minimum cross-entropy loss function using the subsequences of output and input obtained in Steps 3.1 and 3.2, and update the model parameters;

[0027] Furthermore, step 4 includes the following steps:

[0028] Step 4.1: Input the subsequences from the test set processed in Step 1 into the model in sequence. After two stages, Step 3.1 and Step 3.2, obtain the three outputs corresponding to the input subsequences.

[0029] Step 4.2: Calculate the anomaly score using the output obtained in Step 4.1;

[0030] Step 4.3: Use a threshold selection algorithm combined with the anomaly score obtained in Step 4.2 to determine anomalies in the current input subsequence;

[0031] A computer device is characterized by comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism described in this invention.

[0032] A computer-readable storage medium is characterized in that it is used to store a computer program that executes the multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism described in this invention.

[0033] A multidimensional time-series data anomaly detection system based on adversarial training and frequency-domain improved self-attention mechanism is characterized in that the system comprises: a data acquisition module for acquiring and preprocessing the original dataset; a modeling module for constructing an anomaly detection model based on adversarial training and frequency-domain improved self-attention mechanism; a training module for training the anomaly detection model based on adversarial training and frequency-domain improved self-attention mechanism; and a judgment module for inputting the time-series data to be detected into the trained model for anomaly judgment.

[0034] Compared to existing technologies, this invention improves the performance and robustness of the generative model by introducing adversarial training. Secondly, this invention fully considers the importance of frequency domain information for time series data. Based on a simplified TransFormer encoder-decoder structure, it introduces a frequency-domain enhanced self-attention module based on discrete wavelet decomposition and Fourier transform, replacing the original self-attention algorithm that is not optimized for time series data. Compared to other models, the method designed in this invention exhibits better robustness and is applicable to various datasets, and can reduce the impact of noise on the model's anomaly detection performance to a certain extent.

[0035] The method described in this invention is applicable to anomaly detection in continuous multidimensional time-series data and has wide applications in many fields. Specifically, in the industrial sector, where sensor devices monitor the operating status of various systems such as industrial equipment and production lines in real time, this method can detect abnormal states and faults and problems that may affect the normal operation of the system based on the data monitored by the sensors. Specific application scenarios include, but are not limited to, the following:

[0036] Equipment fault detection: By real-time monitoring and analysis of the operating status of key components of industrial equipment, such as the vibration frequency of mechanical bearings of rotating shafts, abnormal conditions and faults and problems that may affect the normal operation of the equipment can be detected, and the equipment can be repaired and key parts replaced in advance.

[0037] Production line anomaly detection: By monitoring and analyzing the production line's operating status in real time, key nodes with abnormal states can be detected, and faults and problems that may affect production efficiency can be located in a timely manner. For example, monitoring parameters such as air pressure and hydraulic pressure on the production line can detect abnormal states such as pipe blockages or leaks. Alternatively, monitoring the position and movement of materials on the production line can detect abnormal states such as material blockages or material shortages.

[0038] In conclusion, industrial time-series data anomaly detection can be applied to various stages of industrial production, thereby helping enterprises improve production efficiency, reduce costs, improve product quality, and ensure worker safety. Attached Figure Description

[0039] Figure 1 A flowchart of a multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism;

[0040] Figure 2 A framework diagram of a multidimensional time-series data anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism;

[0041] Figure 3 A diagram of the encoder and decoder structure in the adversarial training architecture;

[0042] Figure 4 Here is a diagram of the wavelet decomposition module framework; Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figures 1 to 4 As shown, this invention provides a method for detecting anomalies in multidimensional time-series data based on adversarial training and a frequency-domain improved self-attention mechanism, specifically including the following steps:

[0045] Step 1: Obtain the raw dataset and preprocess it;

[0046] Step 1 specifically includes the following steps:

[0047] Step 1.1: Divide the original dataset into training and test sets. For datasets containing anomalous labels, this portion of the data should be used as part of the test set, while only the normal data without anomalous labels should be used for training. If the original dataset is unlabeled, it can be directly divided.

[0048] Step 1.2: Define a sliding window (containing continuous time series data at fixed time points). To ensure real-time anomaly detection, set the sliding step size to 1. The size of the sliding window is usually determined by factors such as the periodicity of the original dataset and the sampling frequency of the sensor in the application scenario. Different appropriate window sizes should be selected according to different application scenarios.

[0049] Step 1.3: Slide the sliding window defined in Step 1.2 across the original dataset and divide it into fixed-length subsequences. For example, if the sliding window size is 10 and the sliding step size is 1, the first subsequence will contain data from time points 1 to 10, the second subsequence will contain data from time points 2 to 11, and so on.

[0050] Step 1.4: Store the subsequences obtained in Step 1.3 in a set for model training;

[0051] Step 2: Construct an anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism;

[0052] Step 2 specifically includes the following steps:

[0053] Step 2.1: Construct a positional encoding layer after input, calculate the positional encoding information of the input subsequence, and append it to the input subsequence. The positional encoding calculation method is the trigonometric function method, and the calculation method is as follows:

[0054]

[0055] Two of them are location The first encoding vector , One portion, This is the dimension of the location vector. After calculating the location code for data at different locations, following the usual practice, it is directly added to the data at the corresponding time points of the original input subsequence;

[0056] Step 2.2: As Figure 2 As shown, the encoder structure is built after the position encoding layer in step 2.1. Figure 3 As shown, the encoder consists of a frequency-domain enhanced self-attention module based on wavelet decomposition and Fourier transform, residual links and normalization and fully connected neural networks cascaded together.

[0057] The frequency-domain enhanced self-attention module based on wavelet decomposition and Fourier transform in step 2.2 takes as input the original input subsequence with added positional coding information obtained in step 2.1 concatenated with a zero vector of the same size as the input:

[0058]

[0059] in It's an encoder. This represents an input subsequence with added positional encoding. This subsequence is first processed by a wavelet decomposition matrix. This invention uses Python's dwt library to implement the discrete wavelet method, and the parameters of the decomposition matrix can be set according to different wavelet implementations. After discrete wavelet decomposition, the sequence outputs one high-frequency component and one low-frequency component. For example... Figure 4 As shown, two Fourier transform modules are used to process the high-frequency and low-frequency components respectively. Since discrete wavelet decomposition is a multi-level recursive process, an original backup should be retained before the low-frequency component is processed by the Fourier transform module for use in the next level of discrete wavelet decomposition.

[0060] The Fourier transform submodule in step 2.2 comprises two processes: a random sampling process and an attention calculation process. Both processes are performed after the Fourier transform. The input to the Fourier transform submodule is the low-frequency and high-frequency components obtained from the wavelet decomposition in step 2.2; these two components are essentially sequence data. The calculation process of the Fourier transform submodule is as follows:

[0061]

[0062] in Represents the Fourier transform operation. , and This represents the frequency domain information retained after random sampling in the frequency domain. This represents the inverse Fourier transform. First, similar to a general self-attention mechanism, the low-frequency and high-frequency components obtained after wavelet decomposition in step 2.2 are transformed into three matrices, Q, K, and V, using three MLPs. Fourier transforms are then performed on these matrices to obtain the frequency domain representation of the input. The Fourier transform makes the signal characteristics more apparent in the frequency domain, while general signals exhibit sparsity in the frequency domain. That is, by retaining very few points in the frequency domain, the time-domain signal can be reconstructed almost without loss. The more points retained, the less information is lost, and vice versa.

[0063] Considering this, frequency domain information is randomly discarded here. The proportion can be adjusted according to the effect, but it usually does not exceed 1 / 2. This step can remove some noise and prevent model overfitting, thus enhancing the model's robustness. After processing the data in the frequency domain, in order to ensure that the inverse Fourier transform can reconstruct the original input length, zeros need to be padded at the uncollected frequency points, and then the inverse Fourier transform is used to project it back to the time domain. In addition, the attention calculation process is as follows:

[0064]

[0065] The residual connections and normalization layers in step 2.2 are designed to accelerate training and improve model performance. Residual connections can solve the vanishing gradient problem in deep neural networks and make the network easier to optimize. In this invention, each sub-layer except the last fully connected layer of the decoder has a residual connection, which adds the input directly to the sub-layer output, thereby allowing information to flow more effectively.

[0066] Specifically, let's set Indicates the input of the sublayer. If we represent the output of the sublayer, then the formula for calculating the residual connection is:

[0067]

[0068] Normalization is used to prevent numerical instability caused by an excessive number of network layers, and it also accelerates training. This invention employs a method called Layer Normalization for normalization, and its calculation formula is as follows:

[0069]

[0070] in, Represents the i-th feature. and These represent the mean and standard deviation, respectively. ϵ is a very small number used to avoid division by zero. and These are learnable parameters.

[0071] Step 2.3: As Figure 2 As shown, two parallel decoder structures are constructed after the encoder structure in step 2.2. Each decoder consists of a frequency-domain enhanced self-attention module based on wavelet decomposition and Fourier transform, residual connections and normalization layers, fully connected layers, and a softmax layer. The two parallel decoder structures are identical. Compared to the encoder, to make the model structure more flexible, an additional linear layer is introduced at the decoder end, providing richer expressive power for the decoder when generating target sequences.

[0072] Step 3: Train the anomaly detection model described in Step 2, which is based on adversarial training and frequency domain improved self-attention mechanism;

[0073] Step 3 specifically includes the following steps:

[0074] Step 3.1: Input the subsequences in the training set processed in Step 1 into the multidimensional time series data anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism described in Step 2, and obtain the outputs of the first stage through two decoders respectively;

[0075]

[0076] in, These represent the first decoder. and the second decoder The output of the first stage, It is the original input data with added positional encoding information, which is a concatenated 0 vector as described in step 2.1.

[0077] Step 3.2: Calculate the residual matrix using the partial output described in Step 3.1, append it to the input subsequence, and input it again into the encoder structure of the model. The output of the second stage training is obtained through the second decoder.

[0078]

[0079] in, This indicates the use of the method described in step 3.1. The process of calculating the residual matrix from the output involves concatenating it with the input subsequence containing additional positional information, replacing the zero vector concatenated from the input subsequence in step 3.1, and only through... Obtain the output of the second stage .

[0080] Step 3.3: Calculate the minimum cross-entropy loss function using the outputs obtained in Steps 3.1 and 3.2 along with the input subsequences, and update the model parameters. The specific loss function is as follows:

[0081] in: It is the number of training iterations. It is a training parameter that approaches 1. At the beginning of training, the reconstruction loss (the first part of the formula) has a higher weight. This is to ensure training stability when reconstruction performance is poor in the early stages. If the reconstruction stage performs poorly, the residual matrix used in the second stage will be even less reliable, so it cannot be used as a prior for the second stage reconstruction. As the reconstruction performance improves and the generated data becomes increasingly similar to the original data, the reliability of the residual matrix increases significantly, and the weight of the adversarial loss (the second part of the formula) increases accordingly. Since the loss curve in the training process of deep neural networks usually follows an exponential distribution, this invention uses the negative of the iteration number as... The exponent is reflected in the loss function.

[0082] Step 4: Input the time series data to be detected into the model trained in Step 3 to perform anomaly detection:

[0083] Step 4 specifically includes the following steps:

[0084] Step 4.1: Input the subsequences from the test set processed in Step 1 into the model sequentially. After two stages, Steps 3.1 and 3.2, obtain the three outputs corresponding to the input subsequences. and ;

[0085] Step 4.2: Calculate the anomaly score using the output obtained in Step 4.1:

[0086]

[0087] In the formula This represents the anomaly score for this input. The two terms before and after correspond to the loss function in step 3.3. After training, both decoders have good reconstruction performance, so their contributions in the two stages are considered equally. In the calculation of the anomaly score, their weights are assigned to 1 / 2 respectively.

[0088] Step 4.3: The current input subsequence is assessed for anomalies using a threshold selection algorithm combined with the anomaly score obtained in Step 4.2. This invention employs the POT threshold selection algorithm. If the anomaly score exceeds a specific value calculated by the algorithm, the input subsequence is classified as anomalous data; otherwise, it is classified as normal data.

[0089] In step 4.3, this invention uses the Python function library SPOT to implement the calculation process of the POT threshold selection algorithm. The detailed calculation steps are as follows:

[0090] Step 4.3.1: Run the fit() method in SPOT to import the outlier scores from the training set and the test set;

[0091] Step 4.3.2: Run the initialize() method in SPOT to initialize the POT method based on a given probability level value and some other parameters;

[0092] Step 4.3.3: After the initialization process described in Step 4.3.2, the `run()` method is run to execute the `POT` method and returns a dictionary containing performance metrics and prediction labels. Here, `thresholds` represents the calculated thresholds, and `alarms` represents anomaly markers.

[0093] Step 4.3.4: Average the values ​​in thresholds and multiply by a coefficient to obtain the final threshold;

[0094] In step 4, if the current input subsequence is determined to be abnormal, the online system will issue an alarm.

[0095] The above provides a detailed description of the multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism proposed in this invention. Specific examples are used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A multidimensional time-series data anomaly detection method based on adversarial training and frequency-domain improved self-attention mechanism, characterized in that, The method includes the following steps: Step 1: Obtain the raw dataset and preprocess it. The raw dataset is industrial time-series data, which is the operating status data of key components of industrial equipment or the operating status data of production lines. Step 2: Construct an anomaly detection model based on adversarial training and a frequency domain-modified self-attention mechanism. The method for constructing the anomaly detection model is as follows: Step 2.1: Construct a positional encoding layer, calculate the positional encoding information of the input subsequence, and append it to the input; Step 2.2: Construct the encoder structure, which consists of a frequency-domain enhanced self-attention module based on wavelet decomposition and Fourier transform, residual links, normalization and fully connected neural networks connected in series. The input to the self-attention module is the input subsequence with additional positional encoding obtained in step 2.1 concatenated with a zero vector of the same size as the input, denoted as: ,in It is an input subsequence with positional encoding added; The input subsequence with added position coding is decomposed by discrete wavelet decomposition to output high-frequency and low-frequency components; The high-frequency and low-frequency components are transformed into three matrices, Q, K and V, through three MLPs, and then subjected to Fourier transform operations to obtain the frequency domain representation of the input. In the frequency domain, the data after Fourier transform is randomly discarded, and zeros are padded at the frequency points that were not collected. The zero-padded frequency points are then projected back into the time domain representation through inverse Fourier transform. Step 2.3: Construct two parallel decoder structures. The decoder structure consists of a frequency domain enhanced self-attention module based on wavelet decomposition and Fourier transform, a residual connection and normalization layer, a fully connected layer and a Softmax layer. Step 3: Train the anomaly detection model described in Step 2, which is based on adversarial training and frequency domain improved self-attention mechanism; Step 4: Input the time series data to be detected into the anomaly detection model trained in Step 3 to determine the anomaly.

2. The multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1: Divide the original dataset into a training set and a test set; Step 1.2: Define a sliding window with a sliding step size of 1, and set the window size according to the original dataset; Step 1.3: Slide the sliding window over the original dataset, dividing the data within each sliding window into a fixed-length subsequence; Step 1.4: Store the subsequences obtained in Step 1.3 in a set for use as input for model training.

3. The multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism according to claim 2, characterized in that, The sliding window contains continuous time series data with a fixed length of time points.

4. The multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism according to claim 2, characterized in that, The sliding step size is the distance the sliding window moves on the original dataset each time.

5. The method according to claim 3, characterized in that, Step 3 includes the following steps: Step 3.1: Input the subsequences from the training set processed in Step 1 into the anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism in Step 2, and obtain the outputs of the two first stages through the two parallel decoder structures respectively; Step 3.2: Calculate the residual matrix using the output described in Step 3.1, append it to the input subsequence, and input it again into the encoder structure of the anomaly detection model. The output of the second stage training is obtained through the second decoder. Step 3.3: Calculate the minimum cross-entropy loss function using the subsequences of output and input obtained in Steps 3.1 and 3.2, and update the anomaly detection model parameters.

6. The method according to claim 4, characterized in that, Step 4 includes the following steps: Step 4.1: Input the subsequences from the test set processed in Step 1 into the anomaly detection model in sequence. After two stages, Step 3.1 and Step 3.2, obtain the three outputs corresponding to the input subsequences. Step 4.2: Calculate the anomaly score using the output obtained in Step 4.1; Step 4.3: Use the threshold selection algorithm combined with the anomaly score obtained in Step 4.2 to determine the anomaly of the current input subsequence.

7. A computer device, characterized in that, The method includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that executes the multidimensional time-series data anomaly detection method based on adversarial training and frequency domain improved self-attention mechanism as described in any one of claims 1-6.

9. A multidimensional time-series data anomaly detection system based on adversarial training and frequency-domain improved self-attention mechanism, characterized in that, The system includes: A data acquisition module for acquiring and preprocessing raw datasets, wherein the raw datasets are industrial time-series data; A modeling module for building anomaly detection models based on adversarial training and frequency domain improved self-attention mechanisms; A training module for training the anomaly detection model based on adversarial training and frequency domain improved self-attention mechanism, the training module performing the following steps: Step 2.1: Construct a positional encoding layer, calculate the positional encoding information of the input subsequence, and append it to the input; Step 2.2: Construct the encoder structure, which consists of a frequency-domain enhanced self-attention module based on wavelet decomposition and Fourier transform, residual links, normalization and fully connected neural networks connected in series. The input to the self-attention module is the input subsequence with additional positional encoding obtained in step 2.1 concatenated with a zero vector of the same size as the input, denoted as: ,in It is an input subsequence with positional encoding added; The input subsequence with added position coding is decomposed by discrete wavelet decomposition to output high-frequency and low-frequency components; The high-frequency and low-frequency components are transformed into three matrices, Q, K and V, through three MLPs, and then subjected to Fourier transform operations to obtain the frequency domain representation of the input. In the frequency domain, the data after Fourier transform is randomly discarded, and zeros are padded at the frequency points that were not collected. The zero-padded frequency points are then projected back into the time domain representation through inverse Fourier transform. Step 2.3: Construct two parallel decoder structures. The decoder structure consists of a frequency domain enhanced self-attention module based on wavelet decomposition and Fourier transform, a residual connection and normalization layer, a fully connected layer and a Softmax layer. A judgment module used to input the time series data to be detected into the trained anomaly detection model for anomaly determination.

Images