Time series data anomaly detection method based on data enhancement and double attention contrast learning

Abstract

The invention discloses a time series data anomaly detection method based on data enhancement and double attention contrast learning, and belongs to the technical field of data anomaly detection and deep learning. The problem of poor anomaly detection performance caused by poor generalization ability of a detection model obtained by adopting an existing method is solved. According to the method, the original time series data are input into the enhancement network to learn and obtain the new view of the time series data, the new view after enhancement is beneficial to increase the distinction degree between abnormal points and normal points, and the accuracy of anomaly detection is improved. And inputting the enhanced time sequence data into the double attention contrast learning network by using the prior that the association between the abnormal point and other points is weak, thereby obtaining an anomaly detection result. The dual attention contrast learning network not only can detect data of known abnormal types, but also can detect abnormal data of unknown types, so that the performance of model anomaly detection is improved. The method can be applied to time series data anomaly detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of data anomaly detection and deep learning, and specifically relates to a time series data anomaly detection method based on data enhancement and dual attention contrastive learning. Background Art

[0002] Time series data anomaly detection is a technology with important applications in industry, including industrial equipment status monitoring, fault diagnosis, server load monitoring and many other fields. The rapid development of industrial sensors has made it possible to collect massive time series data generated when the system is running. Reusing the data to train relevant models can effectively discover abnormal patterns in the system's time series data, which is of great significance for ensuring safety and avoiding economic losses. For example, timely detection of anomalies in the time series data of the motor's operating status helps to carry out timely maintenance and troubleshooting.

[0003] However, it is difficult to discover abnormal patterns from large amounts of complex time series data, and abnormal labels are usually difficult to obtain. Most existing supervised or semi-supervised methods cannot guarantee the generalization ability of the detection model given limited labeled training data, and the anomaly detection performance of the models obtained using existing methods is still poor. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the detection model obtained by the existing method has poor generalization ability, resulting in poor performance of anomaly detection, and propose a time series data anomaly detection method based on data enhancement and dual attention contrast learning.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] A time series data anomaly detection method based on data enhancement and dual attention contrastive learning, the method specifically comprises the following steps:

[0007] Step 1: Use sensors to collect time series data x'∈R when the motor is running T×F , where T is the length of the time series, F is the data dimension of each time point, and R is a real number;

[0008] Step 2: Input the time series data x' into the decomposition network h, and calculate the binary mask vector m based on the output of the decomposition network h h , and the binary mask vector m h The length is T, and then the binary mask vector m is used h Calculate the information part x' in the time series data x' * ;

[0009] Step 3: Input the time series data x' into the enhanced network g, and calculate the mask vector m based on the output of the enhanced network g g ;

[0010] The information part x′ * With the mask vector m g Perform matrix dot multiplication to obtain the enhanced information part v * , and give the enhanced information part v * Add random noise to get the data enhancement result v, v∈R T×F ;

[0011] Step 4: Use the data enhancement result v as the input of the dual attention contrast learning network, output the abnormal score Score of the data at each time point in the time series data x' through the dual attention contrast learning network, and compare the abnormal score Score corresponding to the data at each time point with the score threshold δ;

[0012] If there is time point data with an abnormal score Score greater than the score threshold δ in the time series data, then the time point data with an abnormal score Score greater than the score threshold δ is abnormal data;

[0013] If the abnormal scores Score of all time point data in the time series data are less than or equal to the score threshold δ, there is no abnormal data in the time series data x'.

[0014] Furthermore, the value of the data dimension F at each time point is 4, specifically including motor operating current, motor operating temperature, motor outlet air volume and motor vibration amplitude data.

[0015] Furthermore, the decomposition network h includes a CNN convolutional neural network model and a projection layer.

[0016] Furthermore, the specific process of step 2 is as follows:

[0017] Step 21: Take the time series data x' as the input of the decomposition network h. In the decomposition network h, the time series data x' passes through the CNN convolutional neural network model and the projection layer in sequence, and the feature vector output by the projection layer is recorded as π;

[0018] Step 22: Process the feature vector π to obtain the mask vector

[0019]

[0020] Among them, π i is the i-th element in the eigenvector π, ∈ is a random variable that obeys a uniform distribution between (0,1), σ is the sigmoid function, τ is the temperature parameter, is a vector The i-th element in ;

[0021] Step 23: Mask vector Do stretching to get the binary mask vector m h :

[0022]

[0023] Among them, ζ<0, γ>1, is the binary mask vector m h The i-th element in ;

[0024] Step 24: Information part x′ in time series data x′ * for:

[0025] x′ * =m h ⊙x′

[0026] Where ⊙ represents matrix dot product.

[0027] Furthermore, in step 3, the information part x′ * With the mask vector m g Perform matrix dot multiplication to obtain the enhanced information part v * , specifically:

[0028] v * =m g ⊙x * .

[0029] Furthermore, the dual attention contrastive learning network includes a first attention module, a second attention module and a dual contrastive learning module.

[0030] Furthermore, the working process of the dual attention contrast learning network is:

[0031] Step 41: Segment the data enhancement result v into N segments, where the time series length of each segment is P, and P×N=T;

[0032] Step 42: Input all the fragments into the first attention module. In the first attention module, all the input fragments are first encoded by the embedding layer, and each fragment is modeled as a feature of length d. Then, the features of all the fragments are used to form a feature matrix v N ∈R N×d ;

[0033] Feature matrix v N Then, through the multi-head attention mechanism, the attention weight Attn of the relationship between the fragments is calculated N ;

[0034] Step 43: Input all the fragments into the second attention module. In the second attention module, all the input fragments are first encoded by the embedding layer of the second attention module, that is, the data of the first time point in all the fragments is modeled as a feature of length d, and the data of the second time point in all the fragments is modeled as a feature of length d, and so on, until the data of the Pth time point in all the fragments is modeled as a feature of length d, and then the features obtained by modeling are respectively used as feature matrices v P A row of P ∈R P×d ;

[0035] Feature matrix v P Then, through the multi-head attention mechanism, the attention weight Attn of the relationship within the clip is calculated P ;

[0036] Step 4. Attention weight Attn N and Attn P Upsampling is performed separately to obtain the upsampling result (Attn N ) and upsampling(Attn P );

[0037] Step 45: After adjusting the length of the segment processed in step 41, repeat the process from step 42 to step 44 to obtain a new set of upsampling results;

[0038] Continuously adjust the length of the fragments processed in step 4.1 until M groups of upsampling results are obtained;

[0039] Step 46: Send the up-sampled results of the M groups of inter-fragment relationship attention weights and the up-sampled results of the intra-fragment relationship attention weights to the double contrast learning module, and connect the up-sampled results of the M groups of inter-fragment relationship attention weights to obtain the connection result Connect the up-sampled results of the attention weights of the M groups of fragments to get the connection result

[0040] Step 47: Calculate the abnormal score Score of each time point in the time series data x':

[0041]

[0042] Here, KL(·,·) represents the KL divergence.

[0043] Furthermore, the feature matrix v NThen, through the multi-head attention mechanism, the attention weight Attn of the relationship between the fragments is calculated N ; Specifically:

[0044] make H is the total number of attention heads, then and Represent the feature matrix v N The query and key output by the i-th attention head, Represents the feature matrix v N The value output by the i-th attention head, represent The parameter matrix of represent The parameter matrix of

[0045] Then the attention weight of the i-th attention head is The superscript T stands for transpose;

[0046] Connect the attention weights of each attention head to get the attention weight Attn of the relationship between fragments N :

[0047]

[0048] in, is a parameter matrix, Concat represents a concatenation operation;

[0049] The feature matrix v P Then, through the multi-head attention mechanism, the attention weight Attn of the relationship within the clip is calculated P ; Specifically:

[0050] make but and Represent the feature matrix v P The query and key output by the i-th attention head, represent The parameter matrix of represent The parameter matrix of Represents the feature matrix v P The value output by the i-th attention head;

[0051] Then the attention weight of the i-th attention head is

[0052] Connect the attention weights of each attention head to get the attention weight Attn of the relationship within the fragment P :

[0053]

[0054] in, is the parameter matrix.

[0055] Furthermore, during the training process of the decomposition network h, the number of time series data samples in each batch training data is B , the loss function L used in training is:

[0056]

[0057] in, π t is the tth element in the feature vector π, which represents the feature vector output by the projection layer after the sample x passes through the decomposition network h; π* represents x * After the decomposition network h, the feature vector output by the projection layer, x * is the information part in sample x; MMD(π,π*) represents the maximum mean difference between π and π*; represents the mask vector corresponding to x, Represents the mask vector The ath element in Represents the mask vector The pth element in Represents the mask vector The nth element in , γ<0, ζ>1, λ and β are weights.

[0058] Furthermore, the loss function L' used in the training process of the dual attention contrastive learning network is:

[0059]

[0060] Among them, the Stopgrad operation represents stochastic gradient descent, It represents the connection result after upsampling of the attention weights of the M groups of inter-fragment relations corresponding to the sample x. Represents the connection result of the M groups of intra-fragment relation attention weights corresponding to sample x after upsampling.

[0061] The beneficial effects of the present invention are:

[0062] The present invention inputs the original time series data into the enhanced network to learn and obtain a new view of the time series data. The enhanced new view helps to increase the distinction between abnormal points and normal points and improve the accuracy of anomaly detection. Using the prior that the connection between abnormal points and other points is weak, the enhanced time series data is input into the dual attention contrast learning network to obtain anomaly detection results. The dual attention contrast learning network of the present invention can not only detect data of known abnormal types, but also detect abnormal data of unknown types, which solves the problem of poor generalization ability of the detection model due to the difficulty in obtaining abnormal data labels, improves the generalization ability of the model, and improves the performance of model anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of a time series data anomaly detection method based on data enhancement and dual attention contrast learning of the present invention. DETAILED DESCRIPTION

[0064] Specific implementation method 1: Combination Figure 1 The present embodiment describes a method for detecting anomalies in time series data based on data enhancement and dual attention contrastive learning, and the method specifically comprises the following steps:

[0065] Step 1: Use sensors to collect time series data x'∈R when the motor is running T×F , where T is the length of the time series, that is, the number of time points corresponding to the time series data x', F is the data dimension of each time point, and R is a real number;

[0066] Step 2: Input the time series data x' into the decomposition network h, and calculate the binary mask vector m based on the output of the decomposition network h h (i.e., vector m h The value of the elements in is 0 or 1), and the binary mask vector m h The length is T, and then the binary mask vector m is used h Calculate the information part x' in the time series data x' * ;

[0067] Step 3: Input the time series data x' into the enhanced network g, and calculate the mask vector m based on the output of the enhanced network g g ;

[0068] The processing is the same as that of the decomposition network h, except that the enhanced network g does not perform mask vector m g Perform binary processing;

[0069] The information part x′ * With the mask vector m gPerform matrix dot multiplication to obtain the enhanced information part v * , and give the enhanced information part v * Add random noise to get the data enhancement result b, b∈R T×F ;

[0070] The method of adding noise is to randomly mask the first hidden layer of the enhanced network g along the time dimension and then * After the processing, the enhanced network g outputs the data enhancement result v;

[0071] Step 4: Use the data enhancement result v as the input of the dual attention contrast learning network, output the abnormal score Score of the data at each time point in the time series data x' through the dual attention contrast learning network, and compare the abnormal score Score corresponding to the data at each time point with the score threshold δ;

[0072] If there is time point data in the time series data where the abnormal score Score is greater than the score threshold δ, then the time point data where the abnormal score Score is greater than the score threshold δ is abnormal data;

[0073] If the abnormal scores Score of all time point data in the time series data are less than or equal to the score threshold δ, there is no abnormal data in the time series data x'.

[0074] The method of the present invention can be applied to anomaly detection of time series data in power systems.

[0075] Specific implementation method 2: This implementation method is different from specific implementation method 1 in that the data dimension F at each time point has a value of 4, specifically including motor operating current, motor operating temperature, motor outlet air volume and motor vibration amplitude data.

[0076] The other steps and parameters are the same as those in the first embodiment.

[0077] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the decomposition network h includes a CNN convolutional neural network model and a projection layer.

[0078] The other steps and parameters are the same as those in the first or second embodiment.

[0079] Specific implementation method 4: This implementation method is different from the specific implementation methods 1 to 3 in that the specific process of step 2 is as follows:

[0080] Step 21: Take the time series data x' as the input of the decomposition network h. In the decomposition network h, the time series data x' passes through the CNN convolutional neural network model and the projection layer in sequence, and the feature vector output by the projection layer is recorded as π;

[0081] Step 22: Process the feature vector π to obtain the mask vector

[0082]

[0083] Among them, π i is the i-th element in the eigenvector π, ∈ is a random variable that obeys a uniform distribution between (0,1), σ is the sigmoid function, τ is the temperature parameter, is a vector The i-th element in ;

[0084] Step 23: Mask vector Do stretching to get the binary mask vector m h :

[0085]

[0086] Among them, ζ<0, γ>1, is the binary mask vector m h The i-th element in ;

[0087] Step 24: Information part x′ in time series data x′ * for:

[0088] x′ * =m h ⊙x′

[0089] Where ⊙ represents matrix dot product.

[0090] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0091] Specific implementation method 5: This implementation method is different from any one of the specific implementation methods 1 to 4 in that in step 3, the information part x′ * With the mask vector m g Perform matrix dot multiplication to obtain the enhanced information part v * , specifically:

[0092] v * =m g ⊙x *

[0093] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0094] Specific implementation method six: This implementation method is different from any one of specific implementation methods one to five in that the dual attention contrast learning network includes a first attention module, a second attention module and a dual contrast learning module.

[0095] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0096] Specific implementation method 7: This implementation method is different from any one of the specific implementation methods 1 to 6 in that the working process of the dual attention contrast learning network is as follows:

[0097] Step 41: Segment the data enhancement result v into N segments, where the time series length of each segment is P, and P×N=T;

[0098] Step 42: Input all the fragments into the first attention module. In the first attention module, all the input fragments are first encoded by the embedding layer, and each fragment is modeled as a feature of length d. Then, the features of all the fragments are used to form a feature matrix v N ∈R N×d (That is, the features of each fragment are taken as the matrix v N a line of );

[0099] Feature matrix v N Then, through the multi-head attention mechanism, the attention weight Attn of the relationship between the fragments is calculated N ;

[0100] Step 43: Input all the fragments into the second attention module. In the second attention module, all the input fragments are first encoded by the embedding layer of the second attention module, that is, the data of the first time point in all the fragments is modeled as a feature of length d, and the data of the second time point in all the fragments is modeled as a feature of length d, and so on, until the data of the Pth time point in all the fragments is modeled as a feature of length d, and then the features obtained by modeling are respectively used as feature matrices v P A row of P ∈R P×d ;

[0101] Feature matrix v P Then, through the multi-head attention mechanism, the attention weight Attn of the relationship within the clip is calculated P ;

[0102] Step 4. Attention weight Attn N and Attn P Upsampling is performed separately to obtain the upsampling result (Attn N ) and upsampling(Attn P );

[0103] Step 45: After adjusting the length of the segment processed in step 41, repeat the process from step 42 to step 44 to obtain a new set of upsampling results;

[0104] Continuously adjust the length of the fragments processed in step 4.1 until M groups of upsampling results are obtained;

[0105] Step 46: Send the up-sampled results of the M groups of inter-fragment relationship attention weights and the up-sampled results of the intra-fragment relationship attention weights to the double contrast learning module, and connect the up-sampled results of the M groups of inter-fragment relationship attention weights to obtain the connection result Connect the up-sampled results of the attention weights of the M groups of fragments to get the connection result

[0106] Step 47: Calculate the abnormal score Score of each time point in the time series data x':

[0107]

[0108] Here, KL(·,·) represents the KL divergence.

[0109] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0110] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the characteristic matrix v N Then, through the multi-head attention mechanism, the attention weight Attn of the relationship between the fragments is calculated N ; Specifically:

[0111] make H is the total number of attention heads, then and Represent the feature matrix v N The query and key output by the i-th attention head, Represents the feature matrix v N The value output by the i-th attention head (value), represent The learnable parameter matrix, represent The learnable parameter matrix of

[0112] Then the attention weight of the i-th attention head is The superscript T stands for transpose;

[0113] Connect the attention weights of each attention head to get the attention weight Attn of the relationship between fragments N :

[0114]

[0115] in, is a learnable parameter matrix, Concat represents a concatenation operation;

[0116] The feature matrix v P Then, through the multi-head attention mechanism, the attention weight Attn of the relationship within the clip is calculated P ; Specifically:

[0117] make but and Represent the feature matrix v P The query and key output by the i-th attention head, represent The learnable parameter matrix, represent The learnable parameter matrix of Represents the feature matrix v P The value output by the i-th attention module (value); and the parameter matrix of the second attention module Same as the parameter matrix of the first attention module;

[0118] Then the attention weight of the i-th attention head is

[0119] Connect the attention weights of each attention head to get the attention weight Attn of the relationship within the fragment P :

[0120]

[0121] in, is the learnable parameter matrix.

[0122] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0123] Specific implementation method 9: This implementation method is different from any one of the specific implementation methods 1 to 8 in that during the training process of the decomposition network h, the number of time series data samples in each batch training data is B , the loss function L used in training is:

[0124]

[0125] in, π tis the tth element in the feature vector π, which represents the feature vector output by the projection layer after the sample x passes through the decomposition network h; π* represents x * After the decomposition network h, the feature vector output by the projection layer, x * is the information part in sample x; MMD(π,π*) represents the maximum mean difference (MMD) between π and π*; represents the mask vector corresponding to x, Represents the mask vector The ath element in Represents the mask vector The pth element in Represents the mask vector The nth element in , γ<0, ζ>1, λ and β are weights (used to balance the fidelity and diversity of data enhancement, and the specific values ​​are manually adjusted during the actual training process).

[0126] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0127] || h The ||0 term minimizes the number of 0 elements in the mask vector. is a regular term, for the mask vector Randomly select a time point a as the anchor point. Then randomly select a time point p from the left or right of a and create a positive pair (a,p); randomly select another time point n from the left or right of a, and the distance between n and a is greater than the distance between p and a, then the mask value and Should be closer. and Should be farther apart.

[0128] Specific implementation method 10: This implementation method is different from any one of specific implementation methods 1 to 9 in that the loss function L' used in the training process of the dual attention contrast learning network is:

[0129]

[0130] Among them, the Stopgrad operation represents stochastic gradient descent, It represents the connection result after upsampling of the attention weights of the M groups of inter-fragment relations corresponding to the sample x. Represents the connection result of the M groups of intra-fragment relation attention weights corresponding to sample x after upsampling.

[0131] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0132] The present invention uses KL divergence to measure different features The similarity between them is used as the loss function of the entire network, and the Stopgrad operation is used to avoid model collapse.

[0133] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A time series data anomaly detection method based on data enhancement and dual attention contrastive learning, characterized in that: The method specifically comprises the following steps: Step 1: Use sensors to collect time series data x′∈R when the motor is running T×F , where T is the length of the time series, F is the data dimension of each time point, and R is a real number; Step 2: Input the time series data x′ into the decomposition network h, and calculate the binary mask vector m based on the output of the decomposition network h h , and the binary mask vector m h The length is T, and then the binary mask vector m is used h Calculate the information part x' in the time series data x' * ; Step 3: Input the time series data x′ into the enhanced network g, and calculate the mask vector mg based on the output of the enhanced network g g ; The information part x′ * With the mask vector m g Perform matrix dot multiplication to obtain the enhanced information part v * , and give the enhanced information part v * Add random noise to get the data enhancement result v, v∈R T×F ; Step 4: Use the data enhancement result v as the input of the dual attention contrast learning network, output the abnormal score Score of the data at each time point in the time series data x′ through the dual attention contrast learning network, and compare the abnormal score Score corresponding to the data at each time point with the score threshold δ; If there is time point data in the time series data where the abnormal score Score is greater than the score threshold δ, then the time point data where the abnormal score Score is greater than the score threshold δ is abnormal data; If the abnormal scores Score of all time point data in the time series data are less than or equal to the score threshold δ, there is no abnormal data in the time series data x′.

2. The time series data anomaly detection method based on data enhancement and dual attention contrastive learning according to claim 1 is characterized in that: The value of the data dimension F at each time point is 4, which specifically includes the motor operating current, motor operating temperature, motor outlet air volume and motor vibration amplitude data.

3. The time series data anomaly detection method based on data enhancement and dual attention contrastive learning according to claim 2 is characterized in that: The decomposition network h includes a CNN convolutional neural network model and a projection layer.

4. The time series data anomaly detection method based on data enhancement and dual attention contrastive learning according to claim 3 is characterized in that: The specific process of step 2 is as follows: Step 21: Take the time series data x′ as the input of the decomposition network h. In the decomposition network h, the time series data x′ passes through the CNN convolutional neural network model and the projection layer in sequence, and the feature vector output by the projection layer is recorded as π; Step 22: Process the feature vector π to obtain the mask vector Among them, π i is the i-th element in the eigenvector π, ∈ is a random variable that obeys a uniform distribution between (0, 1), σ is the sigmoid function, τ is the temperature parameter, is a vector The i-th element in ; Step 23: Mask vector Do stretching to get the binary mask vector m h : Among them, ζ<0, γ>1, is the binary mask vector m h The i-th element in ; Step 24: Information part x′ in time series data x′ * for: x′ * =m h ⊙x′ Where ⊙ represents matrix dot product.

5. The time series data anomaly detection method based on data enhancement and dual attention contrastive learning according to claim 4 is characterized in that: In step 3, the information part x′ * With the mask vector m g Perform matrix dot multiplication to obtain the enhanced information part v * , specifically: in * =m g ⊙x * 。 6. The time series data anomaly detection method based on data enhancement and dual attention contrastive learning according to claim 5, characterized in that: The dual attention contrastive learning network includes a first attention module, a second attention module and a dual contrastive learning module.

7. The time series data anomaly detection method based on data enhancement and dual attention contrastive learning according to claim 6 is characterized in that: The working process of the dual attention contrastive learning network is: Step 41: Segment the data enhancement result v into N segments, where the time series length of each segment is P, and P×N=T; Step 42: Input all the fragments into the first attention module. In the first attention module, all the input fragments are first encoded by the embedding layer, and each fragment is modeled as a feature of length d. Then, the features of all the fragments are used to form a feature matrix v N ∈R N×d ; Feature matrix v N Then, through the multi-head attention mechanism, the attention weight Attn of the relationship between the fragments is calculated N ; Step 43: Input all the fragments into the second attention module. In the second attention module, all the input fragments are first encoded by the embedding layer of the second attention module, that is, the data of the first time point in all the fragments is modeled as a feature of length d, and the data of the second time point in all the fragments is modeled as a feature of length d, and so on, until the data of the Pth time point in all the fragments is modeled as a feature of length d, and then the features obtained by modeling are respectively used as feature matrices v P A row of P ∈R P×d ; Feature matrix v P Then, through the multi-head attention mechanism, the attention weight Attn of the relationship within the clip is calculated P ; Step 4. Attention weight Attn N and Attn P Upsampling is performed separately to obtain the upsampling result (Attn N ) and upsampling(Attn P ); Step 45: After adjusting the length of the segment processed in step 41, repeat the process from step 42 to step 44 to obtain a new set of upsampling results; Continuously adjust the length of the fragments processed in step 4.1 until M groups of upsampling results are obtained; Step 46: Send the up-sampled results of the M groups of inter-fragment relationship attention weights and the up-sampled results of the intra-fragment relationship attention weights to the double contrast learning module, and connect the up-sampled results of the M groups of inter-fragment relationship attention weights to obtain the connection result Connect the up-sampled results of the attention weights of the M groups of fragments to get the connection result Step 47: Calculate the abnormal score Score of each time point in the time series data x′: Here, KL(·,·) represents the KL divergence.

8. The method for detecting anomalies in time series data based on data enhancement and dual attention contrastive learning according to claim 7, characterized in that: The feature matrix v N Then, through the multi-head attention mechanism, the attention weight Attn of the relationship between the fragments is calculated N ; Specifically: make H is the total number of attention heads, then and Represent the feature matrix v N The query and key output by the i-th attention head, Represents the feature matrix v N The value output by the i-th attention head, represent The parameter matrix of represent The parameter matrix of Then the attention weight of the i-th attention head is The superscript T stands for transpose; Connect the attention weights of each attention head to get the attention weight Attn of the relationship between fragments N : in, is a parameter matrix, Concat represents a concatenation operation; The feature matrix v P Then, through the multi-head attention mechanism, the attention weight Attn of the relationship within the clip is calculated P ; Specifically: make but and Represent the feature matrix v P The query and key output by the i-th attention head, represent The parameter matrix of represent The parameter matrix of Represents the feature matrix v P The value output by the i-th attention head; Then the attention weight of the i-th attention head is Connect the attention weights of each attention head to get the attention weight Attn of the relationship within the fragment P : in, is the parameter matrix.

9. The method for detecting anomalies in time series data based on data enhancement and dual attention contrastive learning according to claim 8, characterized in that: During the training process of the decomposition network h, the number of time series data samples in each batch training data is X B , the loss function L used in training is: in, π t is the tth element in the feature vector π, which represents the feature vector output by the projection layer after the sample x passes through the decomposition network h; π* represents x * After the decomposition network h, the feature vector output by the projection layer, x * is the information part in sample x; MMD(π,π*) represents the maximum mean difference between π and π*; represents the mask vector corresponding to x, Represents the mask vector The ath element in Represents the mask vector The pth element in Represents the mask vector The nth element in , γ<0, ζ>1, λ and β are weights.

10. The method for detecting anomalies in time series data based on data enhancement and dual attention contrastive learning according to claim 9, characterized in that: The loss function L′ used in the training process of the dual attention contrastive learning network is: Among them, the Stopgrad operation represents stochastic gradient descent, It represents the connection result after upsampling of the attention weights of the M groups of inter-fragment relations corresponding to the sample x. Represents the connection result of the M groups of intra-fragment relation attention weights corresponding to sample x after upsampling.

Images