A time series anomaly detection method based on feature distribution similarity

Abstract

The application discloses a time series anomaly detection method based on feature distribution similarity, and belongs to the technical field of time series, comprising data preprocessing and segmentation; the processed time series is taken as input data, is input to a bidirectional GRU reconstruction module and a multi-scale convolution sparse attention module in parallel after context embedding and position embedding, feature extraction is carried out, local features and global features are obtained respectively; the obtained local features and global features are respectively converted into probability distribution and the similarity of corresponding probability distribution through a normalized exponential function and a symmetric phase KL divergence; the divergence of the local features is reduced and the divergence of the global features is increased by using a contrast loss function; the anomaly detection task of the LGS-UAD model on the entire data set is completed, and the final detection result is obtained. The time series anomaly detection method based on feature distribution similarity can meet the demand of detecting anomalies from a large amount of data streams in real environment in time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of time series, and in particular to a time series anomaly detection method based on feature distribution similarity. BACKGROUND

[0002] Time series refers to a numerical sequence obtained by arranging numerical values of the same statistical index in chronological order (for example, the network throughput of a certain server every hour, the access times of a certain website every minute, etc.). Time series anomaly detection refers to detecting whether a certain numerical value or a certain numerical value subsequence in a time series deviates from the normal mode. The existing methods for detecting whether a time series is abnormal and their defects include:

[0003] (1) The method based on density estimation considers that abnormal points will be located in low-density areas, and LOF detects the local density deviation of data points, while considering the local and global attributes of the data set. COF does not use the Euclidean distance but the chain distance to calculate the outlier score. The SOS algorithm calculates the global correlation degree by using the heterogeneity matrix and the correlation matrix to evaluate the abnormal probability. These methods consume a lot of in processing high-dimensional data, and the DAGMM method uses an automatic encoder to convert data to a hidden space to evaluate the abnormality. This assumption regards the input sequence as unstructured information, ignoring the time domain dependency therein.

[0004] (2) The cluster-based method uses single classification to solve the anomaly detection problem, extracts a feature to concentrate normal points around the cluster, and at this time, points falling outside the area will be regarded as abnormal. The SVDD method maps data to zero-one space through a kernel function, and there is a hypersphere in the space that wraps most of the normal points, and the Lagrange method is used to calculate the sphere center. DeepSVDD uses an automatic encoder to replace the kernel function, and the loss function optimizes the network parameters and the hypersphere radius at the same time. The Deep SAD method uses a small amount of labels that may be obtained in practice to assist optimization. Similarly, this assumption also regards the input sequence as unstructured information, ignoring the time domain dependency therein.

[0005] (3) The reconstruction-based model learns the implicit generation rule of data from the posterior, and uses reconstruction error to evaluate the anomaly. LSTM-VAE uses LSTM to extract time domain features, VAE regenerates new data conforming to the normal mode, and points with larger reconstruction error are determined to be abnormal. OmniAnomaly uses a parametric GRU instead of LSTM, and uses a linear Gaussian SSM for the VAE hidden layer variable to make it have time domain dependency, and uses a planar normalization flow to model the non-Gaussian part in the distribution. Donut modifies the original ELBO to deal with the influence of missing values in the input on the learning effect, and adjusts the learning weight according to the proportion of normal values in the window. This method needs to be additionally designed to exclude the interference of unlabeled abnormal data in the original data on the learning of the normal distribution by the model.

[0006] (4) The method based on contrast structure assumes that the features of normal patterns are relatively similar at different perspectives, while the features of abnormal points will be quite different at different perspectives. Therefore, the contrast structure is used to extract the original different feature expressions and evaluate the abnormality by comparing the differences. The prior correlation branch in AnomalyTransformer adjusts the scale parameters of the Gaussian kernel to adapt to the normal pattern, and the sequence association branch uses the attention mechanism to focus on non-adjacent areas. The symmetric KL divergence is used to compare the differences between the two branches to detect abnormalities. DCdetector adopts the idea of ​​PatchTST. Compared with point-wise, the features of time series can be described more accurately by patch-wise. The input is divided into two branches, patch-wise and in-patch, and multi-head attention is used to extract feature representations at different perspectives. However, since the contrast branch structure requires dual models to extract features, the existing methods are slow and consume huge amounts of video memory, making it difficult to meet the needs of timely detecting abnormalities from large amounts of data streams in real environments. Summary of the Invention

[0007] The purpose of the present invention is to provide a time series anomaly detection method based on feature distribution similarity to solve the problems existing in the above background technology.

[0008] To achieve the above object, the present invention provides a time series anomaly detection method based on feature distribution similarity, comprising the following steps:

[0009] S1. Data preprocessing and segmentation to obtain training data set, test data set and validation data set;

[0010] S2. Based on the training data set obtained in step S1, 64 sets of training data are randomly selected each time for training;

[0011] S3, the time series processed in step S1 is used as input data, and then input into the bidirectional GRU reconstruction module and the multi-scale convolutional sparse attention module in parallel for feature extraction after context embedding and position embedding, to obtain local features and global features respectively;

[0012] S4. Convert the local features and global features obtained in step S3 into probability distributions respectively through a normalized exponential function, use the symmetrical phase KL divergence to evaluate the similarity of the probability distributions corresponding to the local features and the global features, and normalize them within the detection window to obtain the similarity evaluation value at each time point;

[0013] S5. Using the contrast loss function, the divergence of local features is reduced and the divergence of global features is amplified. This learns the time domain characteristics of the normal distribution and highlights the abnormal points, completing the training of the LGS-UAD model.

[0014] S6. Based on the validation data set obtained in step S1, all group data are input into the trained LGS-UAD model, and the symmetric phase KL divergence output by the LGS-UAD model is evaluated to determine whether the LGS-UAD model is continuing to optimize;

[0015] S7, repeat steps S2-S6 until the obtained evaluation index no longer improves. At this time, the network parameters are updated and the LGS-UAD model training ends;

[0016] S8. Based on the test data set obtained in step S1, all group data are input into the trained LGS-UAD model. The trained LGS-UAD model first calculates the anomaly score threshold on the training data set, and then evaluates the test data set according to the anomaly score threshold to obtain the anomaly detection result of the LGS-UAD model on the group data;

[0017] S9. Complete the anomaly detection task of the LGS-UAD model on the entire dataset and obtain the final detection results.

[0018] Preferably, the context embedding and position embedding in step S3 are formulated as follows:

[0019] X emb =ContexEmbedding(X)+PostionEmbedding(X)

[0020] in, represents the input sequence, represents the encoded hidden layer variable, ContextEmbedding(·) represents the context embedding transformation function, and PositionEmbedding(·) represents the position embedding transformation function.

[0021] Preferably, the context embedding transformation function includes a one-dimensional convolution, which is used to implement context embedding. The trend information of the time series is retained through local convolution operations, avoiding mismatching caused by ignoring trend information, so that the model can better capture the local dynamic changes of the time series. The parameters are:

[0022] Conv1d(in=d,out=dim,kernel_size=3,padding

[0023] =padding,padding_mode=circular,bias=False)

[0024] The number of input channels is d, the number of output channels is dim, and the convolution kernel of size 3 is in X embThe length dimension is slid, and for each time step in the range, the feature dimension dim=Wd is linearly transformed, and then the transformed features in the range are aggregated according to the convolution kernel parameters to obtain the context-enhanced embedding encoding X context ;

[0025] The position embedding transformation function uses the sine and cosine absolute position encoding. For the pos-th position and the i-th dimension in the sequence, the formula of the position encoding is as follows:

[0026]

[0027] where d is the dimension of the position encoding vector. The sine and cosine values are filled into the even and odd columns of the position encoding matrix respectively to obtain the encoded position embedding encoding X postion ; the context-enhanced embedding encoding X context is superimposed on the encoded position embedding encoding X postion to obtain the embedding encoding representation X emb of the input sequence.

[0028] Preferably, the bidirectional GRU reconstruction module in step S3 is composed of a forward gate recurrent unit layer (forward GRU layer), a neuron random dropout layer (dropout layer), and a reverse gate recurrent unit layer (reverse GRU layer). It can reconstruct a more general sequence representation from randomly masked data, effectively filter abnormal values in time series data, and extract local features of normal time domain characteristics. Compared with unidirectional GRU and LSTM, it can more comprehensively capture the features of the sequence, reduce the risk of overfitting, and more effectively filter abnormal values and extract stable features when reconstructing the sequence representation. The embedding encoding representation X emb First, the forward gate recurrent unit layer learns to reconstruct the representation from the left neighborhood, and then the neuron random dropout layer hides the state and inputs the reverse gate recurrent unit layer to learn the input reconstruction representation from the right neighborhood. Finally, the output local hidden features X local are obtained. t ∈X emb , the output y t ∈X local satisfies the formula

[0029]

[0030] where, represents the hidden state output at t-1 time in the forward gate recurrent unit layer, represents the hidden state output at t+1 time in the reverse gate recurrent unit layer.

[0031] Preferably, the multi-scale convolutional sparse attention module in step S3 consists of a convolutional projection layer, a multi-head partitioning layer, a cutting and downsampling layer, and a dot product attention layer. The convolutional projection layer, the multi-head partitioning layer, and the downsampling layer implement a layered attention mechanism. Different segmentation and downsampling parameters are equivalent to sparse templates to extract global features from different scale perspectives, avoiding the problem of excessive attention computational complexity. Compared with traditional single-scale convolution or fully connected attention mechanisms, it can capture features of different scales in more detail, reducing computational complexity while maintaining sensitivity to global features.

[0032] Preferably, the convolutional projection layer embeds the encoded representation X emb Through three one-dimensional convolution modules with the same structure but no shared parameters, they are transformed into the query, key, and value components required by the attention mechanism, as follows:

[0033] Q = Conv1d q (X emb )

[0034] K = Conv1d k (X emb )

[0035] V=Conv1d v (X emb )

[0036] in, is the query component, is the key component, Is the value component, Conv1d q 、Conv1d k 、Conv1d v The calculation functions corresponding to each component are respectively; a channel q of the Q component j As an example, the detailed calculation process is as follows:

[0037]

[0038] in, Represents the bias term corresponding to channel j in the Q component convolution module, the convolution kernel Among them C out ,C in Represents the number of output channels and the number of input channels respectively, and k is the size of the convolution kernel. In the summation, the convolution kernel weight (j, i, k) corresponding to the input channel i and the corresponding embedded code representation Perform convolution cross-correlation operation ★ to obtain the result q corresponding to output channel j j, combining all output channels, we get the final Q component. Similarly, we can get the K and V components;

[0039] In the multi-head segmentation layer, the channel dimensions of Q, K, and V are segmented according to the preset number of heads, so that the dimension changes from (B, L, dim) to (B, L, heads, dim / heads), where B is the batch size, L is the window size, and dim is the hidden layer channel dimension; the multi-head segmentation result is recorded as Q mh ,K mh ,V mh ;

[0040] In the cutting and downsampling layer, multiple attention heads are first grouped, and different cutting parameters and downsampling parameters are used in each group. The grouping process is formulated as follows:

[0041] group1,...group g_num =Group(head1,...head h )

[0042] group1=(head1,...,head g_size )

[0043] Among them, g_num and g_size represent the number of groups and the number of attention heads in each group respectively, h represents the total number of attention heads, and Group is a grouping function used for division; after the division is completed, according to the preset segmentation parameters segments = [segment1,...,segment g_num ] Split its length on the corresponding group, and after segmentation, group i, i∈g_num mh ,K mh ,V mh The shape becomes (B,L / segment i ,segment i ,heads,dim / heads), denoted as Q seg ,K seg ,V seg ;

[0044] Then according to the preset downsampling parameters dilations=[dilation i ,...,dilation g_num ], to Q seg ,K seg ,V seg On the corresponding group, its length is downsampled. After downsampling, Q in group i, i∈g_num is seg ,K seg ,V segThe shape becomes (B,L / segment i ,segment i / dilation i ,heads,dim / heads), denoted as Q dil ,K dil ,V dil ;

[0045] Finally, to make Q dil ,K dil ,V dil The shape of satisfies the requirements of dot product attention, and the number of segmentation segments is folded to obtain the Q that finally completes multi-scale segmentation and downsampling. msd ,K msd ,K msd , whose shape is:

[0046] (B×L / segment i ,sgement i / dilation i ,heads,dim / heads)

[0047] Among them, B is the batch size, L is the window length, i∈g_num represents the corresponding subscript in the segmentation and downsampling parameter group, segment i ,dilation i All are multi-head attention groups i The corresponding parameters are heads, the number of attention heads, and dim, the hidden layer dimension.

[0048] In the dot product attention layer, the attention result of group i is attn i The calculation formula is as follows:

[0049]

[0050] Then merge the attention outputs of different segmentation parameters and downsampling parameters to output the final global feature X global .

[0051] Preferably, for X local ,X global A residual connection is added, and the formula is as follows:

[0052]

[0053] Preferably, in step S4, the local feature X output in step S3 is first local and global feature X global Converted into probability distribution p local 、p global , the formula is as follows:

[0054] p loacl =softmax(X local ),p global =softmax(X global )

[0055] Then use the symmetric phase KL divergence D symKL Evaluate and compute two probability distributions p local ,p global The similarity between local and global features is evaluated using the symmetric phase KL divergence, which solves the problem of traditional KL divergence where positive and negative values ​​cancel each other out, and more accurately reflects the similarity of time domain feature distribution. The formula is as follows:

[0056]

[0057] Among them, dim is the hidden layer dimension, P local (x i ) represents the probability density at the i-th position in the channel dimension, and ∈ is an infinitesimal quantity used to avoid zero value errors. The average value is taken over all dimensions of the result to obtain the similarity score s within this window. The formula is as follows:

[0058] s=Sim(p local ,p global )=Mean(D symKL (p locla ,p global )).

[0059] Preferably, the contrast loss function in step S5 is:

[0060]

[0061] Here, Sim(·) represents the similarity scoring function, and stopgrad removes this tensor from the PyTorch computation graph, preventing it from participating in backpropagation. This approach can reduce the differences in reconstruction of local features and amplify the differences in global features, causing the local feature reconstructor to tend to reconstruct the local characteristics of the input sequence, while the global feature extractor focuses on prominent abnormal features outside the neighborhood.

[0062] Therefore, the present invention adopts the above-mentioned time series anomaly detection method based on feature distribution similarity. The feature of a point in the time series contains its trend information in addition to its value. The context embedding coding adopted by the present invention is composed of one-dimensional convolution. Through the locality of the convolution operation, the mismatch caused by ignoring the trend information is avoided. The parameters learned by the convolution kernel here actually reflect the periodic scale information in the original input. In addition, in the bidirectional GRU reconstruction module, the bidirectional GRU with dropout layer allows the model to reconstruct a more generalized sequence representation from randomly masked data. With the help of long short-term memory characteristics, the module can better filter outliers in time series data and extract local features that focus on normal time domain characteristics. In the global feature extractor, hierarchical attention is implemented using segmentation and downsampling layers. Different segmentation and downsampling parameters act as manually formulated sparse templates for attention modalities, extracting global features from different scales and perspectives while avoiding the high computational complexity of attention. Furthermore, traditional self-attention methods fail to fully utilize context when calculating point-by-point similarity. To address this, this method uses one-dimensional convolution with a kernel size of 3 rather than linear projection when calculating the Q and K matrices, calculating patch-wise rather than point-wise similarity. Furthermore, the contrastive loss function minimizes differences in local feature reconstruction while amplifying differences in globally extracted features. This results in the local feature reconstructor tending to reconstruct local features of the input sequence, while the global feature extractor focuses on prominent, anomalous features outside the neighborhood. Furthermore, the symmetric phase KL divergence solves the problem of traditional KL divergence, where positive and negative phase differences cancel each other out, and more accurately reflects the similarity of temporal feature distributions.

[0063] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Schematic diagram of the overall architecture of the anomaly detection method LGS-UAD based on feature distribution similarity of the present invention. DETAILED DESCRIPTION

[0065] Example

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0067] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0068] like Figure 1 As shown, a time series anomaly detection method based on feature distribution similarity includes the following steps:

[0069] S1. Data preprocessing and segmentation to obtain training data set, test data set and validation data set, specifically:

[0070] Collect real data sets from five different fields, segment and preprocess them to meet the model's data processing requirements. First, uniformly set the length of each group of data to 256, and segment the entire data set using a sliding window. Use 70% of the data as the training data set, 10% of the data as the validation data set, and 20% of the data as the test data set. Then, for the anomaly detection task, input 64 groups of data each time for training and verification. When finally testing the model's effectiveness, input all the data from the test data set into the model for testing;

[0071] S2. Based on the training data set obtained in step S1, 64 sets of training data are randomly selected each time for training;

[0072] S3, the time series processed in step S1 is used as input data, and then input into the bidirectional GRU reconstruction module and the multi-scale convolutional sparse attention module in parallel for feature extraction after context embedding and position embedding, to obtain local features and global features respectively;

[0073] Context embedding and position embedding, the formula is as follows:

[0074] X emb =ContextEmbedding(X)+PostionEmbedding(X)

[0075] in, represents the input sequence, represents the encoded hidden layer variable, ContextEmbedding(·) represents the context embedding transformation function, and PositionEmbedding(·) represents the position embedding transformation function.

[0076] The context embedding transformation function contains a one-dimensional convolution, which is used to implement context embedding. The trend information of the time series is retained through local convolution operations, avoiding mismatching caused by ignoring trend information, so that the model can better capture the local dynamic changes of the time series. The parameters are:

[0077] Conv1d(in=d,out=dim,kernel_size=3,padding

[0078] =padding,padding_mode=circular,bias=False)

[0079] The number of input channels is d, the number of output channels is dim, and the convolution kernel of size 3 is in X emb Slide on the length dimension, linearly transform the feature dimension dim=Wd for each time step within the range, and then perform weighted aggregation of the transformed features within the range according to the convolution kernel parameters to obtain the context-enhanced embedded code X context ;

[0080] The position embedding transformation function uses sine and cosine absolute position encoding. For the posth position and i-th dimension in the sequence, the position encoding formula is as follows:

[0081]

[0082] Where d is the dimension of the position encoding vector, and the sine and cosine values ​​are filled into the even and odd columns of the position encoding matrix respectively to obtain the encoded position embedding code X postion ; Context-enhanced embedded coding X context and the encoded position embedding code X postion The embedded coding representation X of the input sequence is obtained by superposition emb .

[0083] The bidirectional GRU reconstruction module consists of a forward gated recurrent unit layer (forward GRU layer), a neuron random dropout layer (dropout layer), and a reverse gated recurrent unit layer (reverse GRU layer). It can reconstruct a more generalizable sequence representation from randomly masked data, effectively filtering outliers in time series data and extracting local features of normal time domain characteristics. Compared with unidirectional GRU and LSTM, it can more comprehensively capture sequence features, reduce the risk of overfitting, and more effectively filter outliers and extract stable features when reconstructing sequence representations. emb First, the forward gated recurrent unit layer learns the reconstructed representation from the left neighborhood, then the hidden state of the neuron random discard layer is input into the reverse gated recurrent unit layer to learn the reconstructed representation of the input from the right neighborhood, and finally the output local hidden feature X is obtained.local , for the input x at time t t ∈X emb , whose output y t ∈X local Satisfy the formula

[0084]

[0085] in, represents the hidden state output at time t-1 in the forward gated recurrent unit layer, Represents the hidden state output at time t+1 in the reverse gated recurrent unit layer.

[0086] The multi-scale convolutional sparse attention module consists of a convolutional projection layer, a multi-head partitioning layer, a slicing and downsampling layer, and a dot product attention layer. This layer implements a layered attention mechanism. Different slicing and downsampling parameters act as sparse templates to extract global features from different scales, avoiding the problem of excessive attention computational complexity. Compared to traditional single-scale convolutional or fully connected attention mechanisms, it can capture features at different scales in greater detail, reducing computational complexity while maintaining sensitivity to global features.

[0087] The convolutional projection layer embeds the encoded representation X emb Through three one-dimensional convolution modules with the same structure but independent parameters, they are transformed into the query, key, and value components required by the attention mechanism, as follows:

[0088] Q = Conv1d q (X emb )

[0089] K=COnv1d k (X emb )

[0090] V=Conv1d v (X emb )

[0091] in, is the query component, is the key component, Is the value component, Conv1d q 、Conv1d k 、Conv1d v The calculation functions corresponding to each component are respectively; a channel q of the Q component j As an example, the detailed calculation process is as follows:

[0092]

[0093] in, Represents the bias term corresponding to channel j in the Q component convolution module, the convolution kernel Among them C out ,C in Represents the number of output channels and the number of input channels respectively, and k is the size of the convolution kernel. In the summation, the convolution kernel weight (j, i, k) corresponding to the input channel i and the corresponding embedded code representation Perform convolution cross-correlation operation ★ to obtain the result q corresponding to output channel j j , combining all output channels, we get the final Q component. Similarly, we can get the K and V components;

[0094] In the multi-head segmentation layer, the channel dimensions of Q, K, and V are segmented according to the preset number of heads, so that the dimension changes from (B, L, dim) to (B, L, heads, dim / heads), where B is the batch size, L is the window size, and dim is the hidden layer channel dimension; the multi-head segmentation result is recorded as Q mh ,K mh ,V mh ;

[0095] In the cutting and downsampling layer, multiple attention heads are first grouped, and different cutting parameters and downsampling parameters are used in each group. The grouping process is formulated as follows:

[0096] group1,...group g_num =Group(head1,...head h )

[0097] group1=(head1,...,head g_size )

[0098] Among them, g_num and g_size represent the number of groups and the number of attention heads in each group respectively, h represents the total number of attention heads, and Group is a grouping function used for division; after the division is completed, according to the preset segmentation parameters segments = [segment1,...,segment g_num ] Split its length on the corresponding group, and after segmentation, group i, i∈g_num mh ,K mh ,V mh The shape becomes (B,L / segment i ,segment i ,heads,dim / heads), denoted as Q seg ,K seg ,Vseg ;

[0099] Then according to the preset downsampling parameters dilations=[dilation i ,...,dilation g_num ], to Q seg ,K seg ,V seg On the corresponding group, its length is downsampled. After downsampling, Q in group i, i∈g_num is seg ,K seg ,V seg The shape becomes (B,L / segment i ,segment i / dilationi,heads,dim / heads), recorded as Q dil ,K dil ,V dil ;

[0100] Finally, to make Q dil ,K dil ,V dil The shape of satisfies the requirements of dot product attention, and the number of segmentation segments is folded to obtain the Q that finally completes multi-scale segmentation and downsampling. msd ,K msd ,K msd , whose shape is:

[0101] (B×L / segment i ,segment i / dilation i ,heads,dim / heads)

[0102] Among them, B is the batch size, L is the window length, i∈g_num represents the corresponding subscript in the segmentation and downsampling parameter group, segment i ,dilation i All are multi-head attention groups i The corresponding parameters are heads, the number of attention heads, and dim, the hidden layer dimension.

[0103] In the dot product attention layer, the attention result of group i is attn i The calculation formula is as follows:

[0104]

[0105] Then merge the attention outputs of different segmentation parameters and downsampling parameters to output the final global feature X global .

[0106] To X local ,X global A residual connection is added, and the formula is as follows:

[0107]

[0108] S4. The local features and global features obtained in step S3 are converted into probability distributions respectively through the normalized exponential function. The similarity of the probability distributions corresponding to the local features and global features is evaluated using the symmetrical phase KL divergence, and normalized within the detection window to obtain the similarity evaluation value at each time point. The details are as follows:

[0109] First, the local feature X output in step S3 local and global feature X global Converted into probability distribution p local 、p global , the formula is as follows:

[0110] p loacl =softmax(X local ),p global =softmax(X global )

[0111] Then use the symmetric phase KL divergence D symKL Evaluate and compute two probability distributions p local ,p global The similarity between local and global features is evaluated using the symmetric phase KL divergence, which solves the problem of traditional KL divergence where positive and negative values ​​cancel each other out, and more accurately reflects the similarity of time domain feature distribution. The formula is as follows:

[0112]

[0113] Among them, dim is the hidden layer dimension, P local (x i ) represents the probability density at the i-th position in the channel dimension, and ∈ is an infinitesimal quantity used to avoid zero value errors. The average of all dimensions of the result is taken to obtain the similarity score s within this window. The formula is as follows:

[0114] s=Sim(p local ,p global )=Mean(D symKL (p local ,p global )).

[0115] S5. Using the contrast loss function, we reduce the divergence of local features and amplify the divergence of global features, learn the time domain characteristics of normal distribution and highlight abnormal points, and complete the training of the LGS-UAD model. The contrast loss function is:

[0116]

[0117] Here, Sim(·) represents the similarity scoring function, and stopgrad removes this tensor from the PyTorch computation graph, preventing it from participating in backpropagation. This approach can reduce the differences in reconstruction of local features and amplify the differences in global features, causing the local feature reconstructor to tend to reconstruct the local characteristics of the input sequence, while the global feature extractor focuses on prominent abnormal features outside the neighborhood.

[0118] S6. Based on the validation data set obtained in step S1, all group data are input into the trained LGS-UAD model, and the symmetric phase KL divergence output by the LGS-UAD model is evaluated to determine whether the LGS-UAD model is continuing to optimize;

[0119] S7, repeat steps S2-S6 until the obtained evaluation index no longer improves. At this time, the network parameters are updated and the LGS-UAD model training ends;

[0120] S8. Based on the test data set obtained in step S1, all group data are input into the trained LGS-UAD model. The trained LGS-UAD model first calculates the anomaly score threshold on the training data set, and then evaluates the test data set according to the anomaly score threshold to obtain the anomaly detection result of the LGS-UAD model on the group data;

[0121] S9. Complete the anomaly detection task of the LGS-UAD model on the entire dataset and obtain the final detection results.

[0122] Under the same experimental conditions, the comparison results of the detection results of 16 anomaly detection methods and this method on real multivariate datasets are shown in Table 2, where P, R and F1 represent precision, recall and F1 score respectively, and all results are in %;

[0123] Table 2 Comparison of the detection results of 16 existing anomaly detection methods and this method on real multivariate datasets

[0124]

[0125] As can be seen from the above, LGS-UAD performs the best or second best among 17 methods on 5 public datasets. Compared with the current best model, the F1 score is improved from 87.18% to 91.10% (SMD dataset); 97.02% to 97.25% (SMAP); 97.94% to 98.63% (PSM); and 96.41% to 98.35%. The MACs and CUDAmem indicators that measure computing speed are reduced by 64%.

[0126] Therefore, the present invention adopts the above-mentioned time series anomaly detection method based on feature distribution similarity, which can meet the demand of timely detecting anomalies from large amounts of data streams in real environments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A time series anomaly detection method based on feature distribution similarity, characterized in that: The following steps are involved: S1. Data preprocessing and segmentation to obtain training data set, test data set and validation data set; S2. Based on the training data set obtained in step S1, 64 sets of training data are randomly selected each time for training; S3, the time series processed in step S1 is used as input data, and then input into the bidirectional GRU reconstruction module and the multi-scale convolutional sparse attention module in parallel for feature extraction after context embedding and position embedding, to obtain local features and global features respectively; S4. Convert the local features and global features obtained in step S3 into probability distributions respectively through a normalized exponential function, use the symmetrical phase KL divergence to evaluate the similarity of the probability distributions corresponding to the local features and the global features, and normalize them within the detection window to obtain the similarity evaluation value at each time point; S5. Using the contrast loss function, the divergence of local features is reduced and the divergence of global features is amplified, the time domain characteristics of the normal distribution are learned and the abnormal points are highlighted, completing the training of the LGS-UAD model. S6. Based on the validation data set obtained in step S1, all group data are input into the trained LGS-UAD model, and the symmetric phase KL divergence output by the LGS-UAD model is evaluated to determine whether the LGS-UAD model is continuing to optimize; S7, repeat steps S2-S6 until the obtained evaluation index no longer improves. At this time, the network parameters are updated and the LGS-UAD model training ends; S8. Based on the test data set obtained in step S1, all group data are input into the trained LGS-UAD model. The trained LGS-UAD model first calculates the anomaly score threshold on the training data set, and then evaluates the test data set according to the anomaly score threshold to obtain the anomaly detection result of the LGS-UAD model on the group data; S9. Complete the anomaly detection task of the LGS-UAD model on the entire dataset and obtain the final detection results.

2. The time series anomaly detection method based on feature distribution similarity according to claim 1 is characterized in that: The context embedding and position embedding in step S3 are formulated as follows: ； in, represents the input sequence, represents the encoded hidden layer variables, represents the context embedding transformation function, Represents the position embedding transformation function.

3. The time series anomaly detection method based on feature distribution similarity according to claim 2 is characterized in that: The context embedding transformation function consists of a one-dimensional convolution with the following parameters: ； The number of input channels is d, the number of output channels is dim, and the convolution kernel size is 3. Slide on the length dimension, and linearly transform the feature dimension of each time step within the range , and then the features after transformation within the range are weighted and aggregated according to the convolution kernel parameters to obtain the context-enhanced embedded coding ; The position embedding transformation function uses sine and cosine absolute position encoding. Position and The position encoding formula is as follows: ； in, is the dimension of the position encoding vector, and the sine and cosine values ​​are filled into the even and odd columns of the position encoding matrix respectively to obtain the encoded position embedding code ; Context-enhanced embedded coding Embedded with the encoded position Superposition to obtain the embedded coding representation of the input sequence .

4. The time series anomaly detection method based on feature distribution similarity according to claim 1, characterized in that: The bidirectional GRU reconstruction module in step S3 consists of a forward gated recurrent unit layer, a neuron random dropout layer, and a reverse gated recurrent unit layer; the embedded coding representation First, the forward gated recurrent unit layer learns the reconstructed representation from the left neighborhood, and then the hidden state of the neuron random discard layer is input into the reverse gated recurrent unit layer to learn the reconstructed representation of the input from the right neighborhood, and finally the output local hidden features are obtained. ,for Input at the moment , whose output Satisfy the formula ； in, Represents the forward gated recurrent unit layer The hidden state output at each moment, Represents the reverse gated recurrent unit layer The hidden state output at each moment.

5. The time series anomaly detection method based on feature distribution similarity according to claim 1, characterized in that: The multi-scale convolutional sparse attention module in step S3 consists of a convolutional projection layer, a multi-head partitioning layer, a cutting and downsampling layer, and a point product attention layer.

6. The time series anomaly detection method based on feature distribution similarity according to claim 5, characterized in that: The convolutional projection layer embeds the encoded representation Through three one-dimensional convolution modules with the same structure but no shared parameters, they are transformed into the query, key, and value components required by the attention mechanism, as follows: ； in, is the query component, is the key component, is the value component, are the calculation functions corresponding to each component respectively; In the multi-head division layer, the number of heads is preset ,right The channel dimension is split so that the dimension is becomes ,in is the batch size, is the window size, is the hidden layer channel dimension; the multi-head segmentation result is ; In the cutting and downsampling layer, multiple attention heads are first grouped, and different cutting parameters and downsampling parameters are used in each group. The grouping process is formulated as follows: ； in, Represent the number of groups and the number of attention heads in each group, Represents the total number of attention heads, It is a grouping function used for division; division is completed according to the preset segmentation parameters Divide the length of the corresponding group, and the group after division Inside The shape becomes , recorded as ; Then according to the preset downsampling parameters ,right On the corresponding group, its length is downsampled, and after the downsampling is completed, the group Inside The shape becomes , recorded as ; Finally, the number of segments is folded to obtain the final multi-scale segmentation and downsampling. , whose shape is: ； in, is the batch size, is the window length, Represents the corresponding subscripts in the segmentation and downsampling parameter groups, Both are multi-head attention groups The corresponding parameters, is the number of attention heads, is the hidden layer dimension; In the dot product attention layer, Group attention results The calculation formula is as follows: ； Then merge the attention outputs of different segmentation parameters and downsampling parameters to output the final global features .

7. The time series anomaly detection method based on feature distribution similarity according to claim 6 is characterized in that: right A residual connection is added, and the formula is as follows: 。 8. The time series anomaly detection method based on feature distribution similarity according to claim 1 is characterized in that: In step S4, the local features output in step S3 are first and global features Convert to probability distribution , the formula is as follows: ； Then use the symmetric phase KL divergence Evaluate two probability distributions The similarity is as follows: ； in, is the hidden layer dimension, Indicates the channel dimension The probability density at each position, It is an infinitesimal quantity used to avoid zero value errors; take the average of all dimensions of the result to get the similarity score within this window , the formula is as follows: 。 9. The time series anomaly detection method based on feature distribution similarity according to claim 1, characterized in that: The contrast loss function in step S5 is: ； in, represents the similarity scoring function, To separate the tensor from the pytorch computation graph, it does not participate in back propagation.

Images