A data anomaly detection method based on distillation mechanism

Abstract

The present invention discloses a data anomaly detection method based on a distillation mechanism, comprising the following steps: 1: collecting and preprocessing original data; 2: compressing and encoding the preprocessed data, and projecting the input data into a latent space by an encoder to obtain an initial latent vector; 3: inputting the initial latent vector into an aggregation module in the latent space to obtain a landmark vector and a weight vector of each initial latent vector, and obtaining a constrained latent vector corresponding to the initial latent vector one by one through the landmark vector and the weight vector; 4: performing distillation and filtering on the constrained latent vector to obtain a clean constrained latent vector containing only normal mode features; 5: decoding and reconstructing the clean constrained latent vector to obtain reconstructed data corresponding to the input data; 6: calculating an anomaly score using the input data and the corresponding reconstructed data, setting an anomaly threshold, and a sample whose anomaly score exceeds the anomaly threshold is an abnormal sample; the present invention can adaptively update normal features during training, and has high accuracy in anomaly detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of information processing, and in particular relates to a data anomaly detection method based on a distillation mechanism. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and big data, anomaly detection, as a classic and core key research, has been widely used in various fields of our production and life, such as financial fraud, network attacks, human health detection, urban security construction, and equipment fault diagnosis. In anomaly detection tasks, usually, compared with normal samples, abnormal samples are often very few, and the abnormal conditions of abnormal samples are heterogeneous and unknown. This class imbalance and the unknown and heterogeneous characteristics of abnormal samples lead to the fact that labeling them often requires a lot of time and expert knowledge, which is very difficult. Therefore, unsupervised anomaly detection research has important practical significance.

[0003] Although the task of anomaly detection has been an active research area for many years, there are still some unique and complex challenges. Most of the existing anomaly detection methods assume that the training data is clean, that is, the training data are all normal samples. However, real data often contains a small number of abnormal samples among a large number of normal samples. As mentioned above, it is very difficult to mark a small number of heterogeneous abnormal samples among a large number of normal samples. Therefore, in real situations, there are often noisy instances that are incorrectly marked as opposite category labels, and existing methods are very fragile to this situation, and are prone to false positives and false negatives in the actual detection process. At the same time, most of the existing anomaly detection methods ignore the diversity of normal samples. In actual situations, normal and abnormal often contain multiple categories. Even if there is only one category, there may be multiple different feature descriptions. On the other hand, in the few methods that consider multiple normal modes, it is often necessary to manually predefine the number of normal modes, and it is impossible to adaptively obtain the number of normal modes according to the characteristics of the data samples. Therefore, it is very important to study an unsupervised anomaly detection method that considers the diversity of normal samples under the training condition of noise interference and does not require manual predefinement of the number of normal modes. Summary of the invention

[0004] In view of this, the present invention provides a data anomaly detection method based on a distillation mechanism, which can take into account the diversity of normal sample features and can adaptively update normal features and their quantity during the training process, and has high anomaly detection accuracy.

[0005] The technical solution for implementing the present invention is as follows:

[0006] A data anomaly detection method based on a distillation mechanism comprises the following steps:

[0007] Step 1: Raw data collection and preprocessing;

[0008] Step 2: Input the collected and preprocessed data into the encoder for compression encoding. The encoder projects the input data into the latent space to obtain the initial hidden vector in the latent space.

[0009] Step 3: Input the initial hidden vector into the aggregation module in the latent space to obtain the landmark vector (feature prototype vector, i.e., the cluster center of the normal sample) and the weight vector of each initial hidden vector, and then obtain the constrained hidden vector corresponding to the initial hidden vector one by one through the landmark vector and the weight vector;

[0010] Step 4: Input the landmark vector and the constrained hidden vector into the distillation module at the same time, perform distillation and filtering on the constrained hidden vector to obtain a clean constrained hidden vector containing only normal pattern features;

[0011] Step 5: Input the clean constrained hidden vector into the decoder for decoding and reconstruction to obtain the reconstructed data corresponding to the input data in step 2;

[0012] Step 6: Use the input data and the corresponding reconstructed data to calculate the reconstruction error, take the weighted sum of the reconstruction error and the entropy of the weight vector as the anomaly score, set the anomaly threshold, and the samples whose anomaly scores exceed the anomaly threshold are abnormal samples.

[0013] Furthermore, in step 1, preprocessing includes deleting the continuously missing data, filling the non-continuously missing data with interpolation based on the previous and next data, and standardizing the data.

[0014] Furthermore, in step 3, in the aggregation module, the features on the hidden plane of the normal sample are analyzed, and an initial hidden vector whose local density is greater than that of its surrounding vectors is selected as a feature prototype vector, namely, a landmark vector.

[0015] Furthermore, in step 3, for each initial hidden vector input into the aggregation module, based on the attention mechanism, a constrained hidden vector corresponding to the initial hidden vector is formed by weighted combination of several landmark vectors; several landmark vectors form a convex polygon boundary; the mapping operation based on the attention mechanism makes the formed constrained hidden vectors, i.e., normal features, all fall within the convex polygon boundary composed of landmark vectors, minimizing the intra-class differences of normal samples and ensuring the consistency of their representation in the latent space, so as to better detect abnormal samples.

[0016] Furthermore, in step 4, the mixed noisy instances are filtered out through the distillation module so that they do not participate in the training; during the training process, the samples in the area with the center C of the convex polygon with the landmark vector as the vertex and r as the radius on the hidden plane are deleted so that they do not participate in the training of the model; thereby ensuring the anomaly detection effect of the model when there is interference from abnormal samples in the training set.

[0017] Furthermore, in step 4, the radius r is adaptively reduced to shrink the size of the filter area, thereby reducing the loss of normal samples in the training set.

[0018] Furthermore, the radius r is adaptively reduced, specifically using the following formula:

[0019] r decayed = r·e -ρ·global_step

[0020] Among them, r decayed is the distillation radius after adopting the natural exponential decay strategy, ρ is the set decay rate, global_step is the current number of training iterations, and r is the set initial radius.

[0021] Beneficial effects:

[0022] 1. The method of the present invention is an unsupervised end-to-end anomaly detection method with convenient training, high detection accuracy, low missed detection rate, and good versatility and high efficiency.

[0023] 2. The method of the present invention takes into account the diversity of normal sample features, and can adaptively update normal features and their quantity during the training process, so the anomaly detection accuracy is high;

[0024] 3. The method of the present invention forms a convex polygon boundary through landmark vectors, so that the hidden vectors of normal samples are all within the boundary, thereby minimizing the intra-class differences of normal samples and ensuring the consistency of their representation in the latent space, so that the method can better detect abnormal samples.

[0025] 4. The method of the present invention uses a distillation mechanism based on spatial features, so that the model can perform anomaly detection in the presence of noise interference and ensure accuracy while reducing the computational cost of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0028] like Figure 1As shown, the present invention provides a data anomaly detection model based on a distillation mechanism, which includes an encoder, a decoder, and an aggregation module and a distillation module in a latent space. The model includes an adaptive landmark vector generation mechanism based on local density, which takes into account the diversity of normal patterns, minimizes the intra-class differences of normal samples, and ensures the consistency of their representation in the latent space. At the same time, through the distillation mechanism, the model ensures the anomaly detection accuracy of the model in an actual training environment with noise interference, while reducing the computational cost of the model.

[0029] Step 1 is the collection and preprocessing of raw data. The method can process various one-dimensional data, such as human health detection data, network attack data, and device sensor detection data. The preprocessing of raw data includes deleting the continuously lost data, and for the non-continuously lost data, interpolating and filling according to the previous and next data, as well as data standardization.

[0030] Step 2 is the process of encoding the input data. First, define X to represent the domain where the input data samples are located, and Z to represent the latent space. Let the input data sample set be U = {x i |i=1,...,M}, where x i ∈X represents the i-th data sample. Based on this, the encoding process of the encoder can be expressed as f e (·):X→Z, given a sample x∈X, the encoder maps it from the X domain to the Z domain, z∈Z. The specific process can be expressed as follows:

[0031] z=f e (x;θ e )

[0032] Among them, θ e Represents the parameters of the encoder.

[0033] In step 3, in order to ensure that the landmark vector has a high local density and the distance between different landmark vectors is relatively far and sparse, the initial hidden vector with a local reachable density greater than its surrounding vectors is selected as the feature prototype vector, i.e., the landmark vector. The landmark vector set P is expressed as follows:

[0034]

[0035] Among them, Z represents the initial hidden vector set in the latent space, Ν k (p) is the k nearest neighbors of the p vector, lrd(z) is the local reachability density of the z vector, and the local reachability density is expressed as follows:

[0036]

[0037] Among them, ρ iThe local density is expressed as follows:

[0038]

[0039] If X<0, then χ(x)=1, otherwise χ(x)=0. ij Represents the distance between points i and j, d c is the cutoff distance.

[0040] On this basis, the constraint hidden vector can be obtained as the linear weighted sum of the landmark vectors as follows:

[0041]

[0042]

[0043] Where L is the landmark vector, L∈R F×D , F represents the number of landmark vectors, D represents the dimension of each landmark vector. W is the weight vector, W∈R 1×F ,ω i is the i-th non-negative element of the vector W. For an initial hidden vector z∈Z, ω i The corresponding attention weights for each landmark vector are as follows:

[0044]

[0045] Among them, d(.,.) represents a similarity measure, and the cosine similarity is used here to calculate as follows:

[0046]

[0047] Through the above-mentioned mapping operation based on the attention mechanism, all samples can be mapped into the boundary of a convex polygon with landmark vectors as vertices.

[0048] Step 4 is a distillation operation, the purpose of which is to filter out the mixed noisy instances so that they do not participate in training. Since the landmark vector is selected to represent the prototype feature representation of the normal sample in the hidden plane, after the above mapping operation, the weight vector corresponding to the normal sample should have a smaller entropy value, that is, the mapped normal sample should be located within the boundary of the convex polygon with the landmark vector as the vertex, and distributed close to one of the landmark vectors. On the contrary, for abnormal samples, since the landmark vector cannot represent their characteristics, their weight vector after mapping has a relatively large entropy value. Therefore, the hidden vector of the abnormal sample will be located near the central area of ​​the convex polygon after the above mapping operation. The center of the convex polygon can be expressed as follows

[0049]

[0050] During the training process, the samples in the area with center C as the circle center and radius r on the hidden plane are deleted so that they do not participate in the model training, and finally a clean constrained hidden vector is output. In this way, abnormal samples are filtered out in the training set, ensuring the anomaly detection effect of the model when there are abnormal samples in the training set.

[0051] The above distillation operation will make the abnormal samples in the training set gradually decrease as the number of iterations increases. Therefore, during the training process, the radius r of the filter area should also be adaptively shrunk accordingly to reduce the loss of normal samples in the training set. The natural exponential decay strategy is used to dynamically adjust r, and the formula is as follows:

[0052] r decayed = r·e -ρ·global_step

[0053] Among them, r decayed is the distillation radius after adopting the natural exponential decay strategy, ρ is the set decay rate, and global_step is the current number of training iterations.

[0054] Step 5 is the decoding process. The decoding process of the decoder can be expressed as f d (·):Z→X. The decoder converts the vector in the Z domain Mapped back to the X domain, The specific process can be expressed as follows:

[0055]

[0056] Among them, θ d Represents the parameters of the decoder, To reconstruct the data.

[0057] Step 6 is the abnormal sample detection process, that is, measuring whether the anomaly score is greater than the anomaly threshold. If it is greater, it is abnormal, otherwise it is normal. The anomaly score consists of two parts: reconstruction error and entropy of the weight vector. The reconstruction error can be calculated based on the input data and the reconstructed data, as shown below:

[0058]

[0059] Among them, x i represents the i-th input data, Represents the i-th reconstructed data.

[0060] The entropy calculation formula of the weight vector is as follows:

[0061] E(ω i )=-ω i log(ω i )

[0062] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data anomaly detection method based on distillation mechanism, characterized in that: The following steps are involved: The method is used to process various types of one-dimensional data, including human health detection data, network attack data, or device sensor detection data; Step 1: Raw data collection and preprocessing; The preprocessing of raw data includes deleting the continuously missing data, filling the non-continuous missing data with interpolation based on the previous and next data, and standardizing the data. Step 2: Input the collected and preprocessed data into the encoder for compression encoding. The encoder projects the input data into the latent space to obtain the initial hidden vector in the latent space. Step 3: Input the initial hidden vector into the aggregation module in the latent space to obtain the landmark vector and the weight vector of each initial hidden vector, and then obtain the constrained hidden vector corresponding to the initial hidden vector one by one through the landmark vector and the weight vector; the landmark vector is the feature prototype vector or the cluster center of the normal sample; Step 4: Input the landmark vector and the constrained hidden vector into the distillation module at the same time, perform distillation and filtering on the constrained hidden vector to obtain a clean constrained hidden vector containing only normal pattern features; Step 5: Input the clean constrained hidden vector into the decoder for decoding and reconstruction to obtain the reconstructed data corresponding to the input data in step 2; Step 6: Use the input data and the corresponding reconstructed data to calculate the reconstruction error, take the weighted sum of the reconstruction error and the entropy of the weight vector as the anomaly score, set the anomaly threshold, and the samples whose anomaly scores exceed the anomaly threshold are abnormal samples.

2. The data anomaly detection method based on distillation mechanism as claimed in claim 1, characterized in that: In step 1, preprocessing includes deleting consecutive missing data, filling non-consecutive missing data with interpolation based on previous and next data, and data standardization.

3. The data anomaly detection method based on distillation mechanism as claimed in claim 1, characterized in that: In step 3, in the aggregation module, the features on the hidden plane of the normal sample are analyzed, and the initial hidden vector whose local density is greater than that of its surrounding vectors is selected as the feature prototype vector, that is, the landmark vector.

4. The data anomaly detection method based on distillation mechanism as claimed in claim 1, characterized in that: In step 3, for each initial hidden vector input into the aggregation module, based on the attention mechanism, a constrained hidden vector corresponding to the initial hidden vector is formed by weighted combination of several landmark vectors; several landmark vectors form a convex polygon boundary; the mapping operation based on the attention mechanism makes the formed constrained hidden vector, i.e., the normal features, all fall within the convex polygon boundary composed of the landmark vectors, minimizing the intra-class differences of normal samples and ensuring the consistency of their representation in the latent space, so as to better detect abnormal samples.

5. The data anomaly detection method based on distillation mechanism as claimed in claim 4, characterized in that: In step 4, the mixed noisy instances are filtered out through the distillation module so that they do not participate in the training; during the training process, the samples in the area with the center C of the convex polygon with the landmark vector as the vertex and the radius r on the hidden plane are deleted so that they do not participate in the training of the model; thereby ensuring the anomaly detection effect of the model when there is interference from abnormal samples in the training set.

6. The data anomaly detection method based on distillation mechanism as claimed in claim 5, characterized in that: In step 4, the radius r is adaptively reduced to shrink the size of the filter region, thereby reducing the loss of normal samples in the training set.

7. The data anomaly detection method based on distillation mechanism as claimed in claim 6, characterized in that: Adaptively reduce the radius r, using the following formula: r decayed ＝r·e -ρ·global_step Among them, r decayed is the distillation radius after adopting the natural exponential decay strategy, ρ is the set decay rate, global_step is the current number of training iterations, and r is the set initial radius.

Images