An unsupervised anomaly detection method based on latent feature decomposition

Abstract

The present invention proposes an unsupervised anomaly detection method based on latent feature decomposition, which belongs to the field of general anomaly detection; specifically, the method comprises: first, preprocessing given data to be detected to construct an unlabeled training data set; then, building an FDAE including an encoding module, a decoding module and a feature decomposition module; using the encoding module to transform the data into a latent space to obtain latent features; using the feature decomposition module to respectively extract normal and abnormal features of the latent features; then, using the decoding module to respectively decode the normal features and the abnormal features; based on the decoding results, a two-stage training strategy and an unlabeled data set are adopted to train the FDAE; using a test set to input the trained FDAE, respectively calculating the normal reconstruction error and the abnormal reconstruction error, and judging whether the test data is abnormal; the present invention has a faster training speed, requires fewer hyperparameters, and can process data with complex structures.

Description

Technical Field

[0001] The present invention belongs to the field of general anomaly detection, and in particular relates to an unsupervised anomaly detection method based on potential feature decomposition. Background Art

[0002] The task of machine learning is to identify data and use the data to continuously improve itself; however, there are often many abnormal data that do not follow normal (conventional) patterns that interfere with pattern recognition. The scale of these abnormal data is smaller than that of normal data, and they usually belong to new categories of abnormal patterns or are difficult to define noises. They are called outliers or anomalies (Anomaly detection: A survey). Anomaly detection, which identifies abnormal data, came into being.

[0003] In many practical applications, there are usually not enough labels for the anomaly detection model to identify abnormal data from large-scale data, so unsupervised methods are needed to perform anomaly detection. For example, when searching for images, search engines often need to exclude abnormal images from the search results without prior information; many industrial scenarios need to automatically find the moment when the machine is operating abnormally based on the time series generated by multiple sensors without clearly defined anomalies. The only assumption in these examples is that the scale of abnormal data is lower than that of normal data. Due to the serious imbalance of categories and the vague definition of anomalies, it is difficult to use binary or multi-classification methods for anomaly detection.

[0004] Reconstruction-based unsupervised anomaly detection models are a major mainstream of current research. The most common strategy is to use autoencoders and RPCA technology. Autoencoders are a neural network extension of PCA, and their expected output is the input itself. Generally speaking, autoencoders construct a mapping from input to itself through encoders and decoders. In anomaly detection tasks, autoencoders learn the latent structure of normal data so that the reconstruction error of normal data is lower than that of abnormal data, thereby achieving anomaly detection. In unsupervised scenarios, the data used to train autoencoders are usually mixed with unknown proportions of abnormal data. In order to increase the robustness of the model to these abnormal data, that is, to reduce the interference of abnormal data on the essential characteristics of normal data modeled by the autoencoder, nonlinear encoders and decoders are usually used, or robust objective functions, linear autoencoders, and complex data preprocessing are used.

[0005] Traditional PCA can handle very small noise, but it is fragile to severely damaged observations: even a single damaged observation may seriously interfere with the answer. The proposal of RPCA improves this defect of PCA.

[0006] RPCA can handle sparse noise in the original data, but if most of the data is corrupted, such as abnormal data from new categories, RPCA will no longer be applicable. RSR technology can make up for this shortcoming of RPCA. It has been proven in many literatures that it can cope with different noise patterns, and applying RSR technology to the autoencoder model can significantly improve the robustness of the autoencoder architecture anomaly detection model. For example, the literature: RSR AutoEncoder (RSRAE), (2019-ICLR Conference, Chieh-Hsin Lai, Dongmian Zou, and Gilad Lerman.) "Robust Subspace Recovery Layer for Unsupervised Anomaly Detection".

[0007] From a model perspective, there are two main differences between RSRAE and traditional autoencoder architecture anomaly detection models:

[0008] 1. Traditional autoencoders assume that the latent features of normal data are located in a low-dimensional latent space and map all data to a low-dimensional latent space; RSRAE maps data to a high-dimensional latent space and assumes that the latent structure of normal data is located in a low-dimensional subspace of this space.

[0009] 2. RSRAE does not directly decode the encoded result, but first performs a linear transformation on the decoded result (RSR layer in the figure) and then decodes it.

[0010] RSRAE considers the latent structure of normal patterns in the latent space, but does not consider the latent structure of abnormal data at all. In many cases, if the anomaly detection model of the autoencoder architecture can understand certain abnormal patterns, it is very helpful for modeling normal patterns. Many autoencoder architecture anomaly detection models will give a prior about anomalies and achieve good results in specific abnormal scenarios, but they cannot be applied to complex abnormal scenarios, and defining the prior of anomalies requires a lot of expert knowledge, which is very costly.

[0011] In recent years, in addition to RSRAE, the CoRA model is also an important attempt in the field of anomaly detection. CoRA believes that the potential features of normal data and abnormal data are distributed in different regions, so CoRA designs a subspace structure-preserving regularization.

[0012] CoRA training requires labeled normal data, which is a semi-supervised model and cannot be applied in unsupervised anomaly detection scenarios. In actual industrial environments, most application scenarios are unsupervised, so it is difficult to apply CoRA. In addition, CoRA does not directly consider the latent structure of normal patterns, but makes the features of normal data and abnormal data in different areas in the latent space. In actual big data production environments, the labels of labeled data are usually not completely correct, which will cause CoRA to fail. Summary of the invention

[0013] In view of the following problems faced in unsupervised anomaly detection scenarios: 1) the model cannot adapt to different abnormal data, that is, in the same anomaly detection task, the sources of abnormal data include new categories, damaged data, and difficult-to-define noise; 2) the model cannot model the potential structure of normal data; 3) the model is not robust to any proportion of abnormal data in the training data; 4) the model can only model normal data and cannot utilize the information brought by abnormal data. This paper proposes an unsupervised anomaly detection method based on latent feature decomposition, which can perform threshold-free anomaly detection in an unsupervised situation.

[0014] The unsupervised anomaly detection method based on potential feature decomposition has the following specific steps:

[0015] Step 1: Preprocess the given raw data to be detected, process the raw data into a uniform size, and construct an unlabeled training data set;

[0016] Step 2: construct a feature decomposition autoencoder FDAE including an encoding module, a decoding module and a feature decomposition module according to the data type to be detected;

[0017] The encoding module includes an encoder, the feature decomposition module includes an RSR layer and a noise extractor, the decoding module includes a normal data decoder and an abnormal data decoder, and the model parameters are initialized;

[0018] Step 3: Use the encoding module to transform the original data into the latent space to obtain the latent features of the original data;

[0019] Taking the original vector data as an example, encoder E converts the original data Mapped to a high-dimensional k-dimensional latent space, the potential features in this space are recorded as

[0020] N represents the number of training data, K represents the dimension of the original vector data, and k represents the dimension of the encoder's output z;

[0021] Step 4: Use the RSR layer of the feature decomposition module to extract the normal essential features Az of the data from the potential features (n), using the noise extraction module to extract the abnormal features of the data from the latent features (n) ;

[0022] The loss function L of the RSR layer in the feature decomposition module RSR Including two loss functions L RSR1 and L RSR2 , the specific calculation is:

[0023]

[0024] Where λ1 and λ2 are hyperparameters; A T Represents the parameter matrix of the RSR layer The transposed matrix, I d Represents a single matrix of d×d, considering A as the encoder of the latent features, A T The decoder considered as latent features.

[0025] The noise extractor F is used to extract abnormal information s from the latent features. (n) , that is, s (n) =F(z (n) ) and use it as an abnormal feature.

[0026] Step 5: Use the normal data decoder and the abnormal data decoder to decode the normal essential features of the data Az (n) and abnormal features (n) Decode them separately to get a normal copy and abnormal copies

[0027] Normal data decoder D norm For normal essential characteristics Az (n) Decode and get the original data x (n) A normal copy of the same shape Abnormal Data Decoder D anom For abnormal features (n) Decode and get x (n) Anomalous copies of the same shape

[0028] Step 6: Use a normal copy and abnormal copies Combine the two-stage training strategy with unlabeled datasets to train FDAE;

[0029] The two-stage training strategy includes: in the first stage, the parameters of AE1 are updated, where AE1 consists of encoder E, RSR layer A, and normal data decoder D norm In the second stage, the normal data decoder D is fixed normParameters of A and AE2 are updated; AE2 consists of encoder E, noise extractor F and abnormal data decoder D anom composition.

[0030] The specific process is:

[0031] In the first stage, the autoencoder loss function of AE1 is for:

[0032]

[0033] Combined with the RSR loss function, the overall objective function of FDAE at this stage is:

[0034]

[0035] In the actual update step, this stage backpropagates three expressions in sequence: The expressions related to the RSR loss function are only used to update the RSR layer.

[0036] In the second stage, the autoencoder loss function of AE2 is:

[0037]

[0038] Combined with the robust autoencoder RAE loss function, the overall objective function of FDAE at this stage is:

[0039]

[0040] The RAE loss function refers to the loss function that combines the noise extractor and the RSR layer. The calculation formula is as follows:

[0041]

[0042] where μ is a hyperparameter,

[0043] In the actual update step, this stage backpropagates 4 expressions in sequence: in Only used to update the RSR layer and noise extractor, Only used to update the RSR layer, Only used to update the noise extractor.

[0044] The updated values ​​in the second stage are multiplied by a scaling factor m = (1-1 / ep), where ep is the epoch of training.

[0045] Step 7: Determine whether the model has reached the predetermined number of training rounds. If yes, proceed to step 8; otherwise, return to step 3 and re-convert the data.

[0046] Step 8: Input the new test data into the trained FDAE, calculate the reconstruction error between the normal reconstructed copy (the output of the normal decoder) and the test data, recorded as the normal reconstruction error, and calculate the reconstruction error between the abnormal reconstructed copy (the output of the abnormal decoder) and the test data set, recorded as the abnormal reconstruction error;

[0047] The reconstruction error is calculated as follows:

[0048] For data x (n) , normal reconstruction error The abnormal reconstruction error is

[0049] Step nine: judging whether the test data is abnormal according to the normal reconstruction error and the abnormal reconstruction error;

[0050] There are two specific judgment strategies: 1. The normal reconstruction error is regarded as the abnormal score. If the abnormal score is greater than the pre-set threshold, the test data is judged as abnormal; 2. If the normal reconstruction error is greater than the abnormal reconstruction error, the test data is judged as abnormal.

[0051] The advantages of the present invention are:

[0052] 1) An unsupervised anomaly detection method based on latent feature decomposition. Compared with the traditional unsupervised anomaly detection model, the method proposed in the present invention has a faster training speed, requires fewer hyperparameters, and can process data with complex structures.

[0053] 2) An unsupervised anomaly detection method based on latent feature decomposition. Compared with previous unsupervised anomaly detection models based on autoencoder architecture such as RSRAE, it is more robust to abnormal data with unknown abnormal proportions and categories, has stronger modeling capabilities for normal patterns, and can learn certain abnormal patterns in unsupervised scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of an unsupervised anomaly detection method based on potential feature decomposition of the present invention;

[0055] Figure 2 The schematic diagram of the FDAE of the present invention is composed of an encoding module, a feature decomposition module and a decoding module;

[0056] Figure 3 A flowchart of the present invention using a two-stage training strategy to train FDAE parameters;

[0057] Figure 4 A flowchart used in an embodiment of the present invention;

[0058] Figure 5Schematic diagram of the process of calculating the side length of the feature map output by the encoder in Example 1 of the present invention. DETAILED DESCRIPTION

[0059] The present invention is described in detail below in conjunction with the embodiments and drawings.

[0060] The present invention proposes a feature decomposition autoencoder (FDAE for short), the central idea of ​​which is that FDAE first maps the original data to a high-dimensional latent space O, which has a low-dimensional subspace P (the dimension is much lower than that of O). The points in the low-dimensional subspace P represent the potential structure of normal data. FDAE simulates the orthogonal projection of the low-dimensional subspace P through a linear RSR layer (the parameter matrix is ​​A, and A is used to refer to the RSR layer later), and regards the projection as a linear autoencoder of a potential feature. At the same time, FDAE uses a noise extractor F to extract sparse high-dimensional noise in the potential features, uses the noise to characterize the potential structure of abnormal data, and combines A and F by borrowing the application idea of ​​RPCA on the autoencoder. In the above manner, FDAE can model some abnormal patterns while improving the modeling ability of the RSR layer for normal patterns.

[0061] like Figure 1 As shown, the specific steps are as follows:

[0062] Step 1: Preprocess the given raw data to be detected and construct an unlabeled training data set;

[0063] The original data is of a given arbitrary type and is preprocessed into a uniform shape. For example, for image data, all images are scaled to a size of 100*100.

[0064] Step 2: construct the encoding module and the decoding module according to the data type and initialize them, and determine the feature decomposition module to form the feature decomposition autoencoder FDAE;

[0065] like Figure 2 As shown, FDAE consists of three modules: encoding module, feature decomposition module and decoding module; the encoding module includes an encoder, the feature decomposition module includes an RSR layer and a noise extractor, the decoding module includes a normal data decoder and an abnormal data decoder, and initializes the model parameters;

[0066] Step 3: Use the encoding module to transform the original data into the latent space to obtain the latent features of the original data;

[0067] FDAE can process various types of data. For the sake of convenience, the following section takes vector data as an example for description.

[0068] The encoding module of FDAE contains an encoder E, which is used to convert the original data Mapped to a high-dimensional k-dimensional latent space (there is no connection between K and k, the high dimension of k is relative to the latent space dimension of the traditional autoencoder model), the potential features in this space are recorded as

[0069] N represents the number of training data, K represents the dimension of the original vector data, and k represents the dimension of the encoder's output z;

[0070] The goal of FDAE is to make the latent feature z (n) It is easier to be divided into normal and abnormal parts, that is, the latent representations of normal data and abnormal data belong to different subspaces of the latent space, rather than different regions of the latent space like FDAE.

[0071] Step 4: Use the RSR layer of the feature decomposition module to extract the normal essential features of the data from the potential features, and use the noise extraction module to extract the abnormal features of the data from the potential features;

[0072] The feature decomposition module divides the potential features into normal essential features and abnormal features. The RSR layer is used to extract normal features, and its parameter matrix The following formula is the loss function of the RSR layer (abbreviated as RSR loss function) L RSR There are two loss functions and The specific calculation is:

[0073]

[0074] Where λ1 and λ2 are hyperparameters; A T represents the transposed matrix of the parameter matrix A of the RSR layer, I d represents a d×d single matrix;

[0075] The purpose of the above formula is to make A T A becomes a mapping of a normal feature, and the image space of the mapping is equivalent to the d-dimensional latent subspace (d<<k) corresponding to the latent structure of the normal data. can be regarded as a latent feature z (n) At the coordinate point on this subspace, put Az (n) is called the normal essential feature of the data, and A is regarded as the encoder of the latent feature. T The decoder considered as latent features.

[0076] The noise extractor F is used to extract abnormal information s from the latent features. (n) , that is, s (n) =F(z (n) ) and use it as an abnormal feature.(n) Different, (n) and z (n) The matrix composed of all abnormal features of training data is called The role of S here is similar to that of S in RPCA, both of which represent sparse element-wise noise, but the former is at the feature level and the latter is at the sample level.

[0077] Step 5: Use the normal data decoder and the abnormal data decoder to decode the normal essential feature Az of the data. (n) and abnormal features (n) Decode them separately;

[0078] The decoding module of FDAE includes two decoders: normal data decoder D norm and abnormal data decoder D anom . Normal data decoder D norm For normal essential characteristics Az (n) Decode and get x (n) A normal copy of the same shape Abnormal Data Decoder D anom For abnormal information (n) Decode and get x (n) Anomalous copies of the same shape

[0079] Step 6: Use a normal copy and abnormal copies Combine the two-stage training strategy with unlabeled datasets to train FDAE;

[0080] The two-stage training strategy includes: in the first stage, the parameters of AE1 are updated, where AE1 consists of encoder E, RSR layer A, and normal data decoder D norm In the second stage, the normal data decoder D is fixed norm Parameters of A and AE2 are updated; AE2 consists of encoder E, noise extractor F and abnormal data decoder D anom composition.

[0081] The two-stage training strategy is Figure 3 As shown, the specific process is:

[0082] In the first stage, the autoencoder loss function of AE1 is for:

[0083]

[0084] Combined with the RSR loss function, the overall objective function of FDAE at this stage is:

[0085]

[0086] In the actual update step, this stage backpropagates three expressions in sequence: The expressions related to the RSR loss function are only used to update the RSR layer.

[0087] In the second stage, the autoencoder loss function of AE2 is:

[0088]

[0089] Combined with the Robust AutoEncoder (RAE) loss function, the overall objective function of FDAE at this stage is:

[0090]

[0091] The RAE loss function refers to the loss function that combines the noise extractor and the RSR layer. The calculation formula is as follows:

[0092]

[0093] where μ is a hyperparameter,

[0094] In the actual update step, this stage backpropagates 4 expressions in sequence: in Only used to update the RSR layer and noise extractor, Only used to update the RSR layer, Only used to update the noise extractor.

[0095] The updated values ​​in the second stage are multiplied by a scaling factor m = (1-1 / ep), where ep is the epoch of training. FDAE uses batch data for parameter updates.

[0096] Step 7: Determine whether the model has reached the predetermined number of training rounds. If yes, proceed to step 8; otherwise, return to step 3 and re-convert the data.

[0097] Step 8: Input the new test data into the trained FDAE, calculate the reconstruction error between the output of the normal decoder and the test data, recorded as the normal reconstruction error, and calculate the reconstruction error between the output of the abnormal decoder and the test data set, recorded as the abnormal reconstruction error;

[0098] The reconstruction error is calculated as follows:

[0099] For data x (n) , normal reconstruction error The abnormal reconstruction error is

[0100] Step nine: judging whether the test data is abnormal according to the normal reconstruction error and the abnormal reconstruction error;

[0101] There are two specific judgment strategies: 1. The normal reconstruction error is regarded as the abnormal score. If the abnormal score is greater than the pre-set threshold, the test data is judged as abnormal; 2. If the normal reconstruction error is greater than the abnormal reconstruction error, the test data is judged as abnormal.

[0102] Embodiment 1:

[0103] like Figure 4 As shown, the following is a detailed introduction:

[0104] Step 1: Given an unlabeled image dataset, where the number of images is N, the size format of a single image data is [length, width, number of input channels]. The length and width of each image are not necessarily equal, but the number of channels must be equal.

[0105] Let l = min(min(length of all images), min(width of all images)), scale all image data to a uniform size: [l, l, number of channels]. The pixel value range of the image is [0, 255]. Normalize the pixel values ​​and limit the value range to [0, 1]. Given the batch size C required for training, calculate the floating-point result of N / C, round the result up to get B, and divide the data set into B equal parts. Except for the RSR layer, all subsequent model components use the ReLU activation function. The RSR layer is a linear layer and does not use any activation function. Except for the RSR layer and the last layer of the two decoders, the outputs of other components need to be batch normalized during training.

[0106] Step 2: Encoder E contains 3 layers of convolutional neural network. The parameter sizes of the three convolutional layers are as follows: The parameter format is [output channel number, convolution kernel width, moving step, padding type]: Convolution layer 1 is [channel number 1, width 1, step size 1, same padding]; Convolution layer 2 is [channel number 2, width 2, step size 2, same padding]; Convolution layer 3 has parameters [channel number 3, width 3, step size 3, padding type 3], where padding type 3 is determined by the image l. If l%8=0, same padding is used, otherwise valid padding is used. Channel number 1 < channel number 2 < channel number 3. Introduce the RSR layer, whose parameter matrix is d<<k; introduce the noise extractor F, which is a one-dimensional convolution layer. Its parameter format is [number of channels, convolution kernel length, moving step, padding type], and its specific parameters are [channel 4, length 1, step 4, samepadding], d<number of channels 4≤k. Normal data decoder D norm It contains one fully connected layer and three deconvolution layers (the parameter format is the same as the convolution layer mentioned in step 2). The output dimension of the fully connected layer is k, and the parameter sizes of the three deconvolution layers are: deconvolution layer 1 is [number of channels 2, width 3, padding type 3]; deconvolution layer 2 is [number of channels 1, width 2, padding type 2]; deconvolution layer 3 is [number of input channels, width 1, padding type 1]. Abnormal data decoder D anom Contains 3 deconvolution layers with parameter size and D norm The three deconvolutional layers included have the same size.

[0107] Step 3: During actual training, in each epoch (an epoch represents a training pass through the training data set), a batch of data (hereinafter referred to as batch data) is input each time. The following description takes single data input as an example, and then gives the size of the features obtained by batch processing. After the three convolutional layers of the encoder, the resulting size is [l1, l1, 128]. The calculation steps of l1 are as follows: Figure 5 As shown. Flatten the result to get the k-dimensional potential feature vector z = E (x), k = l1 2 *128.

[0108] Step 4: Use the RSR layer to get z′=Az, normalize z′ to get the normal essential features of the data; while keeping the parameters of z unchanged, change the size of z to [k, 1], and the output of the noise extractor is s=F(z), where the size of s is [k, number of channels 4]. According to the direction of the channel, average s to get the k-dimensional abnormal features s of the data. When the actual input is batch data, C z′s are composed of C s composition

[0109] Step 5: Input z′ into D norm Get a normal reconstructed copy x of the original data norm ; Input s into D anom Get an abnormally reconstructed copy x of the original data anom .

[0110] Step 6: Use a two-stage process to train model parameters. There are 6 optimizers in the training process, all of which use Adam. Their names and optimization ranges are:

[0111] [1]Optimizer_ae: optimizes all parameters of the model, using It works when back propagation is performed;

[0112] [2]Optimizer_rsr1: Optimizes the RSR layer, using It works when back propagation is performed;

[0113] [3]Optimizer_rsr2: Optimizes the RSR layer, using It works when back propagation is performed;

[0114] [4]Optimizer_rae1: Optimizes the RSR layer and noise extractor, using It works when back propagation is performed;

[0115] [5]Optimizer_rae2: Optimize the RSR layer, using It works when back propagation is performed;

[0116] [6]Optimizer_rae3: Optimizes the noise extractor, using It works when back propagation is performed;

[0117] Step 7: After the model training is completed, calculate the two reconstruction errors of the test data, the normal reconstruction error re norm =||xx norm ||2, the abnormal reconstruction error is re anom =||xx anom ||2.

[0118] Step 8: Determine whether the test data is abnormal.

[0119] Example 2

[0120] The following are the details:

[0121] Step 1: Given an unlabeled feature dataset, where the number of samples is N, a single data is a vector composed of multiple features, and the dimension of the vector needs to be unified, with a dimension of K. Normalize the dataset according to different situations. If the values ​​of the original data are all greater than or equal to 0, limit the value range to [0,1], otherwise limit it to [-1,1]. Given the batch size C required for training. Calculate the floating-point result of N / C, round the result up to B, and divide the dataset into B equal parts. Except for the RSR layer, subsequent model components all use activation functions. The RSR layer is a linear layer and does not use any activation function. If the normalized value range is [0,1], the activation function uses ReLU, otherwise the tanh function activation function is used; except for the RSR layer and the last layer of the two decoders, the outputs of other components need to be batch normalized during training.

[0122] Step 2: Encoder E contains three fully connected layers, and the output dimensions of the three fully connected layers are: dimension 1, dimension 2, and dimension 3. Dimension 1 < dimension 2 < dimension 3. The RSR layer is introduced, and its parameter matrix is d<<k; introduce the noise extractor F, which is a one-dimensional convolution layer. Its parameter format is [number of channels, convolution kernel length, moving step, padding type], and its specific parameters are [number of channels 1, length 1, step 4, same padding], and the number of channels 1≤k. Normal data decoder D norm Contains 4 fully connected layers, with output dimensions: k, dimension 2, dimension 1, K; abnormal data decoder D anom It contains 3 fully connected layers, and the output dimensions are: dimension 2, dimension 1, and K.

[0123] Step 3: During actual training, a batch of data (hereinafter referred to as batch data) is input each time. The following description takes single data input as an example, and then gives the feature size obtained by batch processing. Through the three fully connected layers of the encoder, the dimension of the potential feature vector z = E(x) is k = 3.

[0124] Step 4: Through the RSR layer, we get z′=Az, normalize z′ to get the normal essential features of the data; while keeping the parameters of z unchanged, change the size of z to [k, 1], and the output of the noise extractor is s=F(z), where the size of s is [k, number of channels 4]. According to the direction of the channel, s is averaged to get the k-dimensional abnormal features s of the data. When the actual input is batch data, C z′s are composed of C s composition

[0125] Step 5: Input z′ into D norm Get a normal reconstructed copy x of the original datanorm . Input s into D anom Get an abnormally reconstructed copy x of the original data anom .

[0126] Step 6: Use a two-stage process to train model parameters. There are 6 optimizers in the training process, all of which use Adam. Their names and optimization ranges are:

[0127] [1]Optimizer_ae: optimizes all parameters of the model, using It works when back propagation is performed

[0128] [2]Optimizer_rsr1: Optimizes the RSR layer, using It works when back propagation is performed

[0129] [3]Optimizer_rsr2: Optimizes the RSR layer, using It works when back propagation is performed

[0130] [4]Optimizer_rae1: Optimizes the RSR layer and noise extractor, using It works when back propagation is performed

[0131] [5]Optimizer_rae2: Optimizes the RSR layer, using It works when back propagation is performed

[0132] [6]Optimizer_rae3: Optimizes the noise extractor, using It works when back propagation is performed

[0133] Step 7: After the model training is completed, calculate the two reconstruction errors of the test data, the normal reconstruction error re norm =||xx norm ||2, the abnormal reconstruction error is re anom =||xx anom ||2.

[0134] Step 8: Determine whether the test data is abnormal.

[0135] The encoder and decoder of the present invention can be replaced by RNN architecture or transformer architecture, etc.;

[0136] The anomaly score is not limited to the reconstruction error, but can also be other metrics such as cosine similarity that can measure the similarity between data.

[0137] Anomaly scores are not limited to using the original data x and the normal reconstructed copy x norm To perform calculations, you can also use x and xnorm , x, and x anom , x, and x norm +x anom , z and E(x norm ), z and E(x anom ), z and E(x norm +x anom ) and other symmetrical data in the model.

[0138] Threshold-free anomaly detection can replace the reconstruction error with other metrics, and can use multiple combination results for voting and weighted fusion to perform threshold-free anomaly detection.

Claims

1. An unsupervised anomaly detection method based on latent feature decomposition, characterized in that: The specific steps are as follows: for the given raw data to be detected, first preprocess the raw data into a uniform size and construct an unlabeled training data set; Then, a feature decomposition autoencoder FDAE including an encoding module, a decoding module and a feature decomposition module is constructed according to the data type to be detected; the encoding module includes an encoder, the feature decomposition module includes an RSR layer and a noise extractor, and the decoding module includes a normal data decoder and an abnormal data decoder; Use the encoder to transform the original data into the latent space to obtain the latent features of the original data; Using the RSR layer to extract the normal essential features Az of the data from the latent features (n) , using the noise extraction module to extract the abnormal features of the data from the latent features (n) ; and use the normal data decoder and abnormal data decoder to decode the normal essential features of the data Az (n) and abnormal features (n) Decode them separately to get a normal copy and abnormal copies Next, use the normal copy and abnormal copies Combine the two-stage training strategy with the unlabeled dataset to train FDAE until the predetermined number of training rounds is reached; Finally, the new test data is input into the trained FDAE, and the reconstruction errors between the output of the normal decoder and the output of the abnormal decoder and the test data are calculated, which are recorded as the normal reconstruction error and the abnormal reconstruction error, respectively, to determine whether the test data is abnormal.

2. The unsupervised anomaly detection method based on latent feature decomposition according to claim 1, characterized in that: The potential features of the original data obtained by using the encoder are specifically: Encoder E converts the original data Mapped to a high-dimensional k-dimensional latent space, the potential features in this space are recorded as N represents the number of training data, K represents the dimension of the original vector data, and k represents the dimension of the encoder's output z.

3. The unsupervised anomaly detection method based on latent feature decomposition according to claim 1, characterized in that: The loss function L of the RSR layer in the feature decomposition module RSR There are two loss functions and The specific calculation is: Where λ1 and λ2 are hyperparameters; A T Represents the parameter matrix of the RSR layer The transposed matrix, I d Represents a single matrix of d×d, considering A as the encoder of the latent features, A T Decoder considered as latent features; The noise extractor F is used to extract abnormal information s from the latent features. (n) , that is, s (n) =F(z (n) ) and use it as an abnormal feature.

4. The unsupervised anomaly detection method based on latent feature decomposition according to claim 1, characterized in that: The normal data decoder D norm For normal essential characteristics Az (n) Decode and get the original data x (n) A normal copy of the same shape Abnormal Data Decoder D anom For abnormal features (n) Decode and get x (n) Anomalous copies of the same shape 5. The unsupervised anomaly detection method based on latent feature decomposition according to claim 1, characterized in that: The two-stage training strategy includes: in the first stage, the parameters of AE1 are updated, where AE1 consists of encoder E, RSR layer A and normal data decoder D norm In the second stage, the normal data decoder D is fixed norm Parameters of A and AE2 are updated; AE2 consists of encoder E, noise extractor F and abnormal data decoder D anom composition; The specific process is: In the first stage, the autoencoder loss function of AE1 is for: Combined with the RSR loss function, the overall objective function of FDAE at this stage is: In the actual update step, this stage backpropagates three expressions in sequence: The expressions related to the RSR loss function are only used to update the RSR layer; In the second stage, the autoencoder loss function of AE2 is: Combined with the robust autoencoder RAE loss function, the overall objective function of FDAE at this stage is: The RAE loss function refers to the loss function that combines the noise extractor and the RSR layer. The calculation formula is as follows: where μ is a hyperparameter, In the actual update step, this stage backpropagates 4 expressions in sequence: in Only used to update the RSR layer and noise extractor, Only used to update the RSR layer, Only used to update the noise extractor; The updated values ​​in the second stage are multiplied by a scaling factor m = (1-1 / ep), where ep is the epoch of training.

6. The unsupervised anomaly detection method based on latent feature decomposition according to claim 1, characterized in that: The reconstruction error is calculated as follows: (n) , normal reconstruction error The abnormal reconstruction error is 7. The unsupervised anomaly detection method based on latent feature decomposition according to claim 1, characterized in that: There are two strategies for judging whether the test data is abnormal: 1) considering the normal reconstruction error as an abnormal score, and if the abnormal score is greater than a preset threshold, the test data is judged as abnormal; 2) If the normal reconstruction error is greater than the abnormal reconstruction error, the test data is judged as abnormal.

Images