An unsupervised anomaly detection method of memory-augmented adversarial autoencoder

Abstract

The generalization ability of autoencoder is not enough to separate the anomaly from the normal data. It is affected by the spatial locality of CNN, and the poor quality of feature representation leads to poor reconstruction. The feature is not well used for anomaly detection, which affects the data reconstruction. This paper proposes a memory-enhanced adversarial autoencoder for anomaly detection to improve the quality and utilization of features. We propose a "CNN-Trans" block as the backbone, inheriting the advantages of CNN and Transformer in feature representation. Then we build a memory-enhanced adversarial autoencoder. Specifically, the memory module with multiple measurements aims to suppress the generator generalization by cascading multivariate features. A dual-view discriminator is proposed to distinguish the anomaly in the image and latent space. In addition, a generator-discriminator latent loss is proposed to improve the performance of the generator by minimizing the encoding process and the discrimination process. Extensive experiments are conducted on multiple datasets. Compared with other models, the proposed model exhibits superior performance.

Description

Technical Field

[0001] This invention pertains to unsupervised anomaly detection methods and relates to the field of computer vision technology. Background Technology

[0002] To address the detection problem of complex data, reconstruction-based methods have been proposed, arguing that anomalies cannot be accurately represented and reconstructed using models learned only on normal data. For example, Principal Component Analysis (PCA) has been used to learn the feature distribution of normal samples; sparse representation methods jointly learn a data dictionary and sparse representation features of normal samples to better reconstruct positive samples. However, the limited feature representation restricts the performance of these methods. In recent years, based on deep learning tasks in computer vision, several anomaly detection algorithms based on Convolutional Autoencoder (AE) networks have emerged; for instance, deep AEs based on structured energy are used to model the trained samples, jointly encoding vectors and reconstruction errors. Although AE-based models expand the feature representation of samples, the strong generalization ability of AEs sometimes allows them to reconstruct negative samples well, limiting the pace of anomaly detection. To address this issue, MemAE, based on memory networks, was proposed. It amplifies the gap between the reconstructed negative sample data and the original sample, thus explicitly suppressing the generalization ability of the autoencoder. In 2020, Park et al. improved the storage mechanism based on MemAE, designing a new memory-addressing module to suppress the generalization ability of AEs. For a time, refactoring-based approaches achieved great success, but their performance was limited by insufficient design of the underlying space.

[0003] To address these issues, GANs have been applied to anomaly detection. GANs can simulate complex high-dimensional distributions, especially image distributions, through a mini-maximum game process, and the models can successfully generate data with excellent performance. Schlegl et al. proposed AnoGAN, which uses a convolutional structure similar to DCGAN to build a generator trained only on normal images. Later, to reduce the complexity of mapping from images to the latent space, Zenati et al. effectively utilized BiGAN to distinguish anomalies with the same anomaly score. Akcay et al., not satisfied with the existing encoder-decoder structure for reconstruction, proposed GANomaly, which designed an encoder-decoder-encoder network to explore the deep latent representations of normal samples and calculate anomaly scores by the difference between the features of the reconstructed image and the original image. In 2019, Samet et al. proposed Skip-GANomaly based on GANomaly, which combines Unet with GANs to enhance the generator's learning ability. Later, Zaheer et al. proposed an end-to-end testing OGNet model, changing the discriminator's responsibility from distinguishing between true and false to judging the quality of reconstruction; the discriminator's output was used as the anomaly detection score. Although GANs' powerful training capabilities have contributed significantly to anomaly detection, both AE (Anomaly Detection) and GANs are built upon CNNs as their underlying networks. CNNs, based on convolutional kernels, are excellent at extracting local details, but their ability to extract global information is insufficient. To better extract global feature information from images, the Transformer architecture has become increasingly popular among researchers.

[0004] The Transformer is a deep neural network primarily based on self-attention mechanisms, initially applied in natural language processing. It incorporates multi-head attention mechanisms and fully connected feedforward networks. Researchers have increasingly applied it to computer vision to enhance its ability to extract global features from images. The Vision Transformer was the first Transformer architecture applied to images. It simulates word input in sentences by slicing images and uses multi-head attention to grasp global features; the Image Pyramid Transformer extracts image features in different dimensions for complex and dense tasks. The Swin-Transformer is comparable to convolutional neural networks, using CNNs to extract features through dimensional transformations according to certain rules and employing local windows for feature computation. Subsequently, some researchers have also used Transformers instead of GANs in generative tasks. For example, the TransGAN architecture was the first generative adversarial network built using Transformers. Recently, some researchers have argued that using only the Transformer structure may not achieve optimal results, leading them to plan the combination of CNN and Transformer. BoTNet incorporates self-attention into various computer vision tasks to significantly improve performance across these tasks. Southeast University proposed the ConvTransformer structure for video frame sequence learning and synthesis. TransUnet, an attempt in medical segmentation, combines Unet and Transformer structures. On one hand, the Transformer encodes tokenized image patches from CNN feature maps into input sequences for extracting global context. On the other hand, the decoder upsamples the encoded features and combines them with high-resolution CNN feature maps for accurate localization. Currently, researchers are still dedicated to developing more effective Transformer structures to make them easier to apply in various fields. In anomaly detection, to improve the CNN's grasp of global information, we decided to apply the Transformer, combining CNN and Transformer to retain the CNN's extraction of local information while incorporating the Transformer's focus on global information. Summary of the Invention

[0005] The purpose of this invention is to address the problem that previous unsupervised anomaly detection algorithms mostly only compared and reconstructed samples with the original samples in the image space, rarely considering the feature extraction and utilization of both in the latent space.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] S1. Construct an adversarial autoencoder network based on CNN-Transformer to extract key information from both global and local perspectives.

[0008] S2. A multi-scale memory module is added between the encoder and decoder in S1 to record positive sample information.

[0009] S3. Modify the original discriminator in S1 into a dual-view discriminator, which distinguishes between normal and abnormal from the image space and the feature space.

[0010] S4. Combining the networks from S1 and S2 with the modules from S3, an unsupervised anomaly detection method using a memory-reinforced adversarial autoencoder is constructed. Finally, the constructed model is trained.

[0011] First, the adversarial autoencoder generator comprises encoder 1, decoder 1, and encoder 2, built upon a deep convolutional network. Compared to the encoder-decoder structure, the adversarial autoencoder adds encoder 2 to obtain the encoded features of the reconstructed image. We minimize the encoded features of the reconstructed image compared to the encoded features of the original image to control the intermediate processes of the encoder and decoder from the feature latent space, thereby enhancing the generator's reconstruction capability. This process can be represented as:

[0012] Z e1 =f e1 (X;θ e1 (1)

[0013] X rec =f d (Z e1 ;θ d (2)

[0014] Z e2 =f e2 (X rec ;θ e2 (3)

[0015] Where X represents the positive input sample, Z e1 It is the encoded feature X obtained through encoder e1. rec Z represents the reconstructed image. e2 These are the reconstructed features after encoder e2. e1 and f e2 There are two encoder encoding processes, f d This refers to the decoding process. θ e1 θ d θ e2 These are intermediate parameters used in encoding and decoding.

[0016] Because CNNs tend to overlook global features in positive sample images, CNN-Transformer (CNN-Trans) is used to replace the original CNN to improve the network's global representation. Each "CNN-Trans" block contains two parts. On one hand, a CNN network is used to extract multidimensional local features. On the other hand, window-based self-attention (W-MSA) and sliding window-based self-attention (SW-MSA) are used to extract global representations. Furthermore, the MLP built into the Transformer is removed to shorten training time. We will x i The input ∈X is first fed into a CNN network (Conv), and then passed through the BatchNorm (BN) and LeakyReLU (LR) functions to obtain local features z. i ∈Z. This process can be represented as:

[0017] z i =LR(BN(Conv(x) i (4)

[0018] After z i The input ∈Z is fed into the Transformer module, which consists of two parts. Each part contains LayerNorm (LN) and multi-head attention using either W-MSA or SW-MSA. Notably, the outputs from each part are concatenated to the output of the previous part. Finally, we obtain the features extracted by the "CNN-Trans" block. This process can be represented as:

[0019]

[0020]

[0021]

[0022] Z W-MSA and Z SW-MSA These represent window-based attention and sliding window attention, respectively.

[0023] The fundamental idea behind anomaly detection is that samples cannot be well reconstructed from a model that has only learned from positive samples. Therefore, while reconstructing positive samples, it is necessary to suppress the ability to reconstruct negative samples. Traditional Advanced Feature Analyzers (AEs) have strong generalization capabilities, and they can sometimes reconstruct negative samples very well, severely impacting the detection performance of anomalies. To suppress the generalization capability of AEs, a multi-scale memory module based on an attention mechanism is used for memorizing positive sample features.

[0024] This module is positioned between the encoder and the decoder, and its input is the encoded features of encoder e1. A memory slot Mem of size N is used to store positive sample features. First, cosine similarity and Euclidean distance are used to calculate Z respectively. e1 With m j Similarity d of ∈Mem dus and d cos The similarity weights are then normalized using the Softmax() function. These two calculation methods consider the similarity from both qualitative and quantitative perspectives, respectively, resulting in more accurate weights. This leads to two similarity weight matrices. and Then, a weight q is set to connect the matrices so that they are combined into a single similarity weight. This process can be represented as:

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] However, anomalies can be well reconstructed through complex combinations of stored items because some smaller weights still exist after Softmax(), allowing them to capture anomaly information during computation. Therefore, the hard shrinkage operator proposed in MemAE is used to shrink smaller weights, highlighting the role of larger weights. This results in an updated weight matrix. Then With the features m in the memory slot j Multiplication yields the memory-enhanced feature Z. memory It is worth noting that when faced with a large number of positive sample classes, Z... memory Often, more attention is paid to the positive sample information in the memory slot, while the original feature information is ignored. This results in a large difference between the reconstructed positive sample and the original image, which is detrimental to the training of unsupervised anomaly detection models. Therefore, in order to make the model more adaptable to multi-class positive samples, Z... memory The feature Z used in the cascaded original encoder e1 e1 To obtain the encoded feature Z1 after passing through the memory module, the process can be represented as:

[0031]

[0032]

[0033] Z1 = concat(Z memory Z e1 (15)

[0034] Where a and ε represent parameters. This is the updated weight matrix after hard contraction. `concat()` represents concatenation. `B` represents batch.

[0035] The dual-view discriminator consists of two parts: an image discriminator and a feature discriminator. The image discriminator adds a Sigmoid() function to the encoder e1. The feature discriminator incorporates the penultimate CNN layer of encoder e1 and the Sigmoid() function; it is used to discriminate the intermediate features of the penultimate layer output by encoders e1 and e2. For the input original image X and the reconstructed image X... rec The intermediate features F of encoder e1 e1 The intermediate feature F of e2 e2 The dual-view discriminator will obtain four scores: the score S of the original image and the score S of the reconstructed image. x S x_rec And the probabilities S of the two intermediate features f1 and S f2 When training the discriminator, positive samples are treated as 1s and reconstructed samples as 0s. This process can be represented as:

[0036]

[0037] Where p represents the parameter set, and BCE() represents the nn.BCELoss() function. Loss dis Let be the loss function of the discriminator.

[0038] For the input original positive sample image X, the original encoded features Z are first obtained by encoder e1. e1 Then, it is fed into a multi-scale memory module to obtain memory-enhanced coding features Z1, and Z1 is decoded to obtain the reconstructed image X. rec Then use encoder e2 to input X. rec Encoding yields the reconstructed feature Z e2 Combine the intermediate features of the two encoders with X rec X is input to the discriminator for discrimination. The training process includes five loss functions.

[0039] Image contrast loss minimization positive sample reconstruction and normal data x i The distance between ∈X is used so that the model can fully learn the normal pattern. It can be represented as:

[0040]

[0041] The encoded features obtained by minimizing the encoding loss of encoders e1 and e2 and The distance between them is used so that the generator can learn how to encode features for generating images from normal data. It can be represented as:

[0042]

[0043] Adversarial loss is used to minimize the discriminator's discrimination process f dis Original image features f in () i e1 ∈F e1 and reconstructed image features f i e2 ∈F e2 The difference between them. It allows the generator to deceive the discriminator with the generated samples from the feature space. It can be represented as:

[0044]

[0045] To further improve sparsity, a memory loss is used to minimize the sparse regularization matrix during training. It can be represented as:

[0046]

[0047] The generator-discriminator latent loss aims to minimize the features in the encoder e1 and the penultimate layer of the discriminator. The encoding loss and adversarial loss minimize the encoding and discrimination processes, respectively, which are performed separately in the generator or discriminator. The generator-discriminator latent loss connects them, continuously narrowing the four intermediate processes. This makes it impossible for the discriminator to distinguish between real and fake data from the feature space when training the generator, thus improving the generator's performance. It can be expressed as:

[0048]

[0049] In summary, the five losses of the generator together constitute the generator loss, which can be expressed as:

[0050] Loss G =τ×(loss) adv +loss enc +loss G-D )+β×loss con +γ×loss mem (twenty two)

[0051] Furthermore, traditional testing methods obtain results through image-level comparison, which is computationally intensive, especially for large images. Therefore, feature-level comparison is used to replace image-level comparison to reduce computational cost. It can be expressed as:

[0052]

[0053] Here, E1() and E2() represent the encoding processes of encoders e1 and e2. S represents the score of the anomaly, which is regularized between [0, 1].

[0054] Finally, the training details of the memory-enhanced adversarial autoencoder are as follows:

[0055] We primarily implemented the model using PyTorch. First, we redefined the dataset images to a uniform size of 32×32 and set the iteration batch size (B) to 64 / 128 for experiments. Then, we built a CNN convolution process model based on DCGAN. Each step of the CNN feature extraction process requires three steps: Conv2(), BatchNorm(), and LeakyRelu(). The decoder uses deconvolution DConv2(), which also requires BatchNorm() and LeakyRelu(). During the CNN design, the image dimensions change as follows: (32, 32, 3) → (16, 16, 64) → (8, 8, 128) → (4, 4, 256) → (1, 1, Z). The size of Z affects the experimental performance of the model. In MNIST, Z is generally set to 100, while in CIFAR10, it is usually set to 200. Then, we designed the Transformer structure based on the Swin-Transformer. To accommodate the CNN network structure, we set the input dimension to change from 64 to 128 to 256 at each step in the W-MSA. In the Transformer module, we set the head in the multi-head attention mechanism. dim =32, with Layers set to 2 in each CNN-TRANS module, meaning the attention mechanism is used once per step; during image dimension changes, the head_size in the attention mechanism changes from 4→8→16, and the window_size is set to 4. When designing the sample feature memory module, we set the memory slots to store N=10 positive sample features; subsequent experimental results showed that different weights of q, p, and N have a certain impact on the experimental results. Therefore, in the MNIST dataset, q=0.8; p=0.1, N=10, and in the CIFAR10 dataset, q=0.5, p=0.2, N=10; when recalculating the sample feature weights using the hard shrinkage operator, we set a=1 / N=0.1, ε=1e-15, with ε primarily to prevent a denominator of 0; finally, Adam is used for backpropagation, trained at a learning rate lr=0.0002.

[0056] Compared with existing technologies, the beneficial effects of this invention are:

[0057] 1. This invention improves the quality of feature representation from both local and global perspectives by taking into account the advantages of CNN and Transformer, and proposes the "CNN-Trans" block as the backbone of an adversarial autoencoder for anomaly detection.

[0058] 2. This invention improves the utilization of latent features in adversarial autoencoders for anomaly detection. It utilizes a multi-scale memory module in the generator to suppress generalization through cascaded multivariate features. This invention proposes a dual-view discriminator to distinguish between images and reconstructed data in the latent space, achieving high-quality data reconstruction.

[0059] 3. This invention proposes the GD latent loss, which minimizes the encoding and discrimination processes, making the discriminator unable to make a decision when training the generator, thereby improving the generator's performance in high-quality reconstruction. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the memory-enhanced adversarial V-encoder.

[0061] Figure 2 This is a schematic diagram of the global and local feature extraction process.

[0062] Figure 3 and Figure 4 The graph shows a comparison of the results of the memory-enhanced adversarial autoencoder-based unsupervised anomaly detection method with other anomaly detection methods on the CIFAR10 and MNIST datasets.

[0063] Figure 5 and Figure 6 A comparison of the results of an unsupervised anomaly detection method based on memorization-enhanced adversarial autoencoders on the STL10 and CIFAR100 datasets with baseline results.

[0064] Figure 7 Results of the unsupervised anomaly detection method based on memory-enhanced adversarial autoencoders on the MVTec AD dataset.

[0065] Figure 8 Visualization of the unsupervised anomaly detection method based on memory-enhanced adversarial autoencoders on the MVTec AD and MNIST datasets. Detailed Implementation

[0066] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0067] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0068] Figure 1 This is a schematic diagram of a memory-enhanced adversarial autoencoder. Figure 1As shown,

[0069] First, encoder e1 extracts latent encoded features from the input positive samples, then inputs these features into a multi-scale memory module to obtain memory-enhanced features. The features are then decoded to obtain a reconstructed image, which is further encoded by encoder e2 to obtain the reconstructed features. Simultaneously, the original samples, the reconstructed image, and the intermediate features from encoders e1 and e2 are all fed into the proposed dual-view discriminator to jointly determine whether a sample is true or false. This process can be represented as:

[0070] Z e1 =f e1 (X;θ e1 (1)

[0071] X rec =f d (Z e1 ;θ d (2)

[0072] Z e2 =f e2 (X rec ;θ e2 (3)

[0073] Where X represents the positive input sample, Z e1 It is the encoded feature X obtained through encoder e1. rec Z represents the reconstructed image. e2 These are the reconstructed features after encoder e2. e1 and f e2 There are two encoder encoding processes, f d This refers to the decoding process. θ e1 θ d θ e2 These are intermediate parameters used in encoding and decoding.

[0074] Because CNNs tend to overlook global features in positive sample images, CNN-Transformer (CNN-Trans) is used to replace the original CNN to improve the network's global representation. Each "CNN-Trans" block contains two parts. On one hand, a CNN network is used to extract multidimensional local features. On the other hand, window-based v-attention (W-MSA) and sliding window-based self-attention (SW-MSA) are used to extract global representations. This process can be represented as:

[0075] z i =LR(BN(Conv(x) i (4)

[0076] After z iThe input ∈Z is fed into the Transformer module, which consists of two parts. Each part contains LayerNorm (LN) and multi-head attention using either W-MSA or SW-MSA. The outputs from each part are concatenated to the output of the previous part. Finally, we obtain the features extracted by the "CNN-Trans" block. This process can be represented as:

[0077]

[0078]

[0079]

[0080] Z W-MSA and Z SW-MSA These represent window-based attention and sliding window attention, respectively.

[0081] A multi-scale memory module based on an attention mechanism is used for memorizing positive sample features. This module is placed between the encoder and decoder, and its input is the encoded features of encoder e1. A memory slot Mem of size N is used to store positive sample features. First, cosine similarity and Euclidean distance are used to calculate Z respectively. e1 With m j Similarity d of ∈Mem dis and d cos The similarity weights are then normalized using the Softmax() function. These two calculation methods consider the similarity from both qualitative and quantitative perspectives, respectively, resulting in more accurate weights. This leads to two similarity weight matrices. and Then, a weight q is set to connect the matrices so that they are combined into a single similarity weight. This process can be represented as:

[0082]

[0083]

[0084]

[0085]

[0086]

[0087] However, anomalies can be well reconstructed through complex combinations of stored items because some smaller weights still exist after Softmax(), allowing them to capture anomaly information during computation. Therefore, the hard shrinkage operator proposed in MemAE is used to shrink smaller weights, highlighting the role of larger weights. This results in an updated weight matrix. Then With the features m in the memory slot j Multiplication yields the memory-enhanced feature Z. memory When faced with a large number of positive sample categories, Z memory Often, more attention is paid to the positive sample information in the memory slot, while the original feature information is ignored. This results in a large difference between the reconstructed positive sample and the original image, which is detrimental to the training of unsupervised anomaly detection models. Therefore, in order to make the model more adaptable to multi-class positive samples, Z... memory The feature Z used in the cascaded original encoder e1 e1 To obtain the encoded feature Z1 after passing through the memory module, the process can be represented as:

[0088]

[0089]

[0090] Z1 = concat(Z memory Z e1 (15)

[0091] Where a and ε represent parameters. This is the updated weight matrix after hard contraction. `concat()` represents concatenation. `B` represents batch.

[0092] The dual-view discriminator consists of two parts: an image discriminator and a feature discriminator. The image discriminator adds a Sigmoid() function to the encoder e1. The feature discriminator incorporates the penultimate CNN layer of encoder e1 and the Sigmoid() function; it is used to discriminate the intermediate features of the penultimate layer output by encoders e1 and e2. For the input original image X and the reconstructed image X... rec The intermediate features F of encoder e1 e1 The intermediate feature F of e2 e2 The dual-view discriminator will obtain four scores: the score S of the original image and the score S of the reconstructed image. x S x_rec And the probabilities S of the two intermediate features f1 and S f2 When training the discriminator, positive samples are treated as 1s and reconstructed samples as 0s. This process can be represented as:

[0093]

[0094] Where p represents the parameter set, and BCE() represents the nn.BCELoss() function. Loss dis Let be the loss function of the discriminator.

[0095] The generator training process includes five loss functions. The first is the image contrast loss, which minimizes positive sample reconstruction. and normal data x i The distance between ∈X is used so that the model can fully learn the normal pattern. It can be represented as:

[0096]

[0097] The encoded features obtained by minimizing the encoding loss of encoders e1 and e2 and The distance between them is used so that the generator can learn how to encode features for generating images from normal data. It can be represented as:

[0098]

[0099] Adversarial loss is used to minimize the discriminator's discrimination process f dis Original image features f in () i e1 ∈F e1 and reconstructed image features f i e2 ∈F e2 The difference between them. It allows the generator to deceive the discriminator with the generated samples from the feature space. It can be represented as:

[0100]

[0101] To further improve sparsity, a memory loss is used to minimize the sparse regularization matrix during training. It can be represented as:

[0102]

[0103] The generator-discriminator latent loss aims to minimize the features in the encoder e1 and the penultimate layer of the discriminator. The encoding loss and adversarial loss minimize the encoding and discrimination processes, respectively, which are performed separately in the generator or discriminator. The generator-discriminator latent loss connects them, continuously narrowing the four intermediate processes. This makes it impossible for the discriminator to distinguish between real and fake data from the feature space when training the generator, thus improving the generator's performance. It can be expressed as:

[0104]

[0105] In summary, the five losses of the generator together constitute the generator loss, which can be expressed as:

[0106] Loss G =τ×(loss) adv +loss enc +loss G-D )+β×loss con +γ×loss mem (twenty two)

[0107] Furthermore, traditional testing methods obtain results through image-level comparison, which is computationally intensive, especially for large images. Therefore, feature-level comparison is used to replace image-level comparison to reduce computational cost. It can be expressed as:

[0108]

[0109] Here, E1() and E2() represent the encoding processes of encoders e1 and e2. S represents the score of the anomaly, which is regularized between [0, 1].

[0110] Figure 2 This is a schematic diagram illustrating the global and local feature extraction process. (Example) Figure 2 As shown, each local feature block contains features from its neighboring pixel blocks, reflecting the spatial locality of CNNs. Then, attention is computed within each window to ensure globality within the window. Finally, multi-head attention is computed again through a sliding window to enhance the interaction between windows, thus constructing a global extraction method for the entire feature map. This demonstrates that this module can extract more comprehensive feature information.

[0111] Figure 3 and Figure 4 The figure shows a comparison of the results of the memory-enhanced adversarial autoencoder-based unsupervised anomaly detection method with other anomaly detection methods on the CIFAR10 and MNIST datasets. As shown in the figure, the module of this invention achieves higher accuracy on CIFAR10 and MNIST than other models, and guarantees the best results in most categories.

[0112] Figure 5 and Figure 6 The graph shows a comparison of the results of the unsupervised anomaly detection method based on memorized reinforcement adversarial autoencoders on the CIFAR100 and STL10 datasets with the baseline. Figure 5 As shown, this invention ensures that our model significantly outperforms the baseline in most categories of STL10, exceeding the baseline in both maximum and average values. Figure 6As shown, the model results of this invention significantly outperform the baseline results on CIFAAR100, both in terms of maximum value and class average. The comparison across the four datasets fully demonstrates the superiority of the model.

[0113] Figure 7 Experiments on the MVTec AD dataset showcasing an unsupervised anomaly detection method based on a memory-enhanced adversarial autoencoder. Figure 7 As shown, compared with other methods mentioned, this invention achieves excellent average AUC results on both texture and object classes. These experimental results fully demonstrate the effectiveness of the proposed method for anomaly detection.

[0114] Figure 8 The image shows the visualization results of the unsupervised anomaly detection method based on the memory-enhanced adversarial autoencoder on the MVTec AD and MNIST datasets. Figure 8 As shown, on the MVTec AD dataset, the generated images do not contain the defects present in the input images, indicating that our model can learn normal patterns to effectively identify and repair anomalies. Furthermore, we found that when the anomaly category is "9", some generated images from MNIST have shifted towards the other category "8", demonstrating that our model increases the distance between the anomalous images and the reconstructed images.

[0115] This invention proposes an unsupervised anomaly detection method using a memory-enhanced adversarial autoencoder. This method leverages the "CNN-Trans" block to effectively extract global and local features, ensuring high-quality reconstruction. To improve the utilization of latent features, a multi-scale storage module is employed to record normal patterns, promoting the input data to closely approximate normal data. A dual-view discriminator aims to distinguish anomalies from the feature space. Both focus on anomaly variations in the latent space for accurate detection. Furthermore, during generator training, GD latent loss is utilized to enhance generation performance and suppress discriminative power in the latent space. Extensive experiments fully demonstrate the superiority of this invention.

[0116] Finally, the details of the above examples of the present invention are merely illustrative of the invention. Any modifications, improvements, and substitutions to the above embodiments by those skilled in the art should be included within the scope of protection of the claims of the present invention.

Claims

1. An unsupervised anomaly detection method using a memory-reinforced adversarial autoencoder, characterized in that, The method includes the following steps: S1. Construct an adversarial autoencoder network based on CNN-Transformer to extract key information from both global and local perspectives; the specific process of S1 is as follows: The adversarial autoencoder generator comprises encoder e1, decoder 1, and encoder e2, built upon the CNN-Transformer. It minimizes the encoded features of the reconstructed image compared to the encoded features of the original image to control the intermediate processes of the encoder and decoder from the feature latent space, thereby enhancing the generator's reconstruction capability. This process is represented as: （1）； （2）； （3）； in Represents the positive input sample. It is the encoded feature of encoder e1. Represents the reconstructed image. These are the reconstructed features after encoder e2; and There are two encoder encoding processes. For the decoding process; , , These are intermediate parameters used in encoding and decoding; Each "CNN-Transformer" block contains two parts: on the one hand, a CNN network is used to extract multidimensional local features; on the other hand, window-based self-attention and sliding window-based self-attention are used to extract global representations. Furthermore, the MLP built into the Transformer is removed to shorten training time; x i The input ∈X is first fed into a CNN network, and then the local features z are obtained after passing through the BatchNorm and LeakyReLU functions. i ∈Z; This process is represented as: （4）； after The input is fed into the Transformer module, which consists of two parts; each part contains multi-head attention with LayerNorm and W-MSA or LayerNorm and SW-MSA; the outputs from each part are concatenated to the output of the previous part; finally, the features extracted by the "CNN-Transformer" block are obtained. The process is represented as: （5）； （6）； （7）； in and These represent window-based attention and sliding window attention, respectively. ; S2. A multi-scale memory module is added between the encoder and decoder in S1 to record positive sample information; the specific process of S2 is as follows: This module is positioned between the encoder and the decoder, and its input is the encoded features of encoder e1. A memory slot Mem of size N is used to store positive sample features; firstly, cosine similarity and Euclidean distance are used to calculate... and similarity and The similarity weights were then normalized using the Softmax() function. The two calculation methods considered the similarity from both qualitative and quantitative perspectives, respectively, to make the weights more accurate. This resulted in two similarity weight matrices. and Then, a weight q is set to connect the matrices and combine them into a single similarity weight. The process is represented as: （8）； （9）； （10）； （11）； （12）； The hard shrinkage operator proposed in MemAE is used to shrink smaller weights, highlighting the role of larger weights; thus obtaining an updated weight matrix. ;Then With the features in the memory slot Multiplication yields features enhanced by memory. ; Features used in the cascaded original encoder e1 To obtain the encoded features after memory module The process is represented as: （13）； （14）； （15）； in , Representative parameters, It is the weight matrix updated using hard shrinkage; concat() represents concatenation; B represents batch. S3. The dual-view discriminator distinguishes between normal and abnormal from the image space and the feature space; S4. Combine the networks in S1 and S2 with the modules in S3 to construct an unsupervised anomaly detection model using a memory-reinforced adversarial autoencoder; finally, train the constructed model.

2. The unsupervised anomaly detection method of a memory-reinforced adversarial autoencoder according to claim 1, characterized in that, The specific process of S3 is as follows: The dual-view discriminator consists of two parts: an image discriminator and a feature discriminator. The image discriminator adds a Sigmoid() function to the encoder e1. The feature discriminator adds the penultimate CNN layer of encoder e1 and the Sigmoid() function, which is used to discriminate the intermediate features of the penultimate layer output by encoders e1 and e2. For the input original image X and the reconstructed image... The intermediate features F of encoder e1 e1 The intermediate feature F of e2 e2 The dual-view discriminator will obtain four scores: the score for the original image and the score for the reconstructed image. , And the probabilities S of the two intermediate features f1 and S f2 When training the discriminator, positive samples are considered as 1, and reconstructed samples are considered as 0. This process is represented as follows: （16）； in This represents the parameters set; BCE() represents the nn.BCELoss() function; Loss dis Let be the loss function of the discriminator.

3. The unsupervised anomaly detection method of a memory-reinforced adversarial autoencoder according to claim 1, characterized in that, The specific process of S4 is as follows: For the input original positive sample image X, the original encoded features are first obtained by encoder e1. Then, it is fed into a multi-scale memory module to obtain memory-enhanced encoded features. ,right Decoding yields the reconstructed image X rec Then use encoder e2 to input X rec Encoding yields the reconstructed feature Z e2 ; combine the intermediate features of the two encoders and X rec X is input to the discriminator for discrimination; the training process includes five loss functions; Image contrast loss minimization positive sample reconstruction and normal data x i The distance between ∈X, so that the model can fully learn the normal pattern; represented as: （17）； The encoded features obtained by minimizing the encoding loss of encoders e1 and e2 and The distance between them is used so that the generator can learn how to encode features for generating images from normal data; represented as: （18）； Adversarial loss is used to minimize the discriminator's discrimination process f dis Original image features in () ∈F e1 and reconstructing image features ∈F e2 The difference between them; from the feature space, the generator can deceive the discriminator with the generated samples; represented as: （19）； To further improve sparsity, memory loss is used to minimize the sparse regularization matrix during training; denoted as: （20）； The generator-discriminator latent loss aims to minimize the features of the encoder e1 and the penultimate layer in the discriminator; the encoding loss and adversarial loss minimize the encoding and discrimination processes, respectively, which are performed separately in the generator or discriminator; the generator-discriminator latent loss connects them, continuously bringing the four intermediate processes closer together. This makes it impossible for the discriminator to distinguish between real and fake data from the feature space when training the generator, thus improving the generator's performance; expressed as: （21）； In summary, the five losses of the generator together constitute the generator loss, which can be expressed as: （22）； Feature-level comparison is used to replace image-level contrast to reduce computation; it is represented as: （23）； Here, E1() and E2() represent the encoding processes of encoders e1 and e2; S represents the score of anomalies, which is regularized between [0,1].

Images