Image anomaly detection method based on bidirectional potential space network

Through the combination of iterative memory generator and information fusion network, deep and local global features of images are extracted and fused, the problem of insufficient utilization of potential spatial information is solved, the image reconstruction quality and abnormal detection effect are improved, and more accurate abnormalities are achieved Test.

CN119068226APending Publication Date: 2024-12-03CHINA UNIV OF PETROLEUM (EAST CHINA)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310645127.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-02
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the image abnormality detection method based on the reconstruction generation task, the potential spatial information is insufficient, and the simple convolution extracts the feature space information of the image texture feature space neglects the feature information extraction of the image shape feature space, resulting in poor image reconstruction quality and poor reconstruction effect , affects the abnormal detection positioning and detection effect.

Method used

Build an iterative memory generator network and an information fusion network, record and update the image prototype mode through the iterative memory module, extract deep information, and fuse local and global features through the information fusion module to generate high-quality reconstruction images, and use bidirectional latent spatial constraints. The model is trained to improve the quality of the potential vector.

Benefits of technology

The image reconstruction quality and abnormal detection effect are improved, the utilization of image shape characteristics is enhanced, and the accuracy and efficiency of abnormal detection are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004263019050000031
    Figure BDA0004263019050000031
  • Figure BDA0004263019050000038
    Figure BDA0004263019050000038
  • Figure BDA0004263019050000041
    Figure BDA0004263019050000041
Patent Text Reader

Abstract

The invention discloses an image anomaly detection method based on a bidirectional potential space network. The key challenge of this task is to model high-quality reconstruction information. Most of the existing work only completes an end-to-end training process, and research work for high-quality reconstruction information and information flow between modules is little. From the perspective of a bidirectional potential space, the image anomaly detection method based on the bidirectional potential space network is constructed. According to the network, a prototype mode of training data is reserved through an iterative memory module, an image is reconstructed by adopting compressed prototype mode information, and on the other hand, an additional information fusion network is added to carry out image modeling on the reconstructed image in a direction different from that of the iterative memory module. In the process of zooming in the two modeling modes, potential spatial representation of training data is continuously optimized, so that high-quality image reconstruction is carried out. A large number of experiments prove that the image anomaly detection method based on the bidirectional potential space network can perform ideal reconstruction on training data, and a competitive result is obtained on MNSIT and CIFAR10 data sets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to an image anomaly detection method and relates to the technical field of computer vision, image processing and industrial defect detection. Background Art

[0002] Anomaly detection, also known as outlier detection, is a computer vision task that detects rare abnormal samples among many normal samples. In the past few years, the problem of anomaly detection in the field of computer vision has attracted great attention, which is of great significance to many fields such as industry and medicine. With the continuous advancement of this vision task, the available datasets and the fields involved are gradually increasing, but the high imbalance between abnormal and normal sample examples is still a thorny problem. Due to the high cost of collecting abnormal samples, abnormal samples are very scarce, which inhibits the ability of traditional supervised learning to perform on this task.

[0003] Recent studies have shown that GAN has become a promising and leading method for solving unsupervised and semi-supervised problems. GAN has a good performance in encoding high-dimensional information and capturing the distribution of input data. Samet Akcay proposed the GANomaly model, which consists of a generator, a discriminator, and an additional encoder. The generator constructs reconstructed data and latent vectors similar to the source data, the discriminator distinguishes the reconstructed data from the source data, and the additional encoder constructs the latent vector of the reconstructed data. GANomaly is based on the assumption that abnormal instances are significantly different from normal instances not only in high-dimensional space, but also in low-dimensional encoding space. Therefore, GANomaly uses the encoded low-dimensional space to jointly train the model to capture the distribution of normal instances. The limitations of GANomaly are that the mapping method of the latent space is relatively simple, and the encoding of the latent space vector lacks guidance; another point is that the parameters in the reconstructed data are not fully utilized; another point is that the traditional convolutional encoder lacks the association and understanding of the global information of the image, and the classification effect is poor. Samet Akcay tried to add skip links in the generator to achieve multi-scale capture of image information in order to achieve the purpose of generating high-quality images.

[0004] A Vaswani first proposed the Transformer based on the attention mechanism to replace the traditional convolution for machine translation tasks, which attracted widespread attention from industry professionals. Transformer solves the problem that CNN uses fixed-size filters and convolution operations, so it can only use local features. Dosovitskiy extended Transformer to large-scale image recognition tasks and named it VisionTransformer. Training ViTs on ImageNet through extensive data expansion and L2 regularization shows that VITs are difficult to train and often require a large parameter budget and lower accuracy. MobileViT combines the advantages of Transformer and CNN and builds a lightweight, universal and mobile-friendly network, so that VIT can be better trained and also brings better performance. In recent years, Transformer's tentacles have gradually extended to the task of anomaly detection. AnoVIT uses Transformer to assemble encoders to additionally learn the global relationship between image blocks to reflect normal information, so as to reconstruct the input data. UTRAD inputs the multi-scale features extracted from CNN into U-Transformer for feature-level reconstruction. VT-ADL uses Transformer encoding input to obtain the encoding vector, and inputs it into the decoder and Gaussian approximation network for reconstruction and anomaly localization.

[0005] Different from the existing methods, the model in this paper (IMRN) is based on the anomaly detection method of GAN. It constrains the generation process of the latent vector by proposing a horizontal-vertical bidirectional latent space pattern, and then reconstructs a high-quality reconstructed image from the latent vector. Then, the defect location and detection are achieved by comparing the pixels of the original input image with the reconstructed image. This paper uses an iterative memory network to weight the original information of the image and calculate the latent vector with better reconstruction quality; it uses an information fusion network to extract local features and global information from the image respectively, and fuses the image information with the image reconstruction information through the feature fusion module. Then, the latent vector of the latent space different from the iterative memory module is obtained. Finally, by penalizing the training process of the two latent vector constraint models, a better image generation effect is achieved, which is convenient for the anomaly detection of the generated image and the original input image. Summary of the invention

[0006] The purpose of the present invention is to solve the problem that in the image anomaly detection method based on reconstruction generation task, the latent space information is insufficiently utilized, and only the information of the image texture feature space is extracted through simple convolution, while the feature information extraction and utilization of the image shape feature space in anomaly detection are neglected, resulting in poor image reconstruction quality and poor reconstruction effect, thereby affecting the anomaly detection positioning and detection effect.

[0007] The technical solution adopted by the present invention to solve the above technical problems is:

[0008] S1. Construct an iterative memory generator network for image information extraction and image generation;

[0009] S2. Construct an information fusion network to generate local information and global information fusion of the image;

[0010] S3. Combine the iterative memory network in S1 and the information fusion network in S2 to build a network architecture for image anomaly detection based on bidirectional latent space;

[0011] S4. Training of bidirectional latent space image anomaly detection network and generation of anomaly detection results. The specific process is:

[0012] First, we construct an iterative memory network autoencoder network, which consists of an IM encoder G IM-E and a decoder network G D , acts as the generator G part of the model, completing the feature extraction and image reconstruction of the input image.

[0013] The iterative memory module contains a set P of M memory items, where P = {p1, ..., p M}, They record different normal data prototype patterns. The iterative memory module is divided into two steps: reading and updating. The reading of the iterative memory module can record the prototype pattern of the application data set rather than simple image features. The update can make the prototype pattern recorded by the iterative memory module change dynamically, so as to be more suitable for model training.

[0014] In the read phase of the iterative memory module, Q = {q1, q2, ..., q K Each query q in} k with all p in P m The cosine similarity of , we get a two-dimensional similarity graph of M×K. Then, applying the softmax function along the vertical direction, we get the matching probability w k,m as follows:

[0015]

[0016] For each q k, will use all p stored in memory m The calculated w k,m The weighted average of p m Represents the obtained feature map As shown in the following formula:

[0017]

[0018] In all After being represented, it will get a feature set of the same dimension as Q Link it with the query map Q along the channel dimension and Read deep memory as input to get the next feature set This process is iterated until we get The reading stops, and the formula for implementing the iteration process is as follows:

[0019]

[0020]

[0021] Finally, the initial query Q and all query mappings Connect to get IMStack, where C s = dep+1. IMStack integrates the information of all prototype patterns, neither adopting the closest pattern nor discarding the most distant pattern, which takes into account the overall pattern information. At the same time, the features are explored in depth and finally mapped, which allows the model to learn more deep information, which may be very helpful for this task. The superimposed information is convolved, which makes the generated latent vector z rich in deep scale information in multiple modes. Inputting it into the decoder can enhance its understanding of normal capabilities, thereby reducing the powerful representation ability of CNN.

[0022] In the update phase of the iterative memory module, the matching probability formula (1) above is used to generate all queries that are closest to the item. Usually, a single memory item can be similar to multiple queries. Then, use U m Represents the index set corresponding to the query of the mth item, using only U m The memory item is updated by querying the index in the following formula:

[0023]

[0024] A weighted average of queries is used, which allows the module to focus more on queries close to a prototype pattern, that is, near a memory item, making the update more focused. To this end, the matching probability v is calculatedk,m Similar to formula (1), but applying the softmax function to a direction different from w (horizontal direction) obtains an M×K correlation map, as follows:

[0025]

[0026] In addition, this paper renormalizes it to consider the set U m The query for the index is as follows:

[0027]

[0028] Continue to extend the update operation to the depth level, and the memory pools of different depths are updated independently without interfering with each other. The recursive formula is as follows:

[0029]

[0030] After one layer of convolution, all memory blocks IMStack are compressed into a potential vector z, where z∈R d , z is also called the bottleneck feature, which is considered to be the smallest dimension in the best representation of x. Finally, z is input to the decoder G in the generator G D Enlarge vectorz to reconstruct the image At this point, the iterative memory autoencoder reconstructs the input image x into The whole process can be summarized as where z = G IM-E (x),

[0031] In the iterative memory module, the memory items of the prototype features are updated during training, and the updates are stopped during testing. This keeps the prototype patterns of each level consistent during training and testing, and avoids the interference of abnormal images on the memory prototype. On the other hand, the memory suppression of abnormal images also enhances the memory deepening of normal images, so that the iterative memory module can better understand and record the patterns of normal images, while the lack of memory for abnormal images increases the probability of restoring abnormal images to normal images according to the normal pattern, that is, the abnormality cannot be reconstructed.

[0032] In addition, this paper constructs an information fusion network to generate local information and global information fusion of the image. Here, a feature supply generator FSG is proposed to aggregate and fuse multi-channel features, taking into account the historical source information of the generated image and the feature information of the generated image.

[0033] First, the first layer is the decoder G from the previous layer D The historical hierarchical features of the decoder are extracted independently, and G D The decoding structures of HierThen import it into FSG. The second layer is to generate images The extracted two-way feature information, one of which is the convolution feature F Conv , one is to use the feature F extracted by the Transformer module MVIT FSG integrates the three-way information into a rich aggregate information F CoNet , where F CoNet ={F Hier ,F Conv ,F MVIT}, the information fusion network will eventually obtain the feature fusion information F CoNet Compress it to the same dimension as the bottleneck feature z

[0034] After the iterative memory network and information fusion network are built, this paper adopts the GAN-based method to build a bidirectional latent space image anomaly detection network. The network contains an iterative memory generator, an information fusion encoder model, and an information fusion discriminator.

[0035] This paper first uses feature matching loss for adversarial learning. The purpose of the adversarial loss here is to ensure that the network G can be as realistic as possible from x to At the same time, the discriminator network D can classify the input samples and generated samples so that high-quality samples can be generated to deceive D:

[0036]

[0037] Adversarial loss is used to balance the generation of true level samples and the effectiveness of the discriminator, but lacks contextual information between the learning input image and the reconstruction result. In order to enhance the learning of this contextual information so that the generator G fully captures the distribution information of normal data, this paper adds contextual loss:

[0038]

[0039] This paper adopts the generator model based on iterative memory network and the encoder model of information fusion network to model the latent space of input image and generated image respectively. Both of them model two different latent spaces at the same time, completing the expansion of horizontal latent space. The latent space query method with memory property adopted by iterative memory generator retains the prototype information of normal image by modeling latent vector; the method based on multi-channel information fusion adopted by information fusion network models images more richly. Both of them complete the information mining of vertical latent space in their respective latent spaces.

[0040] The iterative memory generator and information fusion encoder model can generate the latent vectors of the input image and the generated image in turn. This paper uses the following loss function to expand the horizontal direction of the latent space:

[0041]

[0042] Finally, the objective loss function proposed in this paper is as follows:

[0043] L=λ adv L adv +λ con L con +λ LaIM L LaIM (12)

[0044] where λ adv , con , LaIM It is the weight parameter corresponding to each loss. Different sizes represent the effect of a single loss on the overall loss function.

[0045] Finally, the training method of the bidirectional latent space image anomaly detection network is as follows:

[0046] The model is implemented by Pytorch 1.0. During the training process, only normal images are input for training. First, each image passes through the iterative memory generator. The image will first be compressed by the iterative memory network into a 512×4×4 query block, and then the query blocks are accumulated. Finally, all memory blocks are feature compressed through a layer of convolution, and a feature vector with a dimension of nz is output. The feature vector then passes through the decoder in the generator and generates an image through deconvolution. For the training of the model, it is divided into two parts: the generator and the discriminator. Both parts use the Adam optimizer to train the model. 15 batches are trained in the MNIST dataset and the CIFAR dataset, and the learning rate is set to 0.0002, λ adv , con , LaIM They are set to 1, 50 and 1 respectively. The two datasets are slightly different in the dimension of the latent vector. MNIST uses a 512-dimensional vector space, while CIFAR uses a 100-dimensional vector space.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. This paper proposes a new bidirectional latent space anomaly detection network IMRN, which not only expands the latent vector in the horizontal dimension, but also innovates in the information modeling of the latent vector, providing a solution for improving the quality of reconstructed images.

[0049] 2. The present invention proposes an iterative memory network and an information fusion network. The former retains richer image information by recording image prototypes, and the latter extracts richer features from image information by fusing local information with global information. Both generate feature vectors from two latent spaces, paving the way for identifying anomalies and inferring the locations of anomalies and defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the network structure for image anomaly detection based on bidirectional latent space.

[0051] Figure 2 Schematic diagram of information fusion network.

[0052] Figure 3 and Figure 4 A comparison chart of anomaly detection results based on the bidirectional latent space image anomaly detection network.

[0053] Figure 5 and Figure 6 The generated image restoration process and anomaly detection visualization result diagram based on the bidirectional latent space image anomaly detection network. DETAILED DESCRIPTION

[0054] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present patent.

[0055] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0056] Figure 1 Figure 1 is a schematic diagram of the network structure for image anomaly detection based on bidirectional latent space. Figure 1 As shown in Figure 1, the entire GAN-based anomaly detection network framework consists of two parts: the generator (left) and the discriminator (right). The dark background on the right is the discriminator of this paper, and the light background is the encoder used to expand the horizontal potential space of this paper.

[0057] In the read phase of the iterative memory generator, Q = {q1,q2,...,q K Each query q in} k with all p in P m The cosine similarity of , we get a two-dimensional similarity graph of M×K. Then, applying the softmax function along the vertical direction, we get the matching probability w k,m as follows:

[0058]

[0059] For each q k , will use all p stored in memory m The calculated w k,mThe weighted average of p m Represents the obtained feature map As shown in the following formula:

[0060]

[0061] In all After being represented, it will get a feature set of the same dimension as Q Link it with the query map Q along the channel dimension and Read deep memory as input to get the next feature set This process is iterated until we get The reading stops, and the formula for implementing the iteration process is as follows:

[0062]

[0063]

[0064] Finally, the initial query Q and all query mappings Connect to get IMStack, where C s = dep + 1. The superimposed information is convolved, which makes the generated potential vector z rich in deep-scale information in multiple modes.

[0065] In the update phase of the iterative memory module, the matching probability formula (1) above is used to generate all queries that are closest to the item. Usually, a single memory item can be similar to multiple queries. m Represents the index set corresponding to the query of the mth item, using only U m The memory item is updated by querying the index in the following formula:

[0066]

[0067] A weighted average of queries is used, which allows the model to focus more on queries close to a prototype pattern, that is, near a memory item, making the update more focused. To this end, the matching probability v is calculated k,m Similar to formula (1), but applying the softmax function to a direction different from w (horizontal direction) obtains an M×K correlation map, as follows:

[0068]

[0069] In addition, this paper renormalizes it to consider the set U m The query for the index is as follows:

[0070]

[0071] Continue to extend the update operation to the depth level, and the memory pools of different depths are updated independently without interfering with each other. The recursive formula is as follows:

[0072]

[0073] After one layer of convolution, all memory blocks IMStack are compressed into a potential vector z, where z∈R d , z is also called the bottleneck feature, which is considered to be the smallest dimension in the best representation of x. Finally, z is input to the decoder G in the generator G D Enlarge vectorz to reconstruct the image At this point, the iterative memory autoencoder reconstructs the input image x into The whole process can be summarized as where z = G IM-E (x), Finally, the reconstructed image is generated

[0074] Figure 2 This is a schematic diagram of the information fusion network. Figure 2 As shown in the figure, the input image of the information fusion network is the image reconstructed by the generator. After the image input, it is divided into two paths for feature extraction. First, Figure 2 The convolutional feature extraction module at the top extracts local feature information; followed by Figure 2 The global feature extraction at the bottom. In the process of global feature extraction, in order to reduce the amount of calculation, this paper first performs local modeling on the image, and then performs global information modeling on the local parts separated by the model, and finally forms a modeling method in which local information is associated and each local part of the global information is associated. Finally, the local feature information of the reconstructed image and the global feature information are converged at the FSG, completing the feature fusion of local information and global information.

[0075] Figure 3 and Figure 4 This is a comparison chart of the anomaly detection results based on the bidirectional latent space image anomaly detection network. Figure 3 and Figure 4 Overall, we can see that the results of this paper are more accurate than other existing methods and can effectively identify anomalies. From the perspective of each individual subclass, the results of this paper are also more advantageous than other existing methods.

[0076] Figure 5 and Figure 6 To generate the image restoration process and anomaly detection visualization results. Figure 5As shown in , given an image, the bidirectional latent space image anomaly detection network model can be gradually trained to generate the corresponding image, which includes the recognition of the target and the fine-grained texture restoration. Figure 6 As shown, given the input of abnormal and defective images, an image without abnormalities and defects is generated based on the bidirectional latent space image anomaly detection network model, so that by comparing the generated image with the input image, it can be determined whether the image has defects and the specific location of the defects.

[0077] The present invention proposes an anomaly detection method based on a bidirectional latent space, which is divided into two parts: iterative memory network construction and information fusion network construction. The iterative memory network focuses more on the prototype pattern of the normal image. The information fusion network focuses more on the fusion of local information, global information and reconstruction process information of the image. Finally, the quality of the latent vector is continuously improved by comparing and constraining the latent vector in two different ways, so as to better utilize the latent vector to restore high-quality reconstructed images. Through training and testing results, the accuracy of the image anomaly detection network method based on bidirectional latent space in this paper has been improved on multiple types of anomaly detection datasets. In the future, we will further study the effectiveness of the bidirectional latent space method in the fields of anomaly detection, industrial defect detection and other visual image tasks.

[0078] Finally, the details of the above examples of the present invention are only examples for explaining the present invention. For those skilled in the art, any modification, improvement and replacement of the above embodiments should be included in the protection scope of the claims of the present invention.

Claims

1. An image anomaly detection method based on a bidirectional latent space network, characterized in that: The method comprises the following steps: S1. Construct an iterative memory generator network for image information extraction and image generation; S2. Construct an information fusion network to generate local information and global information fusion of the image; S3. Combine the iterative memory network in S1 and the information fusion network in S2 to build a network architecture for image anomaly detection based on bidirectional latent space; S4. Training of bidirectional latent space image anomaly detection network and generation of anomaly detection results.

2. The image anomaly detection method based on a bidirectional latent space network according to claim 1, characterized in that: The specific process of S1 is as follows: In the read phase of the iterative memory generator, Q = {q1,q2,...,q K Each query q in} k with all p in P m The cosine similarity of , we get a two-dimensional similarity graph of M×K. Then, applying the softmax function along the vertical direction, we get the matching probability w k,m as follows: For each q k , will use all p stored in memory m The calculated w k,m The weighted average of p m Represents the obtained feature map As shown in the following formula: In all After being represented, it will get a feature set of the same dimension as Q Link it with the query map Q along the channel dimension and Read deep memory as input to get the next feature set This process is iterated until we get The reading stops, and the formula for implementing the iteration process is as follows: Finally, the initial query Q and all query mappings Connect to get IMStack, where C s = dep + 1. The superimposed information is convolved, which makes the generated potential vector z rich in deep-scale information in multiple modes. In the update phase of the iterative memory module, the matching probability formula (1) above is used to generate all queries that are closest to the item. Usually, a single memory item can be similar to multiple queries. m Represents the index set corresponding to the query of the mth item, using only U m The memory item is updated by querying the index in the following formula: A weighted average of queries is used, which allows the model to focus more on queries close to a prototype pattern, that is, near a memory item, making the update more focused. To this end, the matching probability v is calculated k,m Similar to formula (1), but applying the softmax function to a direction different from w (horizontal direction) obtains an M×K correlation map, as follows: In addition, this paper renormalizes it to consider the set U m The query for the index is as follows: Continue to extend the update operation to the depth level, and the memory pools of different depths are updated independently without interfering with each other. The recursive formula is as follows: After one layer of convolution, all memory blocks IMStack are compressed into a potential vector z, where z∈R d , z is also called the bottleneck feature, which is considered to be the smallest dimension in the best representation of x. Finally, z is input to the decoder G in the generator G D Enlarge vectorz to reconstruct the image At this point, the iterative memory autoencoder reconstructs the input image x into The whole process can be summarized as where z = G IM-E (x), Finally, the reconstructed image is generated 3. The image anomaly detection method based on a bidirectional latent space network according to claim 1, characterized in that: The specific process of S2 is: Firstly, an information fusion network is constructed to generate local information and global information fusion of the image. Here, this paper proposes a feature supply generator FSG to aggregate and fuse multi-channel features. First, the first layer is the decoder G from the previous layer D We extract the three layers of features of the decoder independently and use G D The decoding structures of Hier Then import it into FSG. The second layer is to generate images The extracted two-way feature information, one of which is the convolution feature F Conv , one way is to use the feature F extracted by the Transformer module MVIT FSG integrates the three-way information into a rich aggregate information F CoNet , where F CoNet ={F Hier ,F Conv ,F MVIT }, the information fusion network will eventually obtain the feature fusion information F CoNet Compress it to the same dimension as the bottleneck feature z 4. The image anomaly detection method based on a bidirectional latent space network according to claim 1, characterized in that: The specific process of S3 is as follows: After the iterative memory network and information fusion network are built, this paper adopts the GAN-based method to build a bidirectional latent space image anomaly detection network. The network contains an iterative memory generator, an information fusion encoder model, and an information fusion discriminator. This paper first uses feature matching loss for adversarial learning. The purpose of the adversarial loss here is to ensure that the network G can be as realistic as possible from x to At the same time, the discriminator network D can classify the input samples and generated samples so that high-quality samples can be generated to deceive D: Adversarial loss is used to balance the generation of true level samples and the effectiveness of the discriminator, but lacks contextual information between the learning input image and the reconstruction result. In order to enhance the learning of this contextual information so that the generator G fully captures the distribution information of normal data, this paper adds contextual loss: This paper adopts a generator model based on an iterative memory network and an encoder model based on an information fusion network to model the latent spaces of the input image and the generated image respectively. Both model two different latent spaces at the same time, completing the expansion of the horizontal latent space. The iterative memory generator uses a latent space query method with memory properties, and the latent vector modeled retains the prototype information of the normal image; the information fusion network uses a multi-path information fusion method to model the image more richly. Both complete the information mining of the vertical latent space in their respective latent spaces. The iterative memory generator and information fusion encoder model can generate the latent vectors of the input image and the generated image in turn. This paper uses the following loss function to expand the horizontal direction of the latent space: Finally, the objective loss function proposed in this paper is as follows: L=λ adv L adv +λ con L con +λ LaIM L LaIM (12) where λ adv , con , LaIM It is the weight parameter corresponding to each loss. Different sizes represent the effect of a single loss on the overall loss function.

5. The bidirectional latent space image anomaly detection network according to claim 1, characterized in that: The specific process of S4 is as follows: The training method of the bidirectional latent space image anomaly detection network is as follows: The model is implemented in Pytorch 1.

0. During the training process, only normal images are input for training. First, each image passes through the iterative memory generator. The image is first compressed into a 512×4×4 query block by the iterative memory network, and then the query blocks are accumulated. Finally, a layer of convolution is used to compress the features of all memory blocks, and the feature vector with the output dimension of nz is output. The feature vector then passes through the decoder in the generator and generates an image through deconvolution. The model training is divided into two parts: the generator and the discriminator. Both parts use the Adam optimizer to train the model. 15 batches are trained on the MNIST dataset and the CIFAR dataset, with the learning rate set to 0.0002 and λ adv , con , LaIM They are set to 1, 50 and 1 respectively. The two datasets are slightly different in the dimension of the latent vector. MNIST uses a 512-dimensional vector space, while CIFAR uses a 100-dimensional vector space.