Image anomaly detection method based on soft mask contrast loss
Through the unsupervised learning soft mask comparison loss method and synthetic image training data set, the problem of poor detection effect caused by excessive reconstruction ability of image anomaly detection algorithm in the prior art is solved, and the recognition ability of diverse anomaly samples is improved. Detection accuracy.
Patent Information
- Application Number
- CN202210209818.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-04
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-03-04
AI Technical Summary
During the training process, the existing image anomaly detection algorithm has poor abnormal detection effect due to its strong reconstruction ability, and it is difficult to identify high-diversity abnormal samples.
Using the soft mask contrast loss method of unsupervised learning, a synthetic image training data set is generated through superpixel segmentation and Gaussian noise processing, and an image anomaly detection model connected in parallel by the reconstructed autoencoder and mask autoencoder is built, using the soft mask The membrane comparison loss function optimization model increases the reconstruction error of abnormal areas.
The effect of image abnormality detection is improved, the identification types of abnormal samples are enhanced, and better abnormal area positioning and detection results are achieved.
Smart Images

Figure CN114565594B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing technology, and further relates to an image anomaly detection method, which can be applied to industrial product defect detection, medical auxiliary diagnosis, and video violation detection. Background Art
[0002] Image anomaly detection is a task in the field of computer vision used to detect and locate abnormal or defective areas in images. Image anomaly detection has a wide range of applications in production and life. For example, it can detect defective products that accidentally appear in the production process of electronic components, textile semi-finished products and other industrial products due to improper operation and other reasons, and assist in the elimination of defective products; help medical personnel locate lesions in pathological images; and detect illegal behaviors in surveillance videos. Since the probability of abnormal samples in real life is extremely small, it takes a lot of cost to obtain a dataset of abnormal samples with sufficient number and rich categories. Therefore, most of the current image anomaly detection datasets only contain normal samples. Image anomaly detection algorithms are usually trained on normal datasets in an unsupervised manner, trying to fit the normal sample distribution, and judging whether the image is abnormal by judging whether the test sample conforms to the learned distribution. Since the method of manually constructing image features is extremely sensitive to the image acquisition environment and image quality, traditional image anomaly detection algorithms have never been able to reach the same level as manual detection.
[0003] In recent years, with the rapid development of deep learning technology, the features extracted using convolutional neural networks trained on data sets have lower requirements on image quality. Therefore, image anomaly detection algorithms based on deep learning have made long-term progress and achieved satisfactory detection accuracy.
[0004] Chengdu Dongfang Tiancheng Intelligent Technology Co., Ltd. disclosed in its patent document with patent application number: 202111288603.2 "A system, method and storage medium for anomaly detection based on generative adversarial networks". The system includes a feature mapping network part, a generator network part and a discriminator network part. The feature mapping network part is used to map image samples into latent space tensors, and the generator network part is used to learn the true distribution of images from latent space tensors. The discriminator network part is used to distinguish between real images and reconstructed images during training; the system aims to reconstruct normal samples and attempts to learn the distribution of normal samples from normal samples, so that during the test process, anomaly detection is achieved through high reconstruction errors in abnormal areas. However, this method, which only uses reconstruction as the objective function, can often reconstruct abnormal areas because its generator's reconstruction ability is too powerful, resulting in the reconstruction error of abnormal images being also very small, which leads to poor anomaly detection effect.
[0005] Hunan University proposed "a surface anomaly detection method based on hybrid supervised learning" in its patent document with application number: 202111161614.4. This method first obtains normal samples with normal surfaces and abnormal samples with abnormal surfaces for preprocessing; and constructs a neural network model including anomaly localization network, self-attention network and anomaly discrimination network, and then inputs the preprocessed data into the neural network for training to obtain an anomaly detection model for training to obtain an anomaly detection neural network model; finally, the image data to be tested is input into the anomaly detection neural network model, and then it can automatically determine whether there is an abnormality in the image to be tested and locate the abnormal area. Because this method uses a hybrid supervised learning method to learn, normal samples and a small number of roughly labeled samples are used as training sets, so the detection effect on abnormal types of data that have never appeared in the training set is poor. Summary of the invention
[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose an unsupervised learning soft mask contrast loss image anomaly detection method to improve the image anomaly detection effect and expand the types of abnormal samples that can be identified.
[0007] To achieve the above object, the technical solution of the present invention includes the following:
[0008] (1) The MVTec industrial anomaly detection training dataset is processed using the superpixel segmentation algorithm and Gaussian noise to obtain a synthetic image training dataset:
[0009] 1a) All normal image data in the MVTec training data set are segmented into superpixels using a superpixel segmentation algorithm to obtain superpixel images and segmentation mask images;
[0010] 1b) Randomly select one twentieth of the number of superpixels from the segmented superpixel image, add Gaussian noise to it, obtain a synthetic image, and use the segmentation mask image corresponding to the selected superpixel as the mask image label of the synthetic image;
[0011] 1c) Using the synthetic images and their corresponding mask labels as training datasets;
[0012] (2) Build an image anomaly detection model consisting of a reconstruction autoencoder and a mask autoencoder in parallel;
[0013] (3) Use the synthetic image training set to train the image anomaly detection model:
[0014] 3a) Calculate the predicted mask map output by the mask autoencoder The label mask M corresponding to the input synthetic abnormal image syn_ab The L2 error loss between MAE (I syn_ab), and use the L2 error loss as the loss function, use the Adam back propagation optimization algorithm to train the mask autoencoder, and complete its training, where I syn_ab represents the input synthetic abnormal image, represents the masked autoencoder, ω MAE represents the parameters of the mask autoencoder;
[0015] 3b) Train the reconstruction autoencoder once:
[0016] 3b1) Calculate the reconstructed image output by the encoder Compared with the original normal image I nor The L2 error loss between res (I syn_ab ), and use the mask autoencoder obtained from one training to obtain the soft mask map: Among them represents the masked autoencoder obtained through one training, Represents the parameters of the mask autoencoder obtained through one training;
[0017] 3b2) Multiply the soft mask image calculated in 3b1) by the synthetic abnormal image pixel by pixel to obtain the negative sample in the soft mask contrast loss: N = M soft *I syn_ab ;
[0018] 3b3) Calculate the soft mask contrast loss L between the reconstructed image A of the reconstruction encoder, the negative sample N, and the original normal image P corresponding to the synthesized image CL (A,P,N), and the L2 error loss L res (I syn_ab ) and the soft mask contrast loss L CL The weighted sum of (A, P, N) is used as the loss function L of the reconstruction autoencoder RAE (I syn_ab ,P,N), and use the Adam back-propagation optimization algorithm to train and reconstruct the autoencoder to complete its first training;
[0019] 3d) alternately executing 3a) and 3b) until the corresponding loss functions converge, stop iteration, and obtain a trained image anomaly detection model;
[0020] (4) Image anomaly location and detection:
[0021] 4a) Take the image I in the MVTec test dataset as input and input it into the trained image anomaly detection model, which outputs the predicted mask image S through the mask autoencoder mask (I), reconstruct the image by reconstructing the self-encoder output and passing the L2 error S L2 (I) and contrast error SCL (I) Obtain the abnormal score map S of the reconstructed autoencoder res (I);
[0022] 4b) Predict the mask image S through the output of the mask autoencoder mask (I) and the anomaly score map S of the reconstructed autoencoder res (I), obtain the pixel-level anomaly score map S(I) for anomaly localization:
[0023] S(I)=S res (I)+h · ·S mask (I)
[0024] Among them, h · is the confidence parameter of the mask autoencoder;
[0025] 4c) Select the maximum value in the pixel-level anomaly score map to obtain the image-level anomaly score η for anomaly detection:
[0026]
[0027] Where w represents the width of the pixel-level anomaly score map, h represents the height of the pixel-level anomaly score map, and S(I)(i,j) represents the anomaly score at the i-th row and j-th column in the pixel-level anomaly score map;
[0028] 4d) Set a threshold T based on the original normal image of the MVTec training dataset, and compare the image-level anomaly score η of the test image I with the set threshold:
[0029] If η>T, image I is judged as abnormal;
[0030] If η≤T, the image I is judged to be normal.
[0031] Compared with the prior art, the present invention has the following advantages:
[0032] First, in addition to using the reconstruction error loss as the objective function, the present invention introduces a contrast loss function that meets the definition of anomaly detection. While learning the normal distribution, it stays away from the abnormal distribution, which makes the model have poor reconstruction effect on the abnormal area, increases the reconstruction error of the abnormal area, and improves the anomaly detection result.
[0033] Secondly, the present invention uses a synthetic image training data set with Gaussian noise superpixels to train the image anomaly detection model, and simulates the diversity of anomalies by randomly generating noise superpixels in the synthetic image, thereby expanding the types of abnormal samples that can be identified. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a general flow chart for realizing the present invention;
[0035] Figure 2 It is a diagram of the image anomaly detection model constructed in the present invention;
[0036] Figure 3 It is a sub-flow chart for training the image anomaly detection model in the present invention. DETAILED DESCRIPTION
[0037] The embodiments and effects of the present invention are further described in detail below with reference to the accompanying drawings.
[0038] Reference Figure 1 , the implementation steps of this example are as follows.
[0039] Step 1: Build a synthetic image training dataset.
[0040] 1.1) Obtain the MVTec industrial anomaly detection dataset from a public website, and use the SEEDs superpixel segmentation algorithm to perform superpixel segmentation on all normal image data in the MVTec training dataset to obtain superpixel images and segmentation mask images;
[0041] 1.2) Randomly select one twentieth of the superpixels from the segmented superpixel image, add Gaussian noise to it, obtain a synthetic image, and use the segmentation mask image corresponding to the selected superpixel as the mask image label of the synthetic image;
[0042] 1.3) Use the synthetic images and their corresponding mask labels as training datasets.
[0043] Step 2: Build an image anomaly detection model.
[0044] Reference Figure 2 ,The image anomaly detection model includes a reconstruction autoencoder and a mask autoencoder, and its structure is as follows:
[0045] 2.1) Establish a reconstruction autoencoder consisting of a cascade of an encoder and a reconstruction decoder with 3 output channels:
[0046] The encoder is composed of a convolution layer, a first maximum pooling layer, four residual modules and a second maximum pooling layer in cascade, wherein the convolution kernel size of the convolution layer is 7×7, the step size is 2, and the padding pixel is 1; the step sizes of the two maximum pooling layers are both 2; the four residual modules are respectively composed of 3, 4, 6, and 3 residual convolution groups, each residual convolution group is composed of a first convolution layer and a second convolution layer connected, the convolution kernel size of the two convolution layers is 3×3, the step size is 1, and the padding pixel is 1, and the output feature of the second convolution layer is added to its input feature channel by channel to obtain the output feature of the residual convolution group;
[0047] The reconstruction decoder is composed of six decoding modules and a convolution module cascaded in sequence, wherein each decoding module is composed of a bilinear interpolation upsampling layer with a step size of 2, a third convolution layer, and a fourth convolution layer; the convolution module is composed of two convolution layers, namely, a fifth convolution layer and a sixth convolution layer; the convolution kernel size of the third convolution layer and the fourth convolution layer is 3×3, the step size is 1, and the padding pixel is 1; the convolution kernel size of the fifth convolution layer and the sixth convolution layer is 3×3, the step size is 2, the padding pixel is 1, and the output channel is 3;
[0048] 2.2) Establish a masked autoencoder consisting of a cascade of an encoder and a reconstruction decoder with output channel 1:
[0049] The encoder has the same structure as the encoder in the reconstruction autoencoder;
[0050] The mask decoder has the same structure as the reconstruction autoencoder in the reconstruction autoencoder, and its output channel is 1;
[0051] 2.3) The reconstruction autoencoder and the mask autoencoder are connected in parallel to form an image anomaly detection model, where:
[0052] The reconstruction autoencoder is used to reconstruct the input image, that is, the encoder encodes the input image into a feature map, and the reconstruction decoder decodes the feature map into a 3-channel image to complete the reconstruction of the input image;
[0053] The mask autoencoder is used to predict anomalies in the input image, that is, the input image is encoded into a feature map through the encoder, and the mask decoder decodes the feature map into a 1-channel mask map to complete the anomaly prediction of the anomaly in the input image.
[0054] Step 3: Train the image anomaly detection model.
[0055] Reference Figure 3 , the specific implementation of this step is as follows:
[0056] 3.1) Train the mask encoder once:
[0057] 3.1.1) Calculate the predicted mask map output by the mask autoencoder The label mask M corresponding to the input synthetic abnormal image syn_ab The L2 error loss between MAE (I syn_ab ), and used as the loss function of the masked autoencoder:
[0058]
[0059] Among them, I syn_abrepresents the input synthetic abnormal image, n is the number of data in the synthetic image dataset, represents the masked autoencoder, ω MAE represents the parameters of the mask autoencoder;
[0060] 3.1.2) Use the Adam back-propagation optimization algorithm to minimize the loss function L MAE (I syn_ab ), update the mask autoencoder parameters, and complete a training of the mask autoencoder;
[0061] 3.2) Train the reconstruction autoencoder once:
[0062] 3.2.1) Calculate and reconstruct the reconstructed image output by the encoder Compared with the original normal image I nor The L2 error loss between res (I syn_ab ):
[0063]
[0064] Where n is the number of data in the synthetic image dataset, To reconstruct the autoencoder, ω RAE To reconstruct the weight parameters of the autoencoder;
[0065] 3.2.2) Using the mask autoencoder obtained from one training to obtain the soft mask image Multiply the soft mask image by the synthetic anomaly image pixel by pixel to obtain the negative sample N in the soft mask contrast loss:
[0066] N=M soft *I syn_ab
[0067] in, represents the masked autoencoder obtained through one training, Represents the parameters of the mask autoencoder obtained through one training;
[0068] 3.2.3) Calculate the soft mask contrast loss L between the reconstructed image A of the reconstruction encoder, the negative sample N and the original normal image P corresponding to the synthesized image CL (A,P,N):
[0069]
[0070] in, P=I nor , To reconstruct the autoencoder, ω RAE To reconstruct the weight parameters of the autoencoder, v i(A) represents the feature map obtained after the reconstructed image passes through the i-th VGG19 feature layer, v i (P) represents the feature map obtained after the original normal image passes through the i-th VGG19 feature layer, v i (N) represents the feature map obtained after the negative sample passes through the i-th VGG19 feature layer, d(v i (A),v i (P)) represents the L1 distance between the reconstructed image feature map and the original normal image feature map, d(v i (A),v i (N)) represents the L1 distance between the reconstructed image feature map and the negative sample feature map, w i represents the weight parameter of the soft mask contrast loss term under the i-th scale feature extraction layer, i ranges from {1,2,...,k}, k represents the number of feature extraction layers of different scales of VGG19 trained on the ImageNet public dataset, k=5, that is, the weight parameters are h represents the soft mask contrast loss The weight of the factor is as follows: in, represents the masked autoencoder obtained through one training, Represents the parameters of the mask autoencoder obtained through one training;
[0071] 3.2.4) L2 error loss L res (I syn_ab ) and the soft mask contrast loss L CL The weighted sum of (A, P, N) is used as the loss function L of the reconstruction autoencoder RAE (I syn_ab ,P,N), expressed as follows:
[0072] L RAE (I syn_ab ,P,N)=L res (I syn_ab )+λ·L CL (A,P,N)
[0073] Among them, λ is the weight hyperparameter that balances the reconstruction loss and the soft mask contrast loss, I syn_ab represents a synthetic abnormal image, P represents an original normal image, and N represents a negative sample. represents the reconstructed image, represents the reconstruction autoencoder, ω RAE Represents the weight parameters of the reconstruction autoencoder;
[0074] 3.2.5) Use the Adam back-propagation optimization algorithm to minimize the loss function L RAE(I syn_ab ,P,N) values, update the reconstruction autoencoder parameters, and complete a training of the reconstruction autoencoder;
[0075] 3.3) Repeat 3.1) and 3.2) to train the mask autoencoder and the reconstruction autoencoder alternately until their corresponding loss functions converge. Stop the iteration and obtain a trained image anomaly detection model.
[0076] Step 4: Image anomaly location and detection.
[0077] 4.1) Take the image I in the MVTec test dataset as input and input it into the trained image anomaly detection model, which outputs the predicted mask image S through the mask autoencoder mask (I)
[0078]
[0079] in, represents the trained mask autoencoder, Represents the weight parameters of the mask autoencoder;
[0080] 4.2) Image I is reconstructed by reconstructing the output of the autoencoder and calculating the L2 error S L2 (I) and contrast error S CL (I)
[0081]
[0082]
[0083] in, represents the trained reconstruction autoencoder, Represents the obtained reconstruction autoencoder weight parameter, w i represents the weight parameter of the soft mask contrast loss term under the i-th scale feature extraction layer, i ranges from {1,2,...,k}, k represents the number of feature extraction layers of different scales of VGG19 trained on the ImageNet public dataset, k=5, that is, the weight parameters are v i (I) represents the feature map obtained after the test image passes through the i-th VGG19 feature layer, Represents the feature map obtained after the reconstructed image passes through the i-th VGG19 feature layer, represents the L1 distance between the reconstructed image feature map and the original normal image feature map, It means that the L1 distance between the reconstructed image feature map and the original normal image feature map is upsampled to the test image size.
[0084] 4.3) Through L2 error S L2 (I) and contrast error S CL (I) Obtain the abnormal score map S of the reconstructed autoencoder res (I)
[0085] S res (I) = S L2 (I)+λ·S CL (I)
[0086] Among them, λ is the weight hyperparameter that balances L2 error and contrast error;
[0087] 4.4) Predict the mask image S through the output of the mask autoencoder mask (I) and the anomaly score map S of the reconstructed autoencoder res (I), obtain the pixel-level anomaly score map S(I) for anomaly localization:
[0088] S(I)=S res (I)+h · ·S mask (I)
[0089] Among them, h · is the confidence parameter of the mask autoencoder:
[0090]
[0091] represents the trained mask autoencoder, Represents the weight parameters of the mask autoencoder;
[0092] 4.5) Select the maximum value in the pixel-level anomaly score map to obtain the image-level anomaly score η for anomaly detection:
[0093]
[0094] Where w represents the width of the pixel-level anomaly score map, h represents the height of the pixel-level anomaly score map, and S(I)(i,j) represents the anomaly score at the i-th row and j-th column in the pixel-level anomaly score map;
[0095] 4.6) Set the threshold T based on all the original normal images in the MVTec training dataset:
[0096] 4.6.1) Calculate the pixel-level anomaly score map of all original normal images in the training data set and obtain the set in is the kth original normal image in the training dataset, Representing images The corresponding pixel-level anomaly score map, n represents the number of original normal images in the training dataset;
[0097] 4.6.2) Calculate the image-level anomaly score μ for each original normal image from the set of pixel-level anomaly scores k :
[0098]
[0099] Where w represents the width of the pixel-level anomaly score map, and h represents the height of the pixel-level anomaly score map. Indicates that in the original normal image The anomaly score at the i-th row and j-th column in the corresponding pixel-level anomaly score map;
[0100] 4.6.3) According to the image-level anomaly score of each original normal image, the image-level anomaly score set corresponding to all original normal images is obtained:
[0101] Ω={μ1,μ2,...μ k ,...,μ n}
[0102] 4.6.4) Select the maximum value from the original normal image image-level anomaly score set Ω as the threshold T:
[0103]
[0104] 4.7) Compare the image-level anomaly score η of the test image I with the set threshold T:
[0105] If η>T, image I is judged as abnormal;
[0106] If η≤T, the image I is judged to be normal, and the abnormality detection of the test image I is completed.
[0107] The effects of the present invention are further described below in conjunction with simulation experiments.
[0108] 1. Simulation experiment conditions:
[0109] The hardware platform of the simulation experiment of the present invention is: Intel Core i7-8700K@3.70GHZ, 32GB RAM, and the graphics card is NVIDIA GeForce GTX 1080Ti.
[0110] The software platform for the simulation experiment of the present invention is: Ubuntu 18.04 operating system, Pytorch 1.6.0, Python 3.6;
[0111] The simulation experiment uses the MVTec dataset, which includes a training dataset and a test dataset. This dataset is a benchmark dataset for evaluating the quality of image anomaly detection and positioning algorithms. It contains 15 different types of industrial products, including 5 types of texture type data: grids, tiles, carpets, wood and leather; and 10 types of object type data: bottle mouths, capsules, tablets, cables, transistors, zippers, hazelnuts, nuts, screws and toothbrushes.
[0112] 2. Simulation content and results analysis:
[0113] Simulation 1, using the present invention and four existing image anomaly localization methods, namely, the image anomaly localization method AE based on the SSIM loss function autoencoder SSIM , energy-iteration-based normal data manifold projection image anomaly localization method VAE-grad, multi-scale knowledge distillation image anomaly localization method MKD, image anomaly localization method US based on discriminative latent vector teacher-student network, respectively calculate the pixel-level anomaly score map for all test images in the MVTec test dataset, and calculate the area AUC enclosed by the pixel-level receiver operating characteristic curve and the horizontal axis of various methods. The results are shown in Table 1.
[0114] Table 1. Comparison of pixel-level AUC evaluation results (%) of five methods
[0115]
[0116] The pixel-level AUC is the area AUC enclosed by the pixel-level receiver operating characteristic curve and the horizontal axis, and is used to measure the effect of the abnormality localization method. Its value is between 0% and 100%. The larger the value, the better the localization effect.
[0117] As can be seen from Table 1, the average value of the pixel-level AUC index of the present invention on the test set is higher than that of the other four methods, which proves that the present invention has a better image anomaly localization effect.
[0118] Simulation 2, using the present invention and three existing image anomaly detection methods, namely, the image anomaly detection method AE based on the SSIM loss function autoencoder SSIM , anomaly detection method DSVDD based on deep support vector data description model, and image anomaly detection method VAE-grad based on energy iteration normal data manifold projection. Each method is used to calculate the image-level anomaly score for all test images in the MVTec test dataset, and the area AUC enclosed by the image-level receiver operating characteristic curve and the horizontal axis of various methods is calculated. The results are shown in Table 2.
[0119] Table 2. Comparison of image AUC evaluation results (%) of four methods
[0120]
[0121] The image-level AUC is the area AUC enclosed by the image-level receiver operating characteristic curve and the horizontal axis, and is used to measure the effect of the anomaly detection method. Its value is between 0% and 100%. The larger the value, the better the detection effect.
[0122] As can be seen from Table 2, the average value of the image-level AUC index of the present invention on the test set is higher than that of the other three methods, which proves that the present invention has a better image anomaly detection effect.
Claims
1. An image anomaly detection method based on soft mask contrast loss, It is characterized in that The steps include: (1) The MVTec industrial anomaly detection training dataset is processed using the superpixel segmentation algorithm and Gaussian noise to obtain a synthetic image training dataset: 1a) All normal image data in the MVTec training data set are segmented into superpixels using a superpixel segmentation algorithm to obtain superpixel images and segmentation mask images; 1b) Randomly select one twentieth of the number of superpixels from the segmented superpixel image, add Gaussian noise to it, obtain a synthetic image, and use the segmentation mask image corresponding to the selected superpixel as the mask image label of the synthetic image; 1c) Using the synthetic images and their corresponding mask labels as training datasets; (2) Build an image anomaly detection model consisting of a reconstruction autoencoder and a mask autoencoder in parallel; (3) Use synthetic image training dataset to train the image anomaly detection model: 3a) Calculate the predicted mask map output by the mask autoencoder The label mask M corresponding to the input synthetic abnormal image syn_ab The L2 error loss between MAE (I syn_ab ), and use the L2 error loss as the loss function, use the Adam back propagation optimization algorithm to train the mask autoencoder, and complete its training, where I syn_ab represents the input synthetic abnormal image, represents the masked autoencoder, ω MAE represents the parameters of the mask autoencoder; 3b) Train the reconstruction autoencoder once: 3b1) Calculate the reconstructed image output by the encoder Compared with the original normal image I nor The L2 error loss between res (I syn_ab ), and use the mask autoencoder obtained from one training to obtain the soft mask map: Among them represents the masked autoencoder obtained through one training, Represents the parameters of the mask autoencoder obtained through one training; 3b2) Multiply the soft mask image calculated in 3b1) by the synthetic abnormal image pixel by pixel to obtain the negative sample in the soft mask contrast loss: N = M soft *I syn_ab ; 3b3) Calculate the soft mask contrast loss L between the reconstructed image A of the reconstruction encoder, the negative sample N, and the original normal image P corresponding to the synthesized image CL (A,P,N), and the L2 error loss L res (I syn_ab ) and the soft mask contrast loss L CL The weighted sum of (A, P, N) is used as the loss function L of the reconstruction autoencoder RAE (I syn_ab ,P,N), and use the Adam back-propagation optimization algorithm to train and reconstruct the autoencoder to complete its first training; 3d) alternately executing 3a) and 3b) until the corresponding loss functions converge, stop iteration, and obtain a trained image anomaly detection model; (4) Image anomaly location and detection: 4a) Take the image I in the MVTec test dataset as input and input it into the trained image anomaly detection model, which outputs the predicted mask image S through the mask autoencoder mask (I), reconstruct the image by reconstructing the self-encoder output and passing the L2 error S L2 (I) and contrast error S CL (I) Obtain the abnormal score map S of the reconstructed autoencoder res (I); 4b) Predict the mask image S through the output of the mask autoencoder mask (I) and the anomaly score map S of the reconstructed autoencoder res (I), obtain the pixel-level anomaly score map S(I) for anomaly localization: S(I)=S res (I)+h · ·S mask (I) Among them, h · is the confidence parameter of the mask autoencoder; 4c) Select the maximum value in the pixel-level anomaly score map to obtain the image-level anomaly score η for anomaly detection: Where w represents the width of the pixel-level anomaly score map, h represents the height of the pixel-level anomaly score map, and S(I)(i,j) represents the anomaly score at the i-th row and j-th column in the pixel-level anomaly score map; 4d) Set a threshold T based on the original normal image of the MVTec training dataset, and compare the image-level anomaly score η of the test image I with the set threshold: If η>T, image I is judged as abnormal; If η≤T, the image I is judged to be normal.
2. The method according to claim 1, It is characterized in that Step 2) The structures of the reconstruction autoencoder and mask autoencoder in the image anomaly detection model are as follows; The reconstruction autoencoder is composed of a cascade of an encoder and a reconstruction decoder; The encoder is composed of a convolutional layer, a first maximum pooling layer, four residual modules and a second maximum pooling layer in cascade, wherein the convolution kernel size of the convolutional layer is 7×7, the step size is 2, and the padding pixel is 1; the step sizes of the two maximum pooling layers are both 2; the four residual modules are composed of 3, 4, 6, and 3 residual convolution groups respectively, and each residual convolution group is composed of the first convolutional layer and the second convolutional layer. The convolution kernel size of the two convolutional layers is 3×3, the step size is 1, and the padding pixel is 1. The output features of the second convolutional layer and its input features are added channel by channel to obtain the output features of the residual convolution group; The reconstruction decoder is composed of six decoding modules and a convolution module in cascade, wherein each decoding module is composed of a bilinear interpolation upsampling layer with a step size of 2, a third convolution layer, and a fourth convolution layer; the convolution module is composed of two convolution layers, namely, a fifth convolution layer and a sixth convolution layer; the convolution kernel size of the third convolution layer and the fourth convolution layer is 3×3, the step size is 1, and the padding pixel is 1; the convolution kernel size of the fifth convolution layer and the sixth convolution layer is 3×3, the step size is 2, and the padding pixel is 1; and the output channel is 3; The mask autoencoder includes an encoder and a mask encoder, and its structure is the same as that of the reconstruction autoencoder, and its output channel is 1.
3. The method according to claim 1, It is characterized in that 3a) Calculate the predicted mask map output by the mask autoencoder The label mask M corresponding to the input synthetic abnormal image syn_ab The L2 error loss between MAE (I syn_ab ), the formula is as follows: Where n is the number of data in the synthetic image dataset.
4. The method according to claim 1, It is characterized in that 3b1) Calculate the reconstructed image output by the encoder Compared with the original normal image I nor The L2 error loss between res (I syn_ab ), the formula is as follows: Where n is the number of data in the synthetic image dataset.
5. The method according to claim 1, It is characterized in that 3b3) calculates the soft mask contrast loss L between the reconstructed image A of the reconstruction encoder, the negative sample N and the original normal image P corresponding to the synthesized image CL (A,P,N), the formula is as follows: in, P=I nor , To reconstruct the autoencoder, ω RAE To reconstruct the weight parameters of the autoencoder, v i (A) represents the feature map obtained after the reconstructed image passes through the i-th VGG19 feature layer, v i (P) represents the feature map obtained after the original normal image passes through the i-th VGG19 feature layer, v i (N) represents the feature map obtained after the negative sample passes through the i-th VGG19 feature layer, d(v i (A),v i (P)) represents the L1 distance between the reconstructed image feature map and the original normal image feature map, d(v i (A),v i (N)) represents the L1 distance between the reconstructed image feature map and the negative sample feature map, w i represents the weight parameter of the soft mask contrast loss term under the i-th scale feature extraction layer, i ranges from {1,2,...,k}, k represents the number of feature extraction layers of different scales of VGG19 trained on the ImageNet public dataset, k=5, h represents the soft mask contrast loss The weight of the factor is as follows: in, 6. The method according to claim 1, It is characterized in that 3b3) Reconstruct the loss function L of the autoencoder RAE (I syn_ab ,P,N), expressed as follows: L RAE (I syn_ab ,P,N)=L res (I syn_ab )+λ·L CL (A,P,N) Among them, λ is the weight hyperparameter that balances the reconstruction loss and the soft mask contrast loss, I syn_ab represents a synthetic abnormal image, P represents an original normal image, and N represents a negative sample. represents the reconstructed image, represents the reconstruction autoencoder, ω RAE Represents the weight parameters of the reconstructed autoencoder.
7. The method according to claim 1, It is characterized in that 4a) The predicted mask image S output by the mask autoencoder mask (I) and the anomaly score map S of the reconstructed autoencoder res (I), respectively, as follows: S res (I)=S L2 (I)+λ·S CL (I) Where I represents the images in the MVTec test dataset, represents the trained mask autoencoder, represents the weight parameter of the mask autoencoder, and λ is the balanced L2 error S L2 (I) and contrast error S CL The weight hyperparameters of (I), represents the trained reconstruction autoencoder, Represents the obtained reconstruction autoencoder weight parameter, w i represents the weight parameter of the soft mask contrast loss term under the i-th scale feature extraction layer, i ranges from {1,2,...,k}, k represents the number of feature extraction layers of different scales of VGG19 trained on the ImageNet public dataset, k=5, that is, the weight parameters are v i (I) represents the feature map obtained after the test image passes through the i-th VGG19 feature layer, Represents the feature map obtained after the reconstructed image passes through the i-th VGG19 feature layer, represents the L1 distance between the reconstructed image feature map and the original normal image feature map, It means that the L1 distance between the reconstructed image feature map and the original normal image feature map is upsampled to the test image size.
8. The method according to claim 1, It is characterized in that The confidence parameter h· of the masked autoencoder in 4b) is calculated as: in, represents the trained mask autoencoder, Represents the weight parameters of the mask autoencoder.
Citation Information
Patent Citations
Anomaly detection system and method based on generative adversarial network and storage medium
CN113744265A
Surface anomaly detection method based on mixed supervised learning
CN113870230A
Image anomaly detection method based on deep convolutional generative adversarial network
CN113658115A
Image anomaly detection and anomaly positioning method and system based on self-supervised mask
CN114022475A
