Unsupervised anomaly detection method based on adaptive diffusion model

Abstract

The unsupervised anomaly detection method based on an adaptive diffusion model belongs to the field of industrial image anomaly detection. The method solves the problems of the traditional unsupervised anomaly detection algorithm, such as the poor image reconstruction effect caused by the traditional training method for training the diffusion model, and the low image anomaly detection accuracy caused by the inability of the image reconstruction process to adaptively add noise. The method uses normal images and synthetic abnormal images to train the diffusion model, supervises the model optimization, and enhances the model reconstruction capability. The trained diffusion model is used to perform step-by-step adaptive denoising and reconstruction on the input image to be detected, the optimal denoising step number is determined in the denoising process, different steps of noise are adaptively added for different abnormalities, and finally, the feature space comparison is performed on the images before and after reconstruction to obtain a spatial comparison abnormal image and an abnormal score. The method is mainly used for image anomaly detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial image anomaly detection. BACKGROUND

[0002] Anomaly detection on product surface is crucial in industrial manufacturing. However, there are various types of anomalies for industrial products, and the design and production process of products are constantly changing. It is very difficult to collect all types of anomaly images in actual production. Therefore, unsupervised anomaly detection methods using only normal images have attracted extensive attention.

[0003] Existing reconstruction-based industrial anomaly detection methods assume that the image reconstruction model trained on normal images can only reconstruct the normal regions of the input image, while the abnormal regions will not be reconstructed. By comparing the two images before and after reconstruction, the abnormal regions are detected and located. Some advanced industrial anomaly detection methods use diffusion models as image reconstruction models. First, a diffusion model is trained using normal samples, and then a sample with added random noise is reconstructed. Because the diffusion model only models the distribution of normal samples during training, the model can reconstruct the input noisy image into a normal image after adding enough noise. These methods achieve high-quality image reconstruction, but still have some shortcomings.

[0004] First, the current unsupervised anomaly detection algorithm uses a traditional diffusion model training method, which can only use normal samples for training, and then use the trained diffusion model to reconstruct normal images by predicting and removing the added Gaussian noise in the input normal samples.

[0005] However, image reconstruction is still needed during the actual application stage to reconstruct abnormal images, so there is a difference in data distribution between the training stage and the actual application stage. For normal regions, it is reasonable to predict and remove Gaussian noise, but for abnormal regions, predicting and removing a Gaussian noise may still reconstruct the abnormality, and the traditional diffusion model training method is not suitable for anomaly detection, resulting in poor reconstruction effect during the actual application stage and low accuracy of anomaly detection. Therefore, how to train the diffusion model using synthetic abnormal samples is a problem to be solved.

[0006] In addition, during the reconstruction process, these methods using diffusion models to reconstruct add a fixed number of random noises to the input image, which is not suitable for all abnormal samples. Compared to subtle scratches and other inconspicuous abnormalities, some obvious abnormal types of missing parts need to add more noise to ensure that the reconstructed image does not contain abnormal regions. Therefore, this method of adding a fixed number of random noises during the reconstruction of the image is not a universal solution. In practice, different degrees of noise need to be adaptively added for different types of abnormalities. SUMMARY

[0007] The application aims to solve the problems of poor image reconstruction effect caused by training the diffusion model using the traditional training method, and low image anomaly detection accuracy caused by the inability of the image reconstruction process to adaptively add noise, and provides an unsupervised anomaly detection method based on an adaptive diffusion model.

[0008] The unsupervised anomaly detection method based on the adaptive diffusion model comprises the following steps:

[0009] Step 1, using each original normal image I n to synthesize a corresponding abnormal image I a , and taking each original normal image I n and its corresponding abnormal image I a as a sample;

[0010] Step 2, pre-processing each sample:

[0011] encoding the abnormal image I a in each sample to obtain a feature map x a , adding Gaussian noise ε of a random step number S to the feature map x a to obtain a feature map x encoding the original normal image I n in each sample to obtain a feature map x'; combining the feature map x' after adding Gaussian noise ε of a random step number S with the feature map x a , and taking the combined result as the supervised label data of the sample;

[0012] Step 3, training the diffusion model:

[0013] taking the feature map x of each sample after adding noise as the input data of the diffusion model, taking the supervised label data of the sample as the true value of the diffusion model, and performing noise prediction training on the diffusion model to obtain the trained diffusion model;

[0014] Step 4, using the trained diffusion model to perform adaptive denoising to realize image reconstruction:

[0015] encoding the to-be-detected image I to obtain a feature map x, adding Gaussian noise ε of a fixed step number T to the feature map x to obtain a feature map x T , and inputting the feature map x T to the trained diffusion model to perform first-round step-by-step noise prediction and denoising to determine the optimal denoising step number;

[0016] The abnormal region positioning map under the optimal denoising step number is combined with the normal region positioning map to obtain a combined feature map;

[0017] After adding the noise predicted by the diffusion model under the optimal denoising step number to the combined feature map, the combined feature map is input into the trained diffusion model to perform a second round of step-by-step noise prediction and denoising, and after the noise removal is completed, decoding is performed to obtain a clear abnormal-free image, which is a reconstructed image

[0018] Step 5, performing feature space comparison on the to-be-detected image I and the reconstructed image to obtain a spatial comparison abnormal image, and obtaining an abnormal score according to the spatial comparison abnormal image, so as to complete the anomaly detection.

[0019] Preferably, in step 1, each original normal image I n is used to synthesize a corresponding abnormal image I a The implementation manner includes:

[0020] Step 11, generating an abnormal region on each original normal image I n

[0021] Step 12, generating an enhanced image after image enhancement is performed on the original normal image I n ; randomly selecting one of the enhanced image or a texture image in the texture data set as an abnormal image and adding it to the abnormal region to generate an abnormal image;

[0022] Preferably, in step 5, the to-be-detected image I and the reconstructed image are compared in the feature space to obtain a spatial comparison abnormal image, and the implementation manner includes:

[0023] The to-be-detected image I is subjected to feature extraction to obtain M-layer image features F I ;

[0024] The reconstructed image is subjected to feature extraction to obtain M-layer image features

[0025] The M-layer image features F I and the M-layer image features have a one-to-one correspondence;

[0026] The feature of each position in the image feature F I of each layer is respectively calculated with the features of all positions in the image feature of the corresponding layer, and the minimum value of all the cosine distances is taken as the abnormal value of the position in the abnormal image of the corresponding layer;

[0027] ​By overlaying outliers at corresponding locations in the M-layer anomaly map, a spatial contrast anomaly map is obtained.

[0028] Preferably, the image to be detected I and the reconstructed image Feature extraction is performed using a feature extraction network, specifically DINO.

[0029] Preferably, in step 5, the method for obtaining the anomaly score based on the spatial comparison anomaly map includes:

[0030] The mean of the top K largest outliers in the spatial contrast anomaly map is taken, and this mean is used as the anomaly score of the spatial contrast anomaly map.

[0031] Preferably, the diffusion model is implemented using the UNet network, and the loss function L of its anomalous training paradigm is... ATP The expression is:

[0032]

[0033] in, In the known training data (x′, x a In the case of ), the maximum likelihood estimate of the model parameter θ, where N(0,I) is the standard normal distribution followed by the added noise, 0 is the mean of the standard normal distribution, I is the variance of the standard normal distribution, and p data For the distribution of input data, ε is a parameter that is pre-set and decreases as the number of random steps S increases. θ Here, ||·||2 represents the noise predicted by the diffusion model, and ||·||2 represents the L2 norm.

[0034] Preferably, in step 4, the feature map x after adding noise... T The implementation methods for determining the optimal number of denoising steps in the first round of progressive noise prediction and denoising include:

[0035] Step 411, the first noise prediction and denoising process in the first round of progressive noise prediction and denoising:

[0036] The trained diffusion model receives the feature map x after adding noise. T Then, noise prediction is performed, and the noise ε is predicted. θ1 Using noise ε θ1 In feature map x T After removing noise, we obtain the predicted noisy feature map.

[0037] Step 412: Add Gaussian noise ε of step number Ti to the feature map x to obtain the real noisy feature map x. T-i The initial value of i is 1, and it is less than T;

[0038] Step 413, the i+1th noise prediction and denoising in the first round of step-by-step noise prediction and denoising process:

[0039] The predicted noisy feature map is input into the trained diffusion model, and noise prediction is performed to predict the noise θ(i+1) ;

[0040] Step 414, feature space comparison:

[0041] After the predicted noisy feature map θ(i+1) and the real noisy feature map x T-i are completely denoised using the noise , the predicted clear feature map and the real clear feature map x T-i→0 are obtained respectively, the predicted clear feature map and the real clear feature map x T-i→0 are compared in feature space, and the anomaly score f i is obtained according to the obtained spatial comparison anomaly map, it is judged whether f i ≥ δ, δ is a threshold value, the result is yes, then the optimal denoising step number is determined as T-i = T-t, t is the total number of the first round of step-by-step denoising, and the spatial comparison anomaly map under the optimal denoising step number T-t is taken as the anomaly region positioning map m, otherwise, step 415 is executed.

[0042] Step 415, denoising the predicted noisy feature map θ(i+1) by one step using the noise to obtain the predicted noisy feature map , and then i = i+1 is executed. Step 412.

[0043] Preferably, in step 414, after the predicted clear feature map and the real clear feature map x T-i→0 are compared in feature space, the anomaly score f i is obtained according to the obtained spatial comparison anomaly map, the implementation manner includes:

[0044] Feature extraction is performed on the predicted clear feature map to obtain M-layer predicted clear image features;

[0045] Feature extraction is performed on the real clear feature map x T-i→0 to obtain M-layer real clear image features;

[0046] The M-layer predicted clear image features and the M-layer real clear image features have a one-to-one correspondence;

[0047] The cosine distance is calculated between each position in the feature of each layer of the predicted clear image feature and the feature of all positions in the real clear image feature of the corresponding layer, and the minimum value of all cosine distances is taken as the anomaly value of the corresponding position in the anomaly map of the corresponding layer; the anomaly values of the corresponding positions in the M-layer anomaly map are superimposed to obtain a spatial contrast anomaly map; the mean of the first K maximum anomaly values in the spatial contrast anomaly map is taken as the anomaly score f of the spatial contrast anomaly map i .

[0048] Preferably, in step 4, the feature synthesis is performed on the anomaly region positioning map and the normal region positioning map under the optimal denoising step number to obtain a synthesized feature map, and the implementation manner of the synthesized feature map includes:

[0049] Step 421, subtracting the pixel value of the corresponding pixel point of the anomaly region positioning map m from the image with all pixel values being 1 to obtain a normal region positioning map Figure 1 m; the anomaly region positioning map m has the same size as the image with all pixel values being 1;

[0050] Step 422, multiplying the anomaly value at each position in the anomaly region positioning map m by the feature at the corresponding position in the predicted clear feature map corresponding to the optimal denoising step number to obtain an anomaly region feature map;

[0051] The normal region positioning map Figure 1 m; multiplying the value at each position in the normal region positioning map by the feature at the corresponding position in the real clear feature map corresponding to the optimal denoising step number to obtain a normal region feature map;

[0052] Step 423, adding the values of the corresponding positions of the anomaly region feature map and the normal region feature map to obtain a synthesized feature map.

[0053] Preferably, in step 4, after adding the noise predicted by the diffusion model under the optimal denoising step number to the synthesized feature map, the synthesized feature map is input into the trained diffusion model for a second round of step-by-step noise prediction and denoising, and after the noise removal is completed, decoding is performed to obtain a clear and anomaly-free image, which is the reconstructed image The implementation manner of the reconstructed image includes:

[0054] Step 431, adding the noise predicted by the diffusion model under the optimal denoising step number to the synthesized feature map to form a predicted noisy feature map t is the total number of the first round of step-by-step denoising;

[0055] Step 432, j-th noise prediction and denoising in the second round of step-by-step noise prediction and denoising:

[0056] The initial value of j is 1, and the predicted noisy feature map is input into the trained diffusion model, noise prediction is performed, and noise ε θj, utilize the noise θj In the prediction of the noisy feature map The noise prediction and denoising step is added to the predicted noisy feature map ε θj The noise predicted by the jth noise prediction of the diffusion model in the second round of step-by-step noise prediction and denoising process.

[0057] Step 433, the j+1th noise prediction and denoising in the second round of step-by-step noise prediction and denoising process:

[0058] The predicted noisy feature map After inputting the predicted noisy feature map into the trained diffusion model, the noise is predicted, and the noise θ(j+1) , utilize the noise θ(j+1) In the prediction of the noisy feature map The noise prediction and denoising step is added to the predicted noisy feature map

[0059] Step 434, determine whether j+1=T-t is reached, and the result is yes, which means that the complete denoising of the synthesized feature map with the optimal number of added denoising steps is completed, and the synthesized feature map after denoising at this time is decoded to obtain a clear and abnormal image, which is the reconstructed image The result is no, and j=j+1 is set, and step 433 is executed.

[0060] The advantages of the present application are:

[0061] 1. The unsupervised anomaly detection method based on the adaptive diffusion model of the present application introduces a synthesized abnormal image training method in the training process of the diffusion model, generates supervised label data using noisy normal images and synthesized abnormal images, and supervises the diffusion model optimization process, so that the model can better reconstruct the abnormal area as a normal area, and the reconstruction ability of the model is improved. In addition, in the inference process, the optimal denoising step is determined by comparing the information difference between the predicted noisy feature map and the noisy real noisy feature map, and different steps of noise are added adaptively for different abnormalities.

[0062] Finally, the clear and abnormal image after reconstruction is obtained based on the original input image, and the difference between the reconstructed image and the original input image is compared to locate and identify the abnormality.

[0063] 2. The unsupervised anomaly detection method based on the adaptive diffusion model of the present application proposes a diffusion model training method suitable for abnormal samples, enhances the reconstruction ability of the model, and improves the accuracy of anomaly detection and positioning.

[0064] 3、The application first realizes the adaptive selection of the number of denoising steps of the to-be-detected image, so that the diffusion model can ensure abnormal reconstruction while maximizing the retention of normal region information in the to-be-detected image. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 A principle diagram for abnormal detection of a to-be-detected image I by using the unsupervised anomaly detection method based on the adaptive diffusion model of the application;

[0066] Figure 2 A flowchart for training the diffusion model;

[0067] Figure 3 A principle diagram for image reconstruction by adaptive denoising;

[0068] Figure 4 A principle diagram for feature space comparison. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the application.

[0070] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0071] In view of the problems that the existing unsupervised anomaly detection algorithm uses a traditional training method to train the diffusion model, only normal images can be used for training, the model has limited ability to reconstruct abnormal regions as normal regions, and on the other hand, the image reconstruction process cannot adaptively add noise, further leading to poor reconstruction effect and ultimately leading to low accuracy of image anomaly detection, the application provides an unsupervised anomaly detection method based on an adaptive diffusion model. The anomaly detection method realizes abnormal reconstruction while retaining normal region information to the greatest extent, and improves the accuracy of anomaly detection and positioning.

[0072] Reference Figure 1 and Figure 2 The present embodiment is based on an unsupervised anomaly detection method based on an adaptive diffusion model, which includes the following steps:

[0073] Step 1, using each original normal image I n to synthesize a corresponding abnormal image I a , and each original normal image I n and its corresponding abnormal image I aAs a sample;

[0074] As a preferred example, the original normal images I n are used to synthesize the corresponding abnormal images I a The implementation includes:

[0075] Step 11, generating abnormal regions on the original normal images I n

[0076] Step 12, after image enhancement on the original normal images I n , generating an enhanced image; randomly selecting one of the enhanced image or the texture image in the texture data set as an abnormal image, adding it to the abnormal region to generate an abnormal image;

[0077] Step 2, pre-processing each sample:

[0078] Encoding the abnormal image I a in each sample to obtain a feature map x a , adding Gaussian noise ε of a random step number S to the feature map x a to obtain a feature map x Encoding the original normal image I n in each sample to obtain a feature map x'; after adding Gaussian noise ε of a random step number S to the feature map x', combining it with the feature map x a , and the combined result is used as the supervision label data of the sample;

[0079] Step 3, training the diffusion model:

[0080] The feature map x of each sample after adding noise is used as the input data of the diffusion model, and the supervision label data of the sample is used as the true value of the diffusion model. The diffusion model is trained for noise prediction to obtain the trained diffusion model;

[0081] The above introduces the training of the diffusion model using the synthesized abnormal image, and uses the abnormal information in the synthesized abnormal image after adding Gaussian noise of a random step number S on the normal image as the supervision label data to supervise the optimization process of the diffusion model, overcoming the defect that the diffusion model in the prior art cannot be trained using abnormal images, and ensuring the accuracy of image reconstruction by the subsequent diffusion model;

[0082] Step 4, using the trained diffusion model for adaptive denoising to realize image reconstruction:

[0083] Encoding the to-be-detected image I to obtain a feature map x, adding Gaussian noise ε of a fixed step number T to the feature map x to obtain a feature map x T ​The feature graph x T The first round of step-by-step noise prediction and denoising is performed to determine the optimal denoising step number.

[0084] The feature synthesis is performed on the abnormal region positioning map and the normal region positioning map under the optimal denoising step number to obtain a synthesized feature graph.

[0085] After adding the noise predicted by the diffusion model under the optimal denoising step number to the synthesized feature graph, the second round of step-by-step noise prediction and denoising is performed on the trained diffusion model, and after the noise removal is completed, decoding is performed to obtain a clear abnormal-free image, which is a reconstructed image.

[0086] Step 5, the to-be-detected image I and the reconstructed image The feature space comparison is performed to obtain a spatial comparison abnormal image, and then an abnormal score is obtained according to the spatial comparison abnormal image, so that the abnormal detection is completed.

[0087] The present application performs two rounds of step-by-step noise prediction and denoising, the first round of step-by-step noise prediction and denoising is to obtain the optimal denoising step number, and the purpose of determining the optimal denoising step number is to add the optimal step number to the input sample to ensure that most abnormal information can be covered while retaining the most normal region information, and the second round of step-by-step noise prediction and denoising is to perform step-by-step denoising on the synthesized feature graph obtained under the optimal denoising step number, the synthesized feature graph adds the noise predicted by the diffusion model under the optimal denoising step number, and the synthesized feature graph removes the residual abnormal information in the optimal step number, which can ensure the accuracy of image reconstruction.

[0088] The pixel value of each pixel point of the image after the feature space comparison of the present application is a value between 0 and 1.

[0089] Referring to Figure 3 , a preferred embodiment is provided, and the feature graph x T The implementation manner of the first round of step-by-step noise prediction and denoising to determine the optimal denoising step number includes:

[0090] Step 411, the first noise prediction and denoising in the first round of step-by-step noise prediction and denoising:

[0091] After the trained diffusion model receives the feature graph x T , noise prediction is performed to predict noise ε θ1 , the noise ε θ1 is used to denoise the feature graph x T by one step to obtain a predicted noise-containing feature graph

[0092] Step 412, add the Gaussian noise ε of the step number T-i to the feature map x to obtain the real noisy feature map x T-i , the initial value of i is 1 and less than T;

[0093] Step 413, the i+1th noise prediction and denoising in the first round of step-by-step noise prediction and denoising:

[0094] After inputting the predicted noisy feature map x into the trained diffusion model, noise prediction is performed to predict the noise ε θ(i+1) ;

[0095] Step 414, feature space comparison:

[0096] After using the noise ε θ(i+1) to completely denoise the predicted noisy feature map x and the real noisy feature map x T-i , the predicted clear feature map x and the real clear feature map x T-i→0 are obtained, respectively, after the feature space comparison between the predicted clear feature map x and the real clear feature map x T-i→0 , the abnormal score f i is obtained according to the obtained space comparison anomaly map, and it is judged whether f i ≥ δ, δ is a threshold value, and the result is yes, then the optimal denoising step number is T-i = T-t, t is the total number of the first round of step-by-step denoising, and the space comparison anomaly map under the optimal denoising step number T-t is taken as the abnormal region positioning map m, otherwise, step 415 is executed;

[0097] Step 415, use the noise ε θ(i+1) to denoise the predicted noisy feature map x by one step to obtain the predicted noisy feature map x , then let i = i+1, and execute step 412.

[0098] In the preferred embodiment, for each noise prediction, the predicted clear feature map formed after completely removing the predicted noise, and the real clear feature map formed after completely denoising the real noisy feature map containing the Gaussian noise of the corresponding step number are compared, the information difference between the completely denoised predicted clear feature map and the real clear feature map is compared to determine the optimal denoising step number, the comparison between the noisy feature and the clear feature is converted into the comparison between the clear features, since the predicted clear feature map does not contain abnormal information, whether the real clear feature map contains a large amount of abnormal information can be detected more simply and efficiently by comparing the two clear feature maps, so that the optimal denoising step number can be accurately predicted.

[0099] Furthermore, the predicted clear feature map x and the real clear feature map x T-i→0 After the feature space comparison, the abnormal score f is obtained according to the obtained space comparison anomaly map i The implementation mode of the real clear feature map x

[0100] The predicted clear feature map x is extracted to obtain M-layer predicted clear image features;

[0101] The real clear feature map x T-i→0 is extracted to obtain M-layer real clear image features;

[0102] The M-layer predicted clear image features and the M-layer real clear image features have a one-to-one correspondence;

[0103] The cosine distance is calculated between the feature of each position in each layer of the predicted clear image feature and the features of all positions in the real clear image feature of the corresponding layer, and the minimum value of all cosine distances is taken as the abnormal value of the corresponding position in the anomaly map of the corresponding layer. The corresponding position abnormal values in the M-layer anomaly map are superimposed to obtain a space comparison anomaly map; the mean value of the first K maximum abnormal values in the space comparison anomaly map is taken as the abnormal score f of the space comparison anomaly map i .

[0104] In the preferred embodiment, the feature of each position in each layer of the predicted clear image feature and the features of all positions in the real clear image feature of the corresponding layer are vectors with a length of n.

[0105] Referring to Figure 3 , a preferred embodiment is further provided, and in step 4, the anomaly region positioning map and the normal region positioning map under the optimal denoising step number are used for feature synthesis to obtain the implementation mode of the synthesized feature map, which includes:

[0106] Step 421, subtracting the pixel value of the corresponding pixel point of the anomaly region positioning map m from the map with all pixel values being 1 to obtain the normal region positioning Figure 1 The anomaly region positioning map m and the map with all pixel values being 1 are the same in size;

[0107] Step 422, multiplying the abnormal value of each position on the anomaly region positioning map m by the feature of the corresponding position in the predicted clear feature map corresponding to the optimal denoising step number to obtain an anomaly region feature map; the feature of each position on the predicted clear feature map is a vector with a length of n;

[0108] The normal region positioning Figure 1 -m is multiplied by the feature of the corresponding position in the real clear feature map corresponding to the optimal denoising step number to obtain a normal region feature map; the feature of the corresponding position in the clear feature map is a vector with a length of n;

[0109] Step 423, add the values of the abnormal region feature map and the normal region feature map at the corresponding positions to obtain a synthesized feature map.

[0110] In the preferred embodiment, the abnormal region positioning map m, the normal region positioning map n, the abnormal region feature map and the normal region feature map are all images of the same size. Figure 1 The predicted clear feature map corresponding to the optimal denoising step number m and the real clear feature map corresponding to the optimal denoising step number m are both images of the same size. The advantage of using the above step of synthesizing a feature map is that a feature map completely free of abnormal information can be generated, thereby ensuring the reconstruction of an image free of abnormalities.

[0111] Referring to Figure 3 , a preferred embodiment is provided. In step 4, after adding the noise predicted by the diffusion model at the optimal denoising step number to the synthesized feature map, the synthesized feature map is input into the trained diffusion model to perform a second round of step-by-step noise prediction and denoising. After the noise is removed, decoding is performed to obtain a clear image free of abnormalities, which is a reconstructed image The implementation of the method includes the following steps:

[0112] Step 431, after adding the noise predicted by the diffusion model at the optimal denoising step number to the synthesized feature map, a predicted noisy feature map is formed t is the total number of first-round step-by-step denoising;

[0113] Step 432, j-th noise prediction and denoising in the second-round step-by-step noise prediction and denoising process:

[0114] The initial value of j is 1. After the predicted noisy feature map is input into the trained diffusion model, noise prediction is performed to predict noise ε θj . The noise ε θj is used to denoise the predicted noisy feature map by one step to obtain a predicted noisy feature map ε θj is the noise predicted by the j-th noise prediction of the diffusion model in the second-round step-by-step noise prediction and denoising process.

[0115] Step 433, j+1-th noise prediction and denoising in the second-round step-by-step noise prediction and denoising process:

[0116] After the predicted noisy feature map is input into the trained diffusion model, noise prediction is performed to predict noise ε θ(j+1) . The noise ε θ(j+1) is used to denoise the predicted noisy feature map by one step to obtain a predicted noisy feature map

[0117] Step 434, judging whether j+1=T-t is reached, and the result is yes, which means that the full denoising of the synthesized feature map added with the noise predicted by the diffusion model under the optimal denoising step number is completed, and the synthesized feature map after denoising at this time is decoded to obtain a clear and abnormal-free image, which is the reconstructed image. The result is no, and j=j+1 is set, and step 433 is executed.

[0118] In the preferred embodiment, after the synthesized feature map is added with the noise predicted by the diffusion model under the optimal denoising step number, step 431 to step 434 are used to perform step-by-step noise prediction and denoising, the synthesized feature map is added with the noise predicted by the diffusion model under the optimal denoising step number, the synthesized feature map removes the residual abnormal information in the optimal step number, and the accuracy of image reconstruction can be ensured.

[0119] Referring to Figure 4 Further, in step 5, the to-be-detected image I and the reconstructed image are compared in a feature space to obtain a spatial comparison abnormal image, and the implementation manner includes:

[0120] The to-be-detected image I is subjected to feature extraction to obtain M-layer image features F I ;

[0121] The reconstructed image is subjected to feature extraction to obtain M-layer image features

[0122] The M-layer image features F I have a one-to-one correspondence with the M-layer image features ;

[0123] The features of each position in each layer of the image features F I are respectively calculated with the features of all positions in the corresponding layer of the image features , and the minimum value of all cosine distances is taken as the abnormal value of the corresponding position in the corresponding layer of the abnormal image; and the corresponding position abnormal values in the M-layer abnormal image are superimposed to obtain a spatial comparison abnormal image.

[0124] As a preferred example, the to-be-detected image I and the reconstructed image are subjected to feature extraction by using a feature extraction network, and the feature extraction network can be DINO.

[0125] Further, in step 5, the implementation manner of obtaining an abnormal score according to the spatial comparison abnormal image includes:

[0126] The mean value of the first K maximum abnormal values in the spatial comparison abnormal image is taken as the abnormal score obtained by the spatial comparison abnormal image.

[0127] Principle analysis: the application uses normal images and synthetic abnormal images to train the diffusion model; the combination of noisy normal images and synthetic abnormal images is used as supervised label data to supervise the model optimization process; the trained diffusion model is used to perform step-by-step adaptive denoising reconstruction on the input image to be detected, the optimal denoising step number is determined in the denoising process, different step numbers of noise are adaptively added for different abnormalities; finally, the abnormal score is obtained by comparing the feature space of the image to be detected I and the reconstructed image The feature space is compared to obtain a spatial contrast abnormal image; then, an abnormal score is obtained according to the spatial contrast abnormal image, so that the abnormal detection is completed.

[0128] Although the present application is described herein with reference to particular embodiments, it is to be understood that these examples are merely illustrative of principles and applications of the present application. It should therefore be understood that numerous modifications can be made to the illustrative embodiments and that other arrangements can be devised without departing from the spirit and scope of the present application as defined by the appended claims. It should be understood that the features described in connection with one embodiment can be used in conjunction with other embodiments described herein. It should also be understood that features described in connection with separate embodiments can be used in other described embodiments.

Claims

1. An unsupervised anomaly detection method based on an adaptive diffusion model, characterized in that, The method comprises the following steps: Step 1, using each original normal image to synthesize a corresponding abnormal image and each original normal image with its corresponding abnormal image as a sample; Step 2, pretreatment of each sample: Abnormal images in each sample are encoded to get feature maps Abnormal images in each sample are encoded to get feature maps , and a random step number of Gaussian noise is added to the feature maps , to get feature maps with noise , to get feature maps with noise ;​ For each sample, the original normal image Encoding is performed to obtain feature maps In the feature map Add random steps Gaussian noise After and feature map The data is combined, and the combined result is used as the supervision label data for this sample. Step 3, training of the diffusion model: the feature map of each sample after adding noise As input data of the diffusion model, the supervised label data of the sample is taken as the true value of the diffusion model, the diffusion model is trained for noise prediction, and a trained diffusion model is obtained. Step 4, image reconstruction by using the trained diffusion model for adaptive denoising: Image to be detected Encoding is performed to obtain feature maps In the feature map Add a fixed number of steps Gaussian noise The resulting feature map with added noise The input is fed into the trained diffusion model to add noise to the feature map. Perform the first round of progressive noise prediction and denoising to determine the optimal number of denoising steps; The implementation mode of the determination of the optimal denoising step number comprises: Step 411, the first noise prediction and denoising in the first round of step-by-step noise prediction and denoising: The trained diffusion model receives the feature map added with noise Afterwards, noise prediction is performed to predict noise , and the noise is removed from the feature map in one step to obtain a predicted noisy feature map ; Step 412, adding steps to the feature map upper Gaussian noise , obtaining a real noisy feature map , The initial value of the step is 1, and is less than ; Step 413, the first round of step-by-step noise prediction and denoising process in the first noise prediction and denoising: Predicting a noisy feature map After inputting to the trained diffusion model, noise prediction is performed, and noise ; Step 414, feature space comparison: Utilizing noise On the predicted noisy feature map And the real noisy feature map After all denoising, the predicted clear feature map And the real clear feature map , the predicted clear feature map And the real clear feature map After feature space comparison, the anomaly score is obtained according to the obtained space comparison anomaly map , judge whether there is , Threshold, the result is yes, then determine the optimal denoising step number , The total number of the first round of step-by-step denoising, and the space comparison anomaly map under the optimal denoising step number As an anomaly region positioning map , otherwise, execute step 415; Step 415, using noise In predicting the noisy feature map The up de-noising step, get the predicted noisy feature map Then, let Step 412 is executed; Then, the feature synthesis is performed on the abnormal region positioning map and the normal region positioning map under the optimal denoising step number, and a synthesized feature map is obtained. After adding the noise predicted by the diffusion model under the optimal denoising step number to the synthetic feature map, input it into the trained diffusion model for a second round of step-by-step noise prediction and denoising, and then decode to obtain a clear and abnormal-free image, which is the reconstructed image ; specifically comprising: Step 431, after adding the noise predicted by the diffusion model under the optimal denoising step number to the synthesized feature map, a predicted noisy feature map is formed ; is the total number of first-round gradual denoising Step 432, the second round of step-by-step noise prediction and denoising process and the first noise prediction and denoising: The initial value of the prediction noise is 1, and the prediction noise feature map is input into the trained diffusion model to perform noise prediction, and the noise is predicted. The predicted noise feature map is denoised to obtain the predicted noise feature map . The predicted noise is the noise predicted by the diffusion model in the first step of the second round of gradual noise prediction and denoising. ​ Step 433, the second round of step-by-step noise prediction and denoising process in the first noise prediction and denoising: Predicting a noisy feature map After inputting to the trained diffusion model, noise prediction is performed to predict noise , and the noise is removed from the predicted noisy feature map to obtain a predicted clean feature map ; Step 434, judging whether it reaches , the result is yes, it is determined that the whole denoising of the synthesized feature map with the optimal number of added denoising steps is completed, and the synthesized feature map after denoising at this time is decoded to obtain a clear and abnormal-free image, which is a reconstructed image ; the result is no, let , step 433 is executed; Step 5, performing feature space comparison on the to-be-detected image and the reconstructed image to obtain a spatial comparison anomaly graph; and obtaining an anomaly score according to the spatial comparison anomaly graph, so as to complete anomaly detection.

2. The unsupervised anomaly detection method based on adaptive diffusion model according to claim 1, characterized in that, In step 1, each original normal image is used to synthesize a corresponding abnormal image The implementation includes: Step 11, generating abnormal regions on each original normal image Step 11, generating abnormal regions on each original normal image Step 12, original normal image After image enhancement, an enhanced image is generated; randomly select one from the enhanced image or the texture image in the texture data set as an abnormal image, and add it to the abnormal area to generate an abnormal image.

3. The unsupervised anomaly detection method based on adaptive diffusion model according to claim 1, characterized in that, In step 5, the to-be-detected image and the reconstructed image are compared in a feature space to obtain a spatial comparison anomaly map, and the implementation manner comprises: To the image to be detected Feature extraction is performed to obtain M-layer image features ; reconstructing the image performing feature extraction to obtain M-layer image features ; M-layer image features with M-layer image features there is a one-to-one correspondence; image features of each layer the features of each position in the image features of its corresponding layer the cosine distance between the features of all positions is calculated, and the minimum value of all cosine distances is taken as the anomaly value of the position in the anomaly map of the corresponding layer; The corresponding position abnormal values in the M-layer abnormal map are superimposed to obtain a spatial comparison abnormal map.

4. The unsupervised anomaly detection method based on adaptive diffusion model according to claim 3, characterized in that, The feature extraction is performed on the to-be-detected image and the reconstructed image The feature extraction is implemented by a feature extraction network, and the feature extraction network is .

5. The unsupervised anomaly detection method based on adaptive diffusion model according to claim 1, characterized in that, In step 5, the implementation mode of the abnormal score obtained from the spatial comparison abnormal map comprises: Take the front of the spatial comparison anomaly map The mean of the largest outliers is used as the basis for an anomaly score in a spatial contrast anomaly map.

6. The unsupervised anomaly detection method based on adaptive diffusion model according to claim 1, characterized in that, The diffusion model is implemented by using a UNet network, and the loss function of the abnormal training paradigm of the diffusion model is The expression is: wherein, is the maximum likelihood estimate of the model parameters given the known training data is the standard normal distribution from which the added noise is drawn, is the mean of the standard normal distribution, is the variance of the standard normal distribution, is the distribution of the input data, is a parameter that is set in advance and decreases as the number of random steps increases, is the noise predicted by the diffusion model, is the two-norm.

7. The unsupervised anomaly detection method based on adaptive diffusion model according to claim 1, characterized in that, In step 414, the predicted clear feature map and the real clear feature map After the feature space comparison, the anomaly score is obtained according to the obtained space comparison anomaly map The implementation manner comprises the following steps. To predict the clear feature map Feature extraction is performed to obtain M-layer predicted clear image features; To real clear feature map Feature extraction is performed to obtain M-layer real clear image features; The M-layer predicted clear image features and the M-layer real clear image features have a one-to-one correspondence; The cosine distance is calculated between each position in the predicted clear image feature of each layer and the features of all positions in the real clear image feature of the corresponding layer, and the minimum value of all cosine distances is taken as the anomaly value of the corresponding position in the anomaly map of the corresponding layer; the anomaly values of the corresponding positions in the M-layer anomaly maps are superimposed to obtain a spatial contrast anomaly map; and the mean value of the first maximum anomaly value in the spatial contrast anomaly map is taken as the anomaly score obtained by the spatial contrast anomaly map . 8.The unsupervised anomaly detection method based on adaptive diffusion model according to claim 1, wherein, In step 4, the implementation mode of the feature synthesis performed on the abnormal region positioning map and the normal region positioning map under the optimal denoising step number to obtain a synthesized feature map comprises: Step 421, subtract the abnormal region positioning map from the image with all pixel values being 1 The pixel value of the corresponding pixel point is obtained, and a normal region positioning map is obtained ; Abnormal region positioning map The size of the image with all pixel values being 1 Step 422, positioning a region of abnormality according to the abnormal region positioning map The abnormal region feature map is obtained by multiplying the features of the corresponding positions in the predicted clear feature map corresponding to the abnormal values at the positions in the middle with the features of the corresponding positions in the predicted clear feature map corresponding to the optimal denoising step number. Normal region positioning map The value at each position in the normal region feature map is multiplied by the corresponding position feature in the real clear feature map corresponding to the optimal denoising step number. Step 423, the values of the corresponding positions of the abnormal region feature map and the normal region feature map are added to obtain a synthesized feature map.

Images