Trustworthy surface defect detection method and equipment based on Bayesian conditional diffusion model

Abstract

A reliable surface defect detection method and device based on a Bayesian conditional diffusion model of the present invention comprises constructing a training data set and a test data set; constructing and training an industrial product surface defect detection model based on the conditional diffusion model for uncertainty perception, inputting a test image into the above model for detection, and obtaining a defect prediction image and an uncertainty prediction image of the input result respectively; arranging the original input image, the defect prediction result and the uncertainty prediction result to obtain a conditional guidance image at the second iteration; inputting the obtained conditional guidance image and the image to be detected into the conditional diffusion model to obtain a final defect prediction image and uncertainty measurement. The present invention measures the uncertainty of the results generated by the model based on the Laplace approximation, and optimizes the detection results using the uncertainty of the output results, and then iterates to improve the credibility of the final detection results.

Description

Technical Field

[0001] The present invention relates to the technical field of surface defect detection, and in particular to a reliable surface defect detection method and device based on a Bayesian conditional diffusion model. Background Art

[0002] In recent years, with the emergence of new technologies in fields such as computer vision and deep learning, vision-based industrial defect detection technology has become an effective solution for product appearance quality inspection. However, in safety-sensitive fields such as autonomous driving, face recognition, medical imaging detection, and high-end equipment defect detection, the model needs to provide robust and effective uncertainty estimates when making predictions, which is convenient for subsequent further processing.

[0003] Among them, the deep learning method based on unsupervised setting only requires normal samples that are easy to obtain for model training. It can not only solve the problem that supervised deep learning methods cannot discover unknown defects, but also has a stronger ability to express image features than traditional methods (Luo Dongliang, Cai Yuxuan, Yang Zihao, et al. A review of deep learning methods for industrial defect detection [J]. Chinese Science: Information Sciences, 2022, 52(6): 1002-1039.), so it has been widely studied. As a generative model, the diffusion model has made significant progress in the field of vision in recent years and provides a new solution for unsupervised industrial defect detection. The diffusion model can effectively model the distribution of normal samples by gradually adding noise to the image and learning the denoising process. By generating high-quality normal samples and comparing them with the input samples, it captures richer image details, which helps to improve the accuracy of defect detection.

[0004] The methods of uncertainty estimation mainly include evidence methods, Bayesian methods and integrated methods. The evidence method infers uncertainty by modeling the prior distribution of output, which is fast and saves computing resources. The integrated method estimates uncertainty through the prediction results of multiple models and increases robustness by using the diverse performance of different models on the same task. The Bayesian method introduces probability distribution to the model parameters and predicts the posterior distribution of the parameters, thereby capturing the model and data uncertainties of the output results respectively. Summary of the invention

[0005] The present invention proposes a reliable surface defect detection method and device based on a Bayesian conditional diffusion model, which can solve the problem of unreliable results of defect detection based on a generative model, thereby reducing the probability of false detection.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A reliable surface defect detection method based on a Bayesian conditional diffusion model comprises the following steps:

[0008] Step 1: construct a training data set containing normal samples and a test data set containing normal samples and defect samples.

[0009] Step 2: Construct an uncertainty-aware industrial product surface defect detection model based on the conditional diffusion model.

[0010] Step 3: Train the conditionally guided diffusion model.

[0011] Step 4: Input the test image into the above conditional diffusion model for detection, and obtain a defect prediction image and an uncertainty prediction image of the input result respectively.

[0012] Step 5: Arrange the original input image, defect prediction results and uncertainty prediction results to obtain the conditional guidance image for the second iteration.

[0013] Step 6: Input the obtained conditional guidance image and the image to be detected into the conditional diffusion model to obtain the final defect prediction image and uncertainty measurement.

[0014] Furthermore, step 2 includes the following steps:

[0015] Step 2.1, construction of feature extraction network. ResNet-34 network is selected as the feature extraction network, and the input image is passed through the second and third residual blocks of ResNet-34 to extract deep feature maps. These feature maps contain multi-level semantic information, which is conducive to the subsequent combination of pixel-level differences for accurate defect identification.

[0016] Step 2.2: The parameter distribution of the last layer of the diffusion model network is fitted using the Laplace approximation (LLLA). The last layer of the network is defined to follow a Gaussian distribution. ,in represents the maximum a posteriori estimate point, for about Hessian matrix of the maximum likelihood loss; LLLA constructs an approximate distribution only in the last layer of the network, significantly reducing computational cost and memory requirements while maintaining accuracy.

[0017] Step 2.3, construction of conditional diffusion model. In order to add guidance information in the process of generating images, the conditional diffusion model uses the guidance image for control during the denoising process. In the specific implementation, different intensities of noise are added according to the number of iterations, respectively targeting the identification of large and small defects.

[0018] Furthermore, step 3 includes the following steps:

[0019] Step 3.1, train the network and get the distribution of the last layer weight parameters:

[0020] First, use the following formula to calculate the posterior distribution of the weight parameters of the last layer: :

[0021] Formula (1)

[0022] are the parameters of the network, is the prior distribution of network parameters, Represent the input data and label data respectively, Indicates the data to be tested The results obtained through the network model are Represents the network model, Indicates the multiplication of N results; Use the loss function to train and get the maximum point , get the parameters The posterior approximate Gaussian distribution of , where H is about The Hessian matrix of

[0023] Step 3.2, method for training feature extraction network. First, input the image into the trained conditional diffusion model to obtain the reconstructed image ;

[0024] Fine-tune the feature extraction network using the following loss function: Formula (2)

[0025] It is the feature map similarity loss, which is used to minimize the feature difference between the reconstructed image and the target image to capture the similar features of the two; The distillation loss is used to maintain the generalization ability of the model and prevent the model from being overly biased towards the features of specific images during the dataset adaptation process. The feature extraction network combines the similarity loss with the distillation loss to maintain the generalization ability of the original model after domain adaptation, ensuring robustness to subtle changes and occasional interference in the background.

[0026] Furthermore, step 4 includes the following steps:

[0027] Step 4.1: Use the Monte Carlo method to adjust the network parameters. Sub-sampling, we get Set model parameters, input the data to be tested to obtain The specific calculation formulas for the mean and variance of the reconstruction results are:

[0028] Formula (3)

[0029] Step 4.2, the obtained The reconstruction results of the group are input into the trained feature extraction network to generate The feature images are grouped and the feature differences and pixel-level differences between each group of reconstructed images and the input images are calculated to obtain the defect heat map, where the pixels greater than the threshold are regarded as defect prediction images. The calculation formula of the defect heat map is:

[0030] formula

[0031] in, is the difference at the pixel level, that is, the absolute value of the direct subtraction between the reconstructed image and the original image, is a hyperparameter that controls the ratio between feature map differences and pixel differences. is the difference at the feature map level, and the calculation formula is as follows:

[0032] formula

[0033] in, represents the j-th layer output of the feature extraction network, Indicates a specific layer.

[0034] Step 4.3, The mean and variance of each group of defect prediction images are calculated. Assume that each group of defect prediction images is (in ), for each pixel ,calculate The mean of the pixels at the same position in the group image is taken as the defect prediction result, and the variance is taken as the uncertainty prediction result. The mathematical expression is:

[0035]

[0036] Formula (6)

[0037] is the average defect prediction area obtained in the first iteration; Forecasting uncertainty.

[0038] Furthermore, step 5 includes the following steps:

[0039] A defect mask is roughly generated using the defect prediction image obtained in the first iteration, which preliminarily identifies the potential defect areas. In order to cover the defects more accurately, especially in areas with high uncertainty, the uncertainty prediction results will be used to further guide the inspection.

[0040] The original image, defect prediction image and uncertainty prediction results obtained in step 4 are sorted to obtain the guide image for the second iteration. The second guide image uses the updated , the formula is as follows:

[0041] Formula (7)

[0042]

[0043] ⊙ represents element-by-element multiplication (Hadamard product), which is used to implement weighted fusion operation, and y is the first guided image, that is, the input image itself. is the updated mask image, as shown in formula (8),

[0044] formula .

[0045] Furthermore, step 6 includes the following steps:

[0046] Step 6.1: Input the guide image and the image to be detected into the conditional diffusion model, and convert the second iteration guide image generated in step 5 into And the original image to be detected Input into the conditional diffusion model to obtain the final defect prediction image and uncertainty measure. In the second sampling process of the conditional diffusion model, the initial noise intensity is set to be lower than the first sampling to achieve more refined image reconstruction and detail recovery, so as to ensure that the disturbance of the model in the normal area is reduced while refining the defect features. In the denoising process of the conditional diffusion model, the reconstruction effect of the defect area is further optimized by the following noise term prediction formula:

[0047] Formula (9)

[0048] in, is the noise prediction of the network model, c is the adjustment weight, is the diffusion parameter, For the second guided image, according to the noise addition formula and Add the same noise. This formula makes the generated image gradually approach the original image. The details are brought closer, and the defect features are further enhanced in the defect area, and the parts that were not fully restored in the first iteration are refined and supplemented.

[0049] Step 6.2, repeat the process of step 4, and generate the final defect prediction image and uncertainty measurement result based on the second reconstruction image of the conditional diffusion model. This result can accurately identify the defect area with high resolution and provide reliability evaluation of defect detection through uncertainty measurement.

[0050] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0051] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0052] It can be seen from the above technical scheme that the present invention relates to the uncertainty of deep neural networks and the field of industrial product surface defect detection, and proposes a Bayesian industrial product surface defect detection network based on Laplace approximation uncertainty perception. The method includes: constructing a training data set containing normal samples and a test data set containing normal samples and defect samples; constructing an industrial product surface defect detection model based on the uncertainty perception of the conditional diffusion model; training a diffusion model guided by conditions; inputting the test image into the above conditional diffusion model for detection, and obtaining a defect prediction image and an uncertainty prediction image of the input result respectively; arranging the original input image, the defect prediction result and the uncertainty prediction result to obtain the conditional guided image at the second iteration; inputting the obtained conditional guided image and the image to be detected into the conditional diffusion model to obtain the final defect prediction image and uncertainty measurement. Based on Laplace approximation, this method measures the uncertainty of the results generated by the model, and optimizes the detection results using the uncertainty of the output results, and then iterates to improve the credibility of the final detection results.

[0053] The uncertainty-aware industrial product surface defect detection method based on the conditional diffusion model proposed in the present invention has the following advantages:

[0054] (1) This method introduces uncertainty analysis to enable the detection model to perform uncertainty assessment on the prediction of defect areas. By calculating the variance of the defect results, a measure of the confidence of the detection results is provided, which helps to improve the reliability and robustness of the detection results and is suitable for high-demand industrial environments.

[0055] (2) The step-by-step noise strategy takes into account both semantics and pixel quality. In the reconstruction process, the first iteration uses large-step noise to perturb obvious abnormal areas and repair most defect areas; the second iteration uses small-step noise to preserve image details. This combined strategy effectively balances the semantic quality and pixel quality of the image, thereby improving the accuracy of defect detection.

[0056] (3) The defect recognition accuracy is improved by combining the comparison of feature level and image pixel level. The feature maps of the reconstructed image and the original image are obtained through the pre-trained ResNet-34 feature extraction network. The final difference value of the two images is obtained by combining the image difference at the pixel level.

[0057] (4) Improve reconstruction quality through condition-guided iterative optimization. In the second iteration, this method uses the defect prediction results and uncertainty prediction results to generate a mask image, which can guide reconstruction more accurately. The iterative optimization strategy can improve the accuracy of reconstruction and make the final defect detection results more reliable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of the present invention.

[0059] Figure 2 A flowchart of a single iteration proposed by an embodiment of the present invention.

[0060] Figure 3 This is an uncertainty measurement diagram for the correct prediction result according to an embodiment of the present invention.

[0061] Figure 4 This is an uncertainty measurement diagram for an erroneous prediction result according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0063] As attached Figure 1 As shown, the specific implementation steps of the uncertainty-aware industrial product surface defect detection method based on the conditional diffusion model proposed in the embodiment of the present invention are as follows:

[0064] Step 1: Construct a training data set containing normal samples and a test data set containing normal and defective samples.

[0065] Step 2: Construct an uncertainty-aware industrial product surface defect detection model based on the conditional diffusion model.

[0066] During the training phase, the training of the feature extraction network was added on the basis of the original implicit diffusion model. The latter uses the ResNet-34 network, which aims to compare the similarity of two images from the feature level. At the same time, the parameters of the last layer of the diffusion model network are defined to obey the probability distribution, and the parameter distribution of the last layer is calculated using the Laplace approximation method.

[0067] Step 3: training the uncertainty-aware industrial product surface defect detection model based on the conditional diffusion model. The specific steps are as follows:

[0068] Step 3.1, first, the input images are resized to ensure the consistency of the input data. Specifically, all images will be adjusted to a predetermined standard size, scaled or cropped by appropriate interpolation methods to ensure that each input image has the same resolution. Subsequently, Gaussian noise is added to each image. After adding noise, the image is sent to the network, and the goal of the network is to predict the distribution of Gaussian noise in the image. The mean square error (MSE) between the predicted result and the original noise distribution is used as the loss function to quantify the accuracy of the network in noise modeling. In order to perform the maximum a posteriori estimation of the network parameters, a regularization term is added to the loss function, which is used to prevent overfitting and facilitate the subsequent Laplace approximation of the network parameter distribution.

[0069] Step 3.2, after completing the above steps and obtaining the loss function value, all parameters in the network except the last layer will be frozen, and only the parameters of the last layer will be updated. The Gaussian distribution approximation of the parameters of the last layer can be obtained through the Laplace approximation method. This process derives the mean and variance of the parameter distribution of the last layer through the maximum a posteriori estimation (MAP), providing the necessary statistical information for subsequent inference.

[0070] Step 3.3, after obtaining the reconstruction result of the network, the reconstruction result and the original input image are sent to the pre-trained feature extraction network for further processing. Here, the original ResNet-34 model is used as the feature extractor, and fine-tuned on the basis of this model. By comparing the reconstruction result with the feature map of the original image, the cosine similarity between the two is calculated to measure their feature similarity. The optimization goal is to minimize the loss between the feature maps of the two without weakening the generalization ability of the model, so that the reconstructed image is as close to the original image as possible in the feature space.

[0071] Step 4: Input the test image into the above conditional diffusion model for detection, and obtain a defect prediction image and an uncertainty prediction image of the input result respectively.

[0072] Step 4.1, sample and generate a reconstructed image from the noise prediction network. When generating the reconstructed image for the first iteration, add a larger noise. For more obvious defects, the large step noise can effectively perturb the abnormal area, so that the model pays more attention to the overall semantic features of the image during reconstruction. Since the parameters of the last layer of the network obey a specific distribution, in the actual prediction, the specific parameter values ​​are sampled from the parameter distribution of the last layer of the model network. Since a single sampling result cannot fully and accurately reflect the characteristics of the distribution, the Monte Carlo sampling method is used to repeat the sampling N times to obtain N groups of different reconstructed images.

[0073] Step 4.2, send the above N groups of reconstructed images to the trained feature extraction network to extract their corresponding feature maps and calculate the difference between the feature maps of the two. At the same time, calculate the difference between the reconstructed image and the input image, subtract the two and take the absolute value to obtain the difference in the pixel map. Calculate the pixel map difference and the corresponding feature map difference to obtain the final defect heat map. The high-value area in the heat map indicates the possible defect location. By setting a specific threshold, the points in the heat map that are greater than the threshold are marked as defect areas, thereby obtaining a set of defect prediction images. This processing helps to accurately identify defects in images and further improve the accuracy of defect detection.

[0074] Step 4.3, statistically process the N groups of results obtained by N samplings, and calculate the mean of the reconstructed image, the defect prediction image, and the variance of the defect result. Specifically, first calculate the mean of the N groups of reconstructed images and the mean of the defect prediction image as the final reconstruction result and defect prediction result. Then, calculate the variance of the defect result as the result of uncertainty prediction.

[0075] Step 5: The original input image, defect prediction results and uncertainty prediction results are sorted to obtain the conditional guidance image for the second iteration. Specifically, the defect prediction image generated in the previous step is added to the uncertainty prediction result to generate a mask image. Subsequently, a new guidance image is generated based on the mask image and the original input image.

[0076] Step 6, input the obtained conditional guidance image and the image to be detected into the conditional diffusion model to obtain the final defect prediction image and uncertainty measurement. Repeat the process of step 4 to obtain the final defect prediction image and uncertainty measurement result. In step 4.1, small step noise is added in the second iteration. Small step noise helps to maintain the details of the image and improve the pixel quality of the reconstructed image. When comparing images, the difference between pixels in the normal area will be reduced, reducing the recognition error.

[0077] The following are specific instructions:

[0078] Step 1: Construct a training data set containing normal samples. A training data set contains normal sample images in the same scene. Multiple data sets are constructed for samples in different scenes. In the construction of the test data set, normal samples and defect samples are included. , Representative For training data, they are defect images , and the truth mask corresponding to the defect Here, the true value mask image is a single-channel image containing only 0 and 1 values. The 0 value represents that the pixel area is a normal area, and the 1 value represents that the pixel area is a defective area.

[0079] Step 2: Construct an uncertainty-aware industrial product surface defect detection model based on the conditional diffusion model.

[0080] Step 2.1: Construction of feature extraction network. Figure 2 As shown, a feature extraction network is constructed to measure the reconstruction samples With input sample In order to meet the detection accuracy requirements, the ResNet-34 network is selected as the feature extraction network. The second and third residual blocks of the ResNet-34 network are used to generate feature maps. and , represents the feature extraction network, where Indicates the number of the network residual layer. This feature map represents the embedding of multiple layers of semantic information. By comparing the feature maps of the generated image and the input image, it ensures that the reconstructed image is semantically consistent with the input image.

[0081] Step 2.2, the last layer of the network is fitted using Laplace approximation. In uncertainty estimation, the weight parameters of the last layer of the network are defined to follow a Gaussian distribution. in represents the maximum a posteriori estimate point of the parameter, for about Hessian matrix of the maximum likelihood loss. LLLA significantly reduces computational cost and memory requirements by constructing a Laplace approximation only for the last layer of the network. The posterior distribution of this approximation is:

[0082] formula

[0083] in, By maximizing the posterior probability we get:

[0084] Formula (11)

[0085] in, is the loss function obtained by maximum likelihood estimation of the model, The Laplace approximation method of the last layer can efficiently implement uncertainty estimation in larger models.

[0086] Step 2.3, construction of the conditional diffusion model. In order to control the quality of the generated image, the conditional diffusion model uses the guidance information to adjust the generated image at each sampling and denoising step. In the tth step of sampling, an additional guidance image is introduced , The input image is denoised according to the formula (14) and The image obtained by adding the same noise. The denoising formula of conditional diffusion is:

[0087] Formula (12)

[0088] in represents the prediction noise term, is the intensity of the added noise, is a hyperparameter. The z in the formula represents the noise of Gaussian normal distribution. Under the condition of , the update formula of the noise term is:

[0089] Formula (13)

[0090] in is the weight coefficient, which is used to control the influence of the guide image in the denoising process. Through this conditional guidance strategy, the generated image gradually approaches the characteristics of the original input image, improving the accuracy and quality of reconstruction.

[0091] Step 3: train the above-mentioned uncertainty-aware industrial product surface defect detection model based on the conditional diffusion model.

[0092] Step 3.1: Train the implicit diffusion model. First, scale the images in the training set to uniform size. Then, add Gaussian noise to the input images, using the diffusion model’s noise addition formula:

[0093] Formula (14)

[0094] , is a hyperparameter. The resulting image with Gaussian noise Send it to the network and use the Unet network output to get the predicted noise , optimize the mean square loss function :

[0095] Formula (15)

[0096] is the Unet network parameter, That is, the label data corresponding to the input sample during training. The Adam optimizer is used to train the trusted target detection model. Taking the Adam optimizer in the pytorch framework as an example, the weight_decay parameter is set to 0.05 or according to the optimization situation.

[0097] Step 3.2, train the network and obtain the distribution of the weight parameters of the last layer, and calculate the posterior distribution of the weight parameters of the last layer according to the following formula:

[0098] Formula (1)

[0099] are the parameters of the network, is the prior distribution of network parameters, Represent the input data and label data respectively, Indicates the data to be tested The results obtained through the network model are Represents a network model, Indicates the multiplication of N results.

[0100] Step 3.3, train the feature extraction network. Figure 2 As shown, the image is sent to the trained conditional diffusion model to obtain the reconstructed result. With the original input image Send it to the feature extraction network, fine-tune the original ResNet-34 network, and use the following loss function to train the network: The loss function for domain adaptation is:

[0101] Formula (16)

[0102] refers to the output of the feature extraction network, where is the layer number of the network. Indicates that some specific layers are selected. In this embodiment, the second and third layers of the network are used to extract features. This ensures that the normal image and the reconstructed image are as similar as possible in the semantic feature space. The similarity loss optimizes the feature extractor by minimizing the difference between the features of the reconstructed image and the input image. . Indicates calculating the cosine similarity of A and B.

[0103] Distillation loss Used to maintain the generalization ability of the model. In the process of domain adaptation, that is, in the process of network training using this loss function, the original feature extractor Some generalization features of may be lost. This distillation loss not only considers the similarity between the reconstructed image and the input image, but also the similarity between the reconstructed image and itself, and the similarity between the input image and itself, thereby preventing the model from being overly biased towards the features of a specific image.

[0104] By combining similarity loss and distillation loss, the feature extractor It is able to preserve the generalization ability of the original model during the reconstruction process, thereby remaining robust to subtle changes in the object or occasional interference in the background.

[0105] Step 4: Input the test image into the above conditional diffusion model for detection, and obtain a defect prediction image and an uncertainty prediction image of the input result respectively.

[0106] Step 4.1, sample and generate the reconstructed image from the noise prediction network. Figure 2 As shown. According to the Laplace approximation method, the predicted distribution needs to be integrated and there is no analytical solution. Using the Monte Carlo method to approximate, the network parameters are sampled N times through the obtained distribution to obtain N models. For any test data Obtain N reconstruction results respectively, and calculate the mean and variance of the N reconstruction results. Represents a certain network model. In this case, it means to reconstruct the network model. The calculation formulas for the mean and variance are:

[0107] Formula (3)

[0108] In the present invention, after the last layer of network parameter distribution is obtained through step 3.2, each time sampling is performed, a set of parameter results will be randomly sampled from the distribution as the result of this time, and N sets of results will be obtained by taking N times. In step 4.1, the initial noise added is relatively strong. This step is to remove the significant large area of ​​noise in the image, because the strong noise blurs the texture information of the details, and the reconstructed image can repair most of the defects. If the training noise step value range is , the noise added in the first iteration should be controlled as much as possible within the time step T Within this range.

[0109] Step 4.2: Send the N groups of results obtained above into the trained feature extraction network. Figure 2As shown, N groups of feature images will be obtained, and the feature differences and pixel-level differences between the N groups of reconstructed images and the input images will be calculated respectively. The thermal map of the defect is obtained by calculating the two. The specific calculation formula is:

[0110] formula

[0111] Represents the difference between pixels, that is, the reconstructed image is directly subtracted from the original image to obtain the absolute value. is a hyperparameter that controls the ratio of feature map differences to pixel differences. The max function takes the maximum pixel value on the entire image. Represents the difference at the feature map level. The specific calculation formula is as follows:

[0112] formula

[0113] refers to the output of the feature extraction network, where is the layer number of the network.

[0114] Step 4.3, process the N groups of results to obtain N groups of reconstructed images and N groups of defect prediction images. Each group of defect prediction images is assumed to be ,in =1, 2…N. For each pixel in the defect prediction map, calculate The mean and variance of N groups of images, the mean value is taken as the result of this defect prediction, and the variance is taken as the prediction result of uncertainty. The mathematical expression is:

[0115]

[0116] formula

[0117] is the average defect prediction area obtained in the first iteration; Forecasting uncertainty.

[0118] Step 5: Arrange the original image, defect prediction image and uncertainty prediction result to obtain the guide image for the second iteration. , the formula is as follows:

[0119] formula

[0120]

[0121] ⊙ represents element-by-element multiplication, which is used to implement weighted fusion operations. is the guidance image for the first iteration, is the updated mask image, as shown in formula (8);

[0122] formula ;

[0123] Defect prediction image obtained in the first iteration The defect locations are roughly marked, but not all defects can be covered. Some high uncertainty areas may also contain defects. Specifically, when the uncertainty of a certain area is high, it is considered that there is a probability of defects in this area, so noise is also added to this part, thereby reducing the dependence on the guide image in the second reconstruction.

[0124] This gives us the guidance image for the second step of image sampling. Based on the results of the first iteration, the guidance image is optimized to form a fine-grained guidance for the reconstructed image. This process can more accurately calibrate the defect position, especially in complex or unclear areas, which can improve the accuracy and robustness of defect detection.

[0125] Step 6: The obtained guide image and the image to be detected are brought into the conditional diffusion model to obtain the final defect prediction image and uncertainty measurement.

[0126] Step 6.1, in the second sampling process of the conditional diffusion model, the original image is input at the same time and the second generated guidance image , and a new reconstructed image is obtained. At this time, the intensity of the initial noise added will be lower than that of the first iteration in order to refine the reconstructed image and improve the ability to restore details. The noise added in the second iteration controls the time step number T in In the second iteration, due to the guidance image It contains the defect information after the first prediction and weighted adjustment, hoping to further correct the image in the denoising process through the conditional diffusion model, especially in the details of the defect area. Specifically, the prediction formula of the noise term is:

[0127] formula

[0128] in, is the preliminary noise prediction, c is the adjustment weight, is the diffusion parameter, For the second guided image, according to the noise addition formula and Add the same noise. In this way, the generated image will not only gradually approach the input image It not only restores the details of the original image, but also fully repairs the defect features in the original image, especially those that may not have been fully restored in the first iteration. This can effectively reduce the blurring of the defect area and improve the model's ability to detect defects in detail.

[0129] Step 6.2, after the second denoising process of the conditional diffusion model, a new reconstructed image will be obtained. This image has higher quality than the first reconstructed image, especially the performance at the detail level is clearer, the texture of the defect is sharper, and the difference with the original image in the normal area is significantly reduced. Repeat the process of step 4 to obtain the final defect prediction image and uncertainty measurement result.

[0130] The technical solution of the present invention provides an efficient method for detecting surface defects of industrial parts by combining a conditional diffusion model with uncertainty estimation. By guiding the iterative update of information and noise adjustment, the model can provide uncertainty assessment of defect detection results while maintaining high detection accuracy, thereby improving the robustness and accuracy of the detection system in complex environments. This method can not only be applied to surface defect detection, but can also be extended to image processing and analysis tasks in other fields.

[0131] Figure 3 and Figure 4 The following figure shows some of the experiments of the model on the MVTec dataset. Figure 3 The output of the model when the defect is correctly detected is shown. The model accurately locates the defect area by reconstructing the image, and the generated prediction result is highly consistent with the true label. The result of the uncertainty measurement is used as an evaluation index to further judge the credibility of the model judgment. Figure 3 From the prediction results and uncertainty graph, we can see that the overall uncertainty value of the defect prediction results is low and the overlapping areas are small, indicating that the model is more confident in predicting most defect areas. This reflects the stability and accuracy of the model in these areas.

[0132] Attached Figure 4 The output of the model in the case of defect error detection is shown. At this time, when the model predicts the defect error, the prediction result generated is inconsistent with the true label. Figure 4 From the prediction results and uncertainty graph, we can see that there are many overlapping areas between the two, indicating that the model has a high uncertainty in the predicted defect area. This means that the prediction result is not credible. More accurate testing or manual review by experts is required.

[0133] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0134] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0135] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the trusted surface defect detection methods based on the Bayesian conditional diffusion model in the above embodiments.

[0136] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above methods.

[0137] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0138] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0139] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0140] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A reliable surface defect detection method based on Bayesian conditional diffusion model, characterized by: The following steps are involved: Step 1: construct a training data set containing normal samples and a test data set containing normal samples and defect samples; Step 2, construct an industrial product surface defect detection model based on uncertainty perception of the conditional diffusion model; Step 3, training the industrial product surface defect detection model constructed in step 2; Step 4: input the test image into the above-mentioned industrial product surface defect detection model for detection, and obtain a defect prediction image and an uncertainty prediction image of the input result respectively; Step 5, sorting the original input image, defect prediction results and uncertainty prediction results to obtain the conditional guidance image for the second iteration; Step 6, inputting the obtained conditional guidance image and the image to be detected into the industrial product surface defect detection model to obtain the final defect prediction image and uncertainty measurement; Step 2 includes: Construction of feature extraction network; ResNet-34 network is selected as the feature extraction network, and the input image is passed through the second and third residual blocks of the ResNet-34 network to extract deep feature maps; The parameter distribution of the last layer of the diffusion model network is fitted using Laplace approximation; the last layer of the network is defined to obey Gaussian distribution ,in represents the maximum a posteriori estimate point of the network parameters, for About the Hessian matrix of maximum likelihood loss; Construction of the conditional diffusion model; the conditional diffusion model uses the guide image to control the denoising process, adding noise of different intensities according to the number of iterations, respectively targeting the identification of large and small defects; Step 3 includes: Train the diffusion model network and obtain the distribution of the weight parameters of the last layer. Use the following formula to calculate the posterior distribution of the weight parameters of the last layer: : Formula(1) are the parameters of the network, is the prior distribution of network parameters, Represent the input data and label data respectively, Indicates the data to be tested The results obtained through the network model are Represents the network model, Indicates the multiplication of N results; As the loss function, the maximum a posteriori estimate of the network parameters is obtained by training , get the parameters The posterior approximate Gaussian distribution of , where H is about The Hessian matrix of Method for training feature extraction network; input the image into the trained conditional diffusion model to obtain the reconstructed image , is the size of the image; Fine-tune the feature extraction network using the following loss function: Formula (2) It is the feature map similarity loss, which is used to minimize the feature difference between the reconstructed image and the target image to capture the similar features of the two; is a hyperparameter, is the distillation loss, which controls the proportion of distillation loss in the total loss; are the input image and the reconstructed image respectively.

2. The reliable surface defect detection method based on the Bayesian conditional diffusion model according to claim 1 is characterized in that: Step 4 includes the following steps: Step 4.1: Use the Monte Carlo method to adjust the network parameters. Sub-sampling, we get Set model parameters, input the data to be tested to obtain The reconstruction results are calculated to obtain the mean and variance of the reconstruction results. The specific calculation formula is: Formula(3) represents any test data, and It is used to represent the mean and variance. Represents a certain network model, which here means to reconstruct the network model; Step 4.2, the obtained The reconstruction results of the group are input into the trained feature extraction network to generate The feature images are grouped and the feature differences and pixel-level differences between each group of reconstructed images and the input images are calculated to obtain the defect heat map, where the pixels greater than the threshold are regarded as defect prediction images. The calculation formula of the defect heat map is: official in, is the difference at the pixel level, that is, the absolute value of the direct subtraction between the reconstructed image and the original image, is a hyperparameter that controls the ratio between feature map differences and pixel differences. is the difference at the feature map level, The function gets the maximum value of the pixel in a certain image. The calculation formula is as follows: official in, represents the feature extraction network Layer output, represents a specific layer of the network model, Representation calculation The cosine similarity of Step 4.3, The mean and variance of each group of defect prediction images are calculated. ,in , for each pixel , w, h represent the width and height of the image, calculate The mean of the pixels at the same position in the group image is taken as the defect prediction result, and the variance is taken as the uncertainty prediction result. The mathematical expression is: official ; is the average defect prediction area obtained in the first iteration; Forecasting uncertainty.

3. The reliable surface defect detection method based on the Bayesian conditional diffusion model according to claim 2 is characterized in that: Step 5 includes the following steps: A defect mask is generated using the defect prediction image obtained in the first iteration to preliminarily identify the potential defect areas; The original image, defect prediction image and uncertainty prediction result obtained in step 4 are sorted to obtain the guide image for the second iteration; the second guide image uses the updated , the formula is as follows: Formula(7) ⊙ represents element-by-element multiplication, which is used to implement weighted fusion operations. is the guidance image for the first iteration, is the updated mask image, as shown in formula (8); official .

4. The reliable surface defect detection method based on Bayesian conditional diffusion model according to claim 3 is characterized in that: Step 6 includes the following steps: Step 6.1: Input the guide image and the image to be detected into the conditional diffusion model, and convert the second iteration guide image generated in step 5 into And the original image to be detected Input into the conditional diffusion model to obtain the final defect prediction image and uncertainty measure; In the sampling process of the second iteration of the conditional diffusion model, the initial noise intensity is set to be lower than that of the first sampling. In the denoising process of the conditional diffusion model, the reconstruction effect of the defect area is optimized by the following noise term prediction formula: Formulas (9) in, is the preliminary noise prediction, c is the adjustment weight, is the diffusion parameter, for Add t-step noise to get; this formula (9) makes the generated image close to the original image The details are brought closer, while the defect features are enhanced in the defect area, and the parts that were not fully restored in the first iteration are supplemented; Step 6.2, repeat the process of step 4, and generate the final defect prediction image and uncertainty measurement result based on the second reconstructed image of the conditional diffusion model.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

Images