Product appearance defect detection method and device, computer equipment and storage medium
By using the denoising diffusion model to generate the target image and comparing the difference with the image to be detected, the problem of low defect detection accuracy in the prior art is solved, and higher detection accuracy is achieved.
Patent Information
- Application Number
- CN202411781555.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-03-28
AI Technical Summary
Existing product appearance defect detection methods based on supervised learning lead to low accuracy due to the need for a large number of abnormal samples, especially when the number of abnormal samples is small.
Using the denoising diffusion model, a pre-trained image generation model is used to generate the target image using normal samples, and the product is determined whether there are defects by comparing the difference with the image to be detected.
Improve the accuracy of product appearance defect detection and avoid the reduction in model accuracy due to the small number of abnormal samples.
Smart Images
Figure CN119722608A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of defect detection technology, and in particular to a method, device, computer equipment and storage medium for detecting product appearance defects. Background Art
[0002] In the industrial manufacturing process, product quality assurance is the key to improving production efficiency and reducing costs. At present, industrial product quality detection methods mainly rely on traditional machine vision technology (such as detection methods based on supervised learning). However, this method has certain shortcomings: the detection method based on supervised learning requires a large number of abnormal samples to be labeled. However, in actual production, the number of normal samples of industrial products is far greater than that of abnormal samples, resulting in a small number of abnormal samples. This will lead to a low accuracy of the detection method based on supervised learning. Summary of the invention
[0003] Based on this, it is necessary to provide a product appearance defect detection method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems, which can improve the accuracy of product appearance defect detection.
[0004] In a first aspect, the present application provides a method for detecting product appearance defects, comprising:
[0005] Acquire an image to be detected of a product to be detected, and generate a mask image corresponding to the image to be detected, a text content used to describe the image to be detected, and a noise image corresponding to the image to be detected;
[0006] Encode the text content into the first type of image generation conditions, and encode the image to be detected and the mask image into the second type of image generation conditions;
[0007] The noise image is processed to generate a target image according to the first type of image generation condition and the second type of image generation condition by using a pre-trained denoising diffusion model; wherein the denoising diffusion model is an image generation model obtained by training based on normal samples;
[0008] Extract the original image of the covered area from the image to be detected according to the mask image, and compare the difference between the original image of the covered area and the target image;
[0009] Based on the difference comparison results, determine whether the product to be inspected has defects.
[0010] In a second aspect, the present application provides a product appearance defect detection device, comprising:
[0011] A data processing module, used to obtain an image to be detected of a product to be detected, and generate a mask image corresponding to the image to be detected, a text content used to describe the image to be detected, and a noise image corresponding to the image to be detected;
[0012] An encoding processing module, used for encoding text content into a first type of image generation condition, and encoding the image to be detected and the mask image into a second type of image generation condition;
[0013] An image generation module, used to process the noisy image to generate a target image according to the first type of image generation condition and the second type of image generation condition by using a pre-trained denoising diffusion model; wherein the denoising diffusion model is an image generation model obtained by training based on normal samples;
[0014] A difference comparison module is used to extract the original image of the covered area from the image to be detected according to the mask image, and compare the difference between the original image of the covered area and the target image;
[0015] The defect determination module is used to determine whether the product to be inspected has defects according to the difference comparison result.
[0016] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.
[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.
[0018] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.
[0019] The above-mentioned product appearance defect detection method, device, computer equipment, computer-readable storage medium and computer program product generate a normal target image by using a denoising diffusion model trained with normal samples in combination with determined image generation conditions; and then determine whether there are defects in the product to be detected by comparing the difference between the target image and the image to be detected. Since the denoising diffusion model used in the product appearance defect detection method of the present application is completely based on normal sample training and does not rely on abnormal samples, the model accuracy will not be reduced due to the small number of abnormal samples. In this way, when completing product appearance defect detection based on the denoising diffusion model, the accuracy of product appearance defect detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 An application environment diagram of a product appearance defect detection method provided in an embodiment of the present application;
[0021] Figure 2 A schematic diagram of a process for detecting product appearance defects provided in an embodiment of the present application;
[0022] Figure 3 A structural block diagram of a product appearance defect detection device provided in an embodiment of the present application;
[0023] Figure 4 An internal structure diagram of a computer device provided in an embodiment of the present application;
[0024] Figure 5 An internal structure diagram of another computer device provided in an embodiment of the present application;
[0025] Figure 6 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0027] The product appearance defect detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, image acquisition devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.
[0028] like Figure 2 As shown, the embodiment of the present application provides a method for detecting product appearance defects, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method comprises the following steps:
[0029] S201, obtaining an image to be detected of a product to be detected, and generating a mask image corresponding to the image to be detected, a text content for describing the image to be detected, and a noise image corresponding to the image to be detected.
[0030] In the embodiments of the present application, the product to be detected may be an industrial product involved in industrial production. Exemplarily, the product to be detected may be an industrial product such as a mechanical part, an electronic product, an automotive part, etc. The image to be detected is an appearance image of the industrial product collected by an image acquisition device.
[0031] In the embodiments of the present application, after obtaining the image to be detected, a mask image corresponding to the image to be detected may be generated by means of a sliding window, or a mask image corresponding to the image to be detected may be generated by means of region division. Specifically, generating a mask image corresponding to the image to be detected by means of a sliding window includes: first determining the size of the sliding window; then, sliding on the image to be detected according to the determined size of the sliding window, generating a random number m at each sliding window position, and setting a threshold n. If m < n, all pixels within the sliding window are set to the mask value; if m ≥ n, the pixels within the sliding window remain the original values. After the sliding window traverses the entire image to be detected, a mask image can be generated. Generating a mask image corresponding to the image to be detected by means of region division includes: first, dividing the image to be detected into several different regions; for each divided region, generating a random number s respectively, and setting a probability value p. For any region, if s < p, all pixels in the region are set to the mask value; if s ≥ p, the pixels in the region remain the original values. In this way, the final mask image can be obtained.
[0032] In the embodiments of the present application, after obtaining the image to be detected, the process of generating the text content for describing the image to be detected includes: extracting image features from the image to be detected; performing semantic understanding on the extracted image features; generating the text content for describing the image to be detected according to the understood semantics. In addition, the image to be detected can also be directly input into an image description generation model (a pre-trained image-to-text model), and according to the output of the image description generation model, the text content for describing the image to be detected can be obtained.
[0033] In the embodiments of the present application, after obtaining the image to be detected, the process of generating a noise image corresponding to the image to be detected includes: first, determining the size information of the image to be detected, where the size information is used to accurately add noise at each pixel position, and the size information may include information such as the width, height, and pixel depth of the image to be detected; according to the size information, sequentially traversing each pixel in the image to be detected, and for each pixel traversed, generating a corresponding noise value according to a pre-determined noise type (such as Gaussian noise) and adding it to the original value of the pixel; finally, obtaining a noise image corresponding to the image to be detected.
[0034] S202. Encode the text content into a first type of image generation condition, and encode the image to be detected and the mask image into a second type of image generation condition.
[0035] In an embodiment of the present application, the text content obtained in step S201 can be encoded into a first type of image generation condition through a text encoder (such as Clip); wherein the first type of image generation condition can be expressed in the form of a feature vector, and its function is to guide the subsequent denoising diffusion model to generate the required image according to the semantics in the first type of image generation condition.
[0036] In an embodiment of the present application, the image to be detected and the mask image can be encoded into a second type of image generation condition through an image encoder; wherein the second type of image generation condition can be expressed in the form of a feature vector, and its function is to guide the subsequent denoising diffusion model to reconstruct the image of the masked area.
[0037] S203 , using a pre-trained denoising diffusion model, according to the first type of image generation condition and the second type of image generation condition, the noise image is processed to generate a target image.
[0038] In an embodiment of the present application, the denoising diffusion model is an image generation model obtained by training based on normal samples; wherein the normal samples are composed of images without defects; it is understandable that the abnormal samples corresponding to the normal samples may be composed of images with defects. The structure of the denoising diffusion model may include a feature encoding module and a feature decoding module. Based on this, the specific process of generating a target image may include: inputting the noise image and the first type of image generation conditions into the feature encoding module of the denoising diffusion model, so that the feature encoding module extracts features from the noise image and the first type of image generation conditions and integrates the features, and then inputs the integrated features into the feature decoding module; further, through the feature decoding module, the image of the masked area is reconstructed according to the integrated features and the second type of image generation conditions to obtain the target image.
[0039] It is understandable that, since the denoising diffusion model is an image generation model obtained by training based on normal samples, the target image constructed in this application is an image without defects in theory. Therefore, the newly constructed target image can be used as a reference for judging whether there are defects in the image to be detected.
[0040] S204, extracting the original image of the covered area from the image to be detected according to the mask image, and performing a difference comparison between the original image of the covered area and the target image.
[0041] In an embodiment of the present application, the original image of the covered area is extracted from the image to be detected based on the mask image, that is, the part of the image covered by the mask in the image to be detected is extracted. The specific process may include: traversing each pixel in the mask image in turn, and each time a pixel is traversed, determining whether the pixel value of the pixel is the mask value; if so, extracting the target pixel corresponding to the pixel from the image to be detected; and determining the original image of the covered area based on the target pixel finally extracted.
[0042] In the embodiment of the present application, the difference comparison between the original image of the covered area and the target image includes: determining the difference between the corresponding pixel points of the original image of the covered area and the target image, and using the difference as a difference measurement index; or calculating the mean square error or mean absolute error according to the difference between the corresponding pixel points of the original image of the covered area and the target image, and using the calculated mean square error or mean absolute error as a difference measurement index. In this way, the obtained difference measurement index can be used as a difference comparison result.
[0043] S205: Determine whether the product to be inspected has defects based on the difference comparison result.
[0044] In an embodiment of the present application, different indicator thresholds are pre-set for different difference measurement indicators; then, the difference measurement indicator obtained in step S204 is compared with the corresponding indicator threshold. If it is less than the indicator threshold, it is determined that there is no defect in the original image of the covered area. If it is greater than or equal to the indicator threshold, it is determined that there is a defect in the original image of the covered area, that is, there is a defect in the appearance of the product to be inspected.
[0045] In the embodiment of the present application, the denoising diffusion model is completely trained based on normal samples and does not rely on abnormal samples, so that the model accuracy will not be reduced due to the small number of abnormal samples. In this way, when the present application completes product appearance defect detection based on the denoising diffusion model, the accuracy of product appearance defect detection can be improved.
[0046] In some embodiments, the structure of the denoising diffusion model includes a feature encoding module and a feature decoding module;
[0047] Through the pre-trained denoising diffusion model, the noisy image is processed according to the first type of image generation condition and the second type of image generation condition to generate the target image, including:
[0048] Through the feature encoding module, features are extracted from the input noise image and the first type of image generation conditions and feature integration is performed, and the integrated features are input into the feature decoding module;
[0049] Through the feature decoding module, the target image is constructed based on the integrated features and the second type of image generation conditions.
[0050] Specifically, the denoising diffusion model may be a Mamba diffusion model or a diffusion model of other architectures, which is not specifically limited here. When generating a target image, the noise image and the first type of image generation condition may be provided as input to the feature encoding module of the denoising diffusion model. The feature encoding module processes the input information, for example, analyzes the noise image and extracts its various features, such as pixel value distribution, color features, texture features, shape features, etc. For the first type of image generation condition, the semantic information contained therein is parsed, such as the type, color, position relationship of the object mentioned in the text content, and the extracted semantic information is converted into a feature vector that matches the image feature. Furthermore, the feature encoding module also integrates the feature vectors extracted from the noise image and the first type of image generation condition, so as to fuse information from different sources together, so that the model can comprehensively consider various factors to generate the target image. Finally, the integrated features are input into the feature decoding module. The feature decoding module takes the feature vector from the feature encoding module and the second type of image generation condition as input at the same time, and then performs denoising and image generation according to the input to obtain the target image. It can be understood that when the image is generated, since the second type of image generation conditions include information about the area indicated by the mask, the feature decoding module can repair and fill the area according to the information of the surrounding area to obtain the final target image.
[0051] In this embodiment, since the denoising diffusion model is an image generation model obtained through training based on normal samples and does not rely on abnormal samples, the model accuracy will not be reduced due to the small number of abnormal samples. Moreover, the target image generated by denoising diffusion is an image in which there are no abnormalities in theory, which can be used as a reference for judging whether there are abnormalities in the image to be detected, and provide a basis for the subsequent accurate identification of whether there are abnormalities in the image to be detected.
[0052] In some embodiments, the process of training a denoising diffusion model includes:
[0053] Obtain a general image dataset and an industrial image dataset that is marked as normal;
[0054] For each image in the general image dataset, generate text content used to describe the image, and form the image and the text content used to describe the image into a training sample;
[0055] Based on the training samples, the denoising diffusion model is pre-trained;
[0056] Based on the industrial image dataset, the pre-trained denoising diffusion model is conditionally fine-tuned to obtain the final denoising diffusion model.
[0057] In the embodiment of the present application, the general image dataset may be an ImageNet dataset or other image datasets, which are not specifically limited here. The industrial image dataset marked as normal may be composed of industrial images that are determined to be free of defects.
[0058] In an embodiment of the present application, for each image in a general image data set, a text content for describing the image can be generated by an image description generation model; and then the image and the text content corresponding to the image are formed into a training sample; that is, each training sample constructed exists in the form of an image-text pair. Based on the training samples, the goal of pre-training the denoising diffusion model is to let the denoising diffusion model learn how to generate high-quality images based on the text content in the input training samples. Through multiple iterative training, the denoising diffusion model gradually optimizes its generation ability and context understanding ability. After the pre-training is completed, the denoising diffusion model is conditionally fine-tuned using a data set containing normal industrial images. Unlike traditional supervised learning, in the detection of appearance defects of industrial products, normal samples (such as industrial images without defects) are far more than abnormal samples (such as industrial images with defects). Therefore, only normal industrial image data is used when fine-tuning the denoising diffusion model, and through conditional learning, the denoising diffusion model understands the appearance characteristics of industrial products under normal conditions. This allows the target image generated based on the denoising diffusion model to be used as a reference for product appearance defect detection, thereby ensuring the accuracy of product appearance defect detection.
[0059] In some embodiments, for each image in the general image dataset, generating text content for describing the image includes:
[0060] An image is extracted from the general image dataset in turn, and the extracted image is input into the trained image description generation model, and the text content used to describe the image is determined based on the output of the image description generation model.
[0061] The image description generation model can be a Blip-v2 model or other image-to-text models, which are not specifically limited here. The image description generation model can quickly and accurately generate text content used to describe the image, and then quickly and accurately construct training samples to ensure the efficiency of model training.
[0062] In some embodiments, before pre-training the denoising diffusion model based on the training samples, the method further includes:
[0063] For each image in the training sample, the image is input into the trained classification model, and the classification result is determined according to the output of the classification model; according to the classification result, it is determined whether to remove the training sample.
[0064] Specifically, the classification model may be a Resnet101 model, and the image in each training sample is classified by the Resnet101 model to obtain a classification result including the type of the image and the classification confidence; and whether to remove the training sample is determined according to the classification confidence and a preset confidence threshold. Exemplarily, for a training sample, if the classification confidence of the image in the training sample is less than the confidence threshold, the training sample is removed; otherwise, it is retained.
[0065] It can be understood that by verifying the images in the training samples through the classification model, it is possible to eliminate the wrongly labeled, inconsistent or low-quality training samples; thus, it can ensure that the final training data set is cleaner and more accurate.
[0066] In some embodiments, pre-training the denoising diffusion model based on the training samples includes:
[0067] For any training sample, a noise image corresponding to the image in the training sample is generated, and the text content in the training sample is encoded to obtain the first type of image generation conditions;
[0068] Generate a new image based on the noise image and the first type of image generation conditions through the denoising diffusion model;
[0069] The loss is calculated based on the new image and the images in the training sample, and the parameters of the denoising diffusion model are adjusted based on the calculation results.
[0070] Specifically, the process of generating a noise image corresponding to an image in a training sample includes: determining the size information of the image in the training sample, wherein the size information is used to accurately add noise at each pixel position, and the size information may include information such as the width, height, and pixel depth of the image in the training sample; according to the size information, traversing each pixel of the image in the training sample in turn, and each time a pixel is traversed, generating a corresponding noise value according to a predetermined noise type (such as Gaussian noise), and adding the noise value to the original value of the pixel, and finally obtaining a corresponding noise image.
[0071] Encoding the text content in the training sample to obtain the first type of image generation conditions includes: encoding the text content in the training sample using a text encoder (such as Clip) to obtain the first type of image generation conditions.
[0072] Furthermore, a new image is generated according to the noise image and the first type of image generation condition through the denoising diffusion model; the loss is calculated according to the new image and the image in the training sample, and the parameters of the denoising diffusion model are adjusted according to the calculation results to complete the pre-training of the denoising diffusion model.
[0073] In the embodiment of the present application, by pre-training the denoising diffusion model, the image generation ability and text comprehension ability of the denoising diffusion model can be improved, preparing for subsequent fine-tuning.
[0074] In some embodiments, based on the industrial image dataset, the pre-trained denoising diffusion model is conditionally fine-tuned to obtain the final denoising diffusion model, including:
[0075] For any industrial image in the industrial image dataset, generate a mask image corresponding to the industrial image, text content used to describe the industrial image, and a noise image corresponding to the industrial image;
[0076] Encode the text content into the first type of image generation conditions, and encode the industrial image and mask image into the second type of image generation conditions;
[0077] By using the pre-trained denoising diffusion model, the noisy image is processed according to the first type of image generation condition and the second type of image generation condition to generate a new image;
[0078] According to the loss between the new image and the industrial image, the parameters of the pre-trained denoising diffusion model are fine-tuned to obtain the final denoising diffusion model.
[0079] Specifically, the process of generating a mask image corresponding to the industrial image is similar to the above-mentioned process of generating a mask image corresponding to the image to be detected, and will not be repeated here; similarly, the process of generating text content for describing the industrial image and the process of generating a noise image corresponding to the industrial image can refer to the above-mentioned embodiment of generating text content for describing the image to be detected and generating a noise image corresponding to the image to be detected.
[0080] In this embodiment, the text content in the training sample can be encoded into the first type of image generation conditions through a text encoder, and the industrial image and the mask image can be encoded into the second type of image generation conditions through an image encoder.
[0081] In the embodiment of the present application, the structure of the constructed denoising diffusion model includes a feature encoding module and a feature decoding module; through the pre-trained denoising diffusion model, the noise image is processed according to the first type of image generation conditions and the second type of image generation conditions to generate a new image, including: through the feature encoding module, features are extracted from the input noise image and the first type of image generation conditions and feature integration is performed, and the integrated features are input into the feature decoding module; through the feature decoding module, a new image is constructed according to the integrated features and the second type of image generation conditions. Then, according to the loss between the new image and the industrial image (which can be determined by a preset loss function), the parameters of the pre-trained denoising diffusion model are fine-tuned to obtain the final denoising diffusion model. It can be understood that the fine-tuned denoising diffusion model has learned how to generate normal industrial images, and can subsequently generate normal target images according to the given image generation conditions, so as to determine whether there is an abnormality in the image to be detected based on the target image.
[0082] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0083] Based on the same inventive concept, the embodiment of the present application also provides a product appearance defect detection device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more product appearance defect detection device embodiments provided below can refer to the limitations of the product appearance defect detection method above, and will not be repeated here.
[0084] like Figure 3 As shown, the embodiment of the present application provides a product appearance defect detection device 300, comprising:
[0085] The data processing module 301 is used to obtain an image to be detected of a product to be detected, and generate a mask image corresponding to the image to be detected, a text content used to describe the image to be detected, and a noise image corresponding to the image to be detected;
[0086] The encoding processing module 302 is used to encode the text content into the first type of image generation conditions, and encode the image to be detected and the mask image into the second type of image generation conditions;
[0087] An image generation module 303 is used to process the noise image to generate a target image according to the first type of image generation condition and the second type of image generation condition by using a pre-trained denoising diffusion model; wherein the denoising diffusion model is an image generation model obtained by training based on normal samples;
[0088] The difference comparison module 304 is used to extract the original image of the covered area from the image to be detected according to the mask image, and compare the difference between the original image of the covered area and the target image;
[0089] The defect determination module 305 is used to determine whether the product to be inspected has defects according to the difference comparison result.
[0090] In some embodiments, the structure of the denoising diffusion model includes a feature encoding module and a feature decoding module;
[0091] Accordingly, in terms of processing the noise image to generate the target image according to the first type of image generation condition and the second type of image generation condition by using the pre-trained denoising diffusion model, the image generation module 303 is specifically used for:
[0092] Through the feature encoding module, features are extracted from the input noise image and the first type of image generation conditions and feature integration is performed, and the integrated features are input into the feature decoding module;
[0093] Through the feature decoding module, the target image is constructed based on the integrated features and the second type of image generation conditions.
[0094] In some embodiments, the product appearance defect detection device further includes a model training module. In terms of training the denoising diffusion model, the model training module specifically includes:
[0095] A data acquisition unit, used for acquiring a general image dataset and an industrial image dataset marked as normal;
[0096] A sample construction unit, for generating text content for describing each image in the general image dataset, and combining the image and the text content for describing the image into a training sample;
[0097] A pre-training unit, used for pre-training a denoising diffusion model based on training samples;
[0098] The model fine-tuning unit is used to conditionally fine-tune the pre-trained denoising diffusion model based on the industrial image dataset to obtain the final denoising diffusion model.
[0099] In some embodiments, for each image in the general image dataset, text content for describing the image is generated, and the sample construction unit is specifically used to:
[0100] An image is extracted from the general image dataset in turn, and the extracted image is input into the trained image description generation model, and the text content used to describe the image is determined based on the output of the image description generation model.
[0101] In some embodiments, the product appearance defect detection device further includes:
[0102] The sample classification module is used to input the image into the trained classification model for each image in the training sample, and determine the classification result according to the output of the classification model;
[0103] The sample screening module is used to determine whether to remove training samples based on the classification results.
[0104] In some embodiments, in pre-training the denoising diffusion model based on the training samples, the pre-training unit is specifically used to:
[0105] For any training sample, a noise image corresponding to the image in the training sample is generated, and the text content in the training sample is encoded to obtain the first type of image generation conditions;
[0106] Generate a new image based on the noise image and the first type of image generation conditions through the denoising diffusion model;
[0107] The loss is calculated based on the new image and the images in the training sample, and the parameters of the denoising diffusion model are adjusted based on the calculation results.
[0108] In some embodiments, in terms of conditionally fine-tuning the pre-trained denoising diffusion model based on the industrial image dataset to obtain the final denoising diffusion model, the model fine-tuning unit is specifically used to:
[0109] For any industrial image in the industrial image dataset, generate a mask image corresponding to the industrial image, text content used to describe the industrial image, and a noise image corresponding to the industrial image;
[0110] Encode the text content into the first type of image generation conditions, and encode the industrial image and mask image into the second type of image generation conditions;
[0111] By using the pre-trained denoising diffusion model, the noisy image is processed according to the first type of image generation condition and the second type of image generation condition to generate a new image;
[0112] According to the loss between the new image and the industrial image, the parameters of the pre-trained denoising diffusion model are fine-tuned to obtain the final denoising diffusion model.
[0113] Each module in the above-mentioned product appearance defect detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0114] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the implementation of the product appearance defect detection method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned product appearance defect detection method are implemented.
[0115] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned product appearance defect detection method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0116] Those skilled in the art will understand that Figure 4 or Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0117] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.
[0118] In some embodiments, Figure 6 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0119] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0121] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0122] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0123] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for detecting product appearance defects, characterized in that: include: Acquire an image to be detected of a product to be detected, and generate a mask image corresponding to the image to be detected, a text content used to describe the image to be detected, and a noise image corresponding to the image to be detected; Encoding the text content into a first type of image generation condition, and encoding the image to be detected and the mask image into a second type of image generation condition; The noise image is processed to generate a target image according to the first type of image generation condition and the second type of image generation condition by using a pre-trained denoising diffusion model; wherein the denoising diffusion model is an image generation model obtained by training based on normal samples; Extracting an original image of the covered area from the image to be detected according to the mask image, and performing a difference comparison between the original image of the covered area and the target image; According to the difference comparison result, it is determined whether the product to be inspected has defects.
2. The method according to claim 1, characterized in that The structure of the denoising diffusion model includes a feature encoding module and a feature decoding module; The method of processing the noise image to generate a target image according to the first type of image generation condition and the second type of image generation condition by using the pre-trained denoising diffusion model includes: Extracting features from the input noise image and the first type of image generation conditions through the feature encoding module and integrating the features, and inputting the integrated features into the feature decoding module; The target image is constructed according to the integrated features and the second type of image generation conditions through the feature decoding module.
3. The method according to claim 1 or 2, characterized in that: The process of training the denoising diffusion model includes: Obtain a general image dataset and an industrial image dataset that is marked as normal; For each image in the general image dataset, generate text content for describing the image, and form the image and the text content for describing the image into a training sample; Pre-training the denoising diffusion model based on the training samples; Based on the industrial image dataset, the pre-trained denoising diffusion model is conditionally fine-tuned to obtain a final denoising diffusion model.
4. The method according to claim 3, characterized in that The step of generating text content for describing each image in the general image dataset comprises: An image is extracted from the general image data set in turn, and the extracted image is input into a trained image description generation model, and text content used to describe the image is determined based on the output of the image description generation model.
5. The method according to claim 3, characterized in that: Before pre-training the denoising diffusion model based on the training samples, the method further includes: For each image in the training sample, the image is input into a trained classification model, and a classification result is determined according to an output of the classification model; According to the classification result, it is determined whether to remove the training samples.
6. The method according to claim 3, characterized in that: The pre-training of the denoising diffusion model based on the training sample includes: For any training sample, a noise image corresponding to the image in the training sample is generated, and the text content in the training sample is encoded to obtain a first type of image generation condition; Generate a new image according to the noise image and the first type of image generation condition by using the denoising diffusion model; The loss is calculated according to the new image and the image in the training sample, and the parameters of the denoising diffusion model are adjusted according to the calculation result.
7. The method according to claim 3, characterized in that The conditionally fine-tuning the pre-trained denoising diffusion model based on the industrial image dataset to obtain a final denoising diffusion model includes: For any industrial image in the industrial image dataset, generate a mask image corresponding to the industrial image, text content used to describe the industrial image, and a noise image corresponding to the industrial image; Encoding the text content into a first type of image generation condition, and encoding the industrial image and the mask image into a second type of image generation condition; The noisy image is processed to generate a new image according to the first type of image generation condition and the second type of image generation condition by using a pre-trained denoising diffusion model; According to the loss between the new image and the industrial image, the parameters of the pre-trained denoising diffusion model are fine-tuned to obtain a final denoising diffusion model.
8. A product appearance defect detection device, characterized in that: include: A data processing module, used for acquiring an image to be detected of a product to be detected, and generating a mask image corresponding to the image to be detected, a text content for describing the image to be detected, and a noise image corresponding to the image to be detected; An encoding processing module, used for encoding the text content into a first type of image generation condition, and encoding the image to be detected and the mask image into a second type of image generation condition; An image generation module, configured to process the noise image to generate a target image according to the first type of image generation condition and the second type of image generation condition by using a pre-trained denoising diffusion model; wherein the denoising diffusion model is an image generation model obtained by training based on normal samples; A difference comparison module, used for extracting an original image of a covered area from the image to be detected according to the mask image, and performing a difference comparison between the original image of the covered area and the target image; The defect determination module is used to determine whether the product to be inspected has defects according to the difference comparison result.
9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.