An image defect detection method, device, equipment and storage medium
By using the difference detection of the first and second type image reconstruction models, the problem of detection difficulties caused by the excessive generalization ability of image reconstruction models is solved, and more accurate image defect detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU MEGAROBO TECH CO LTD
- Filing Date
- 2022-10-27
- Publication Date
- 2026-07-10
AI Technical Summary
In existing technologies, image reconstruction models with excessively strong generalization capabilities lead to greater difficulty in detecting image defects and inaccurate detection.
The first and second type of image reconstruction methods are adopted. The image to be detected is input into the first and second type of image reconstruction models that have been trained respectively. The difference between the two types of models is used to determine the defects in the image. The second type of image reconstruction model is trained with the first type of image reconstruction model as the supervision guide.
This reduces the impact of excessive model generalization ability, improves the accuracy and reliability of image defect detection, establishes a joint detection mechanism, and avoids the detection difficulties caused by reconstruction using a single model.
Smart Images
Figure CN115661080B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image defect detection method, apparatus, device, and storage medium. Background Technology
[0002] Depending on the scenario, there may be two reasons for defects in an image. One is that the object included in the image itself has defects; for example, defects in an image of a wafer can reflect quality problems with the wafer itself. The other is that the image quality is poor due to factors such as imaging equipment or shooting techniques, resulting in defects. To address these image defect problems, defect detection is necessary in some scenarios. An image defect detection scheme is introduced below.
[0003] In one example implementation, an image reconstruction model reconstructs a defective input image to obtain an improved image. The defects are then detected by comparing the model's input and output. However, in practical applications, the image reconstruction model used may have strong generalization ability. An overly generalized image reconstruction model can easily reconstruct defects from the input image as well, resulting in defects existing in both the input and reconstructed images. This increases the difficulty of detecting image defects and makes it harder to accurately identify them. Summary of the Invention
[0004] To address the aforementioned issues, this application provides an image defect detection method, apparatus, device, and storage medium, aiming to solve the problems of difficult and inaccurate image defect detection caused by the excessive generalization ability of image reconstruction models.
[0005] The embodiments of this application disclose the following technical solutions:
[0006] The first aspect of this application provides an image defect detection method, including:
[0007] Acquire the image to be detected;
[0008] The image to be detected is input into a trained first-type image reconstruction model and a trained second-type image reconstruction model, respectively, to obtain a first-type reconstructed image output by the first-type image reconstruction model and a second-type reconstructed image output by the second-type image reconstruction model;
[0009] Defects in the image to be detected are determined based on the differences between the first type of reconstructed image and the second type of reconstructed image;
[0010] The second type of image reconstruction model is an image reconstruction model trained with the first type of image reconstruction model as the supervised guidance.
[0011] Optionally, the number of trained second-type image reconstruction models is multiple and different; the step of determining the defects in the image to be detected based on the differences between the first-type reconstructed image and the second-type reconstructed image specifically includes:
[0012] The second-class reconstructed images output by the multiple second-class image reconstruction models are compared one by one with the first-class reconstructed images to obtain multiple preliminary defect detection results of the image to be detected.
[0013] The final defect detection result of the image to be detected is obtained based on the multiple preliminary defect detection results.
[0014] Optionally, obtaining the final defect detection result of the image to be detected based on the plurality of preliminary defect detection results specifically includes:
[0015] Calculate the differences between each pair of the preliminary defect detection results;
[0016] The final defect detection result is obtained by calculating multiple differences.
[0017] Optionally, the step of calculating the final defect detection result based on multiple differences specifically includes:
[0018] The final defect detection result is obtained by summing the multiple differences.
[0019] Optionally, the image defect detection method further includes:
[0020] The first type of image reconstruction model is trained using a first training dataset, and the second type of image reconstruction model is trained using a second training dataset; the first training dataset and the second training dataset are different training datasets.
[0021] Both the first training dataset and the second training dataset include images randomly synthesized from normal images and masked images.
[0022] Optionally, the mask image is randomly generated based on a multi-grid image using one-hot encoding.
[0023] Before determining the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image, the image defect detection method further includes:
[0024] The image subtraction result is obtained by subtracting the image to be detected from the first type of reconstructed image;
[0025] Defects in the image to be detected are identified based on the image subtraction result;
[0026] If the defects in the image to be detected cannot be identified based on the image subtraction result, then the step of determining the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image in the image defect detection method is further executed.
[0027] A second aspect of this application provides an image defect detection apparatus, comprising:
[0028] The image acquisition module is used to acquire the image to be detected.
[0029] The image reconstruction module is used to input the image to be detected into a trained first-type image reconstruction model and a trained second-type image reconstruction model, respectively, to obtain a first-type reconstructed image output by the first-type image reconstruction model and a second-type reconstructed image output by the second-type image reconstruction model;
[0030] A defect determination module is used to determine defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image;
[0031] The second type of image reconstruction model is an image reconstruction model obtained by supervised training guided by the first type of image reconstruction model.
[0032] A third aspect of this application provides an image defect detection device, comprising: a processor and a memory.
[0033] The memory is used to store program code and transmit the program code to the processor;
[0034] The processor is used to execute the steps of the image defect detection method provided in the first aspect according to the instructions in the program code.
[0035] A fourth aspect of this application provides a computer-readable storage medium for storing program code for performing the steps of the image defect detection method provided in the first aspect.
[0036] Compared with the prior art, this application has the following beneficial effects:
[0037] In this application's technical solution, different image reconstruction models are used to reconstruct images, specifically involving a trained first-class image reconstruction model and a second-class image reconstruction model. The second-class image reconstruction model is specifically an image reconstruction model trained under the supervision of the first-class image reconstruction model. In the image defect detection method, the image to be detected is first acquired; then, the image to be detected is input into the first-class image reconstruction model and the second-class image reconstruction model respectively, obtaining a first-class reconstructed image output by the first-class image reconstruction model and a second-class reconstructed image output by the second-class image reconstruction model; finally, the defects in the image to be detected are determined based on the difference between the first-class reconstructed image and the second-class reconstructed image. Since the second-class image reconstruction model is a student model trained using the first-class image reconstruction model as the teacher model, the two are functionally compatible in image reconstruction.
[0038] Because of the differences between the two types of models, even if one model has a strong generalization ability and reconstructs the defect, the other model will not necessarily reconstruct the exact same defect. Therefore, the use of both types of models establishes a joint detection mechanism for image defects. Compared to schemes that use the input and output of a single image reconstruction model to detect defects, the technical solution of this application can reduce the impact of excessive model generalization ability, thus making it easier to accurately detect image defects. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 A flowchart of an image detection method provided in an embodiment of this application;
[0041] Figure 2 A schematic diagram illustrating the determination of defects in an image to be detected, provided as an embodiment of this application;
[0042] Figure 3 This is a schematic diagram illustrating another method for determining defects in an image to be detected, provided as an embodiment of this application.
[0043] Figure 4 This is a schematic diagram illustrating how an image is segmented into multiple sub-images and then further divided into 9 regions in an embodiment of this application;
[0044] Figure 5 A schematic diagram illustrating how to obtain three first defect images based on a first normal image, as provided in an embodiment of this application;
[0045] Figure 6A A flowchart of another image defect detection method provided in the embodiments of this application;
[0046] Figure 6B This is a schematic block diagram of an image detection device provided in an embodiment of this application. Detailed Implementation
[0047] As described above, current image defect detection techniques based on image reconstruction models are prone to reconstructing defects along with the image due to the strong generalization ability of the models themselves. This increases the difficulty of image defect detection and leads to inaccurate detection. To address this issue, this application proposes a novel image defect detection scheme. Specifically, it proposes an image defect detection method, apparatus, device, and storage medium. In this technical solution, a first-class image reconstruction model and a second-class image reconstruction model are employed to establish a joint detection mechanism for image defects. Because the two types of models differ, the detection is less affected by the generalization ability of the models during image reconstruction and defect detection. Compared to schemes that rely on the input and output of a single image reconstruction model to detect defects, this approach more accurately detects image defects.
[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0049] See Figure 1 This figure is a flowchart of an image defect detection method provided in an embodiment of this application. Figure 1 As shown, the image defect detection method includes:
[0050] S101, acquire the image to be detected.
[0051] In this step, the image to be inspected refers to the image for which image defect detection is required. Depending on the detection scenario, the size, format, type, and content of the image to be inspected may vary; therefore, this embodiment does not limit the size, format, type, or content of the image to be inspected. For example, in a wafer quality inspection scenario, the image to be inspected may include the surface of the wafer. In a security scenario, the image to be inspected may be an infrared image. The image to be inspected may be a single, actually captured image or a frame from a video stream.
[0052] S102, the image to be detected is input into the trained first-class image reconstruction model and the trained second-class image reconstruction model respectively, to obtain the first-class reconstructed image output by the first-class image reconstruction model and the second-class reconstructed image output by the second-class image reconstruction model.
[0053] In this embodiment, to achieve accurate image defect detection, a first type of image reconstruction model and a second type of image reconstruction model are employed, which can reconstruct the image to be detected in parallel. For ease of understanding, these two types of image reconstruction models are briefly described below.
[0054] In this embodiment, both the first and second type of image reconstruction models are used for image reconstruction and are trained before step S102, therefore they can be used directly in S102. The second type of image reconstruction model is trained using the first type of image reconstruction model as supervised guidance. The first type of image reconstruction model can be used as the teacher model, and the second type as the student model.
[0055] It is important to note that the second-type image reconstruction model is trained using the first-type image reconstruction model as supervised instruction. Therefore, the trained second-type image reconstruction model is not entirely identical to the trained first-type image reconstruction model. Specifically, this can be seen in the following ways:
[0056] 1) The network type and network parameters of the second type of image reconstruction model are different from those of the first type of image reconstruction model, but the same samples were used for training;
[0057] 2) The network type and network parameters of the second type of image reconstruction model are the same as those of the first type of image reconstruction model, but different samples were used for training;
[0058] 3) The network type and network parameters of the second type of image reconstruction model are different from those of the first type of image reconstruction model, and different samples were used for training.
[0059] For different network types and model parameters, the network of the first type of image reconstruction model can be made more complex, while the network of the second type of image reconstruction model can be relatively simpler. For different sample types used by the models, the first type of image reconstruction model can have more training samples, while the second type of image reconstruction model can have fewer training samples. This is beneficial for the deployment of the second type of reconstruction model.
[0060] In this step, the same image to be detected is used as input to both the first-type image reconstruction model and the second-type image reconstruction model. The first-type and second-type image reconstruction models then perform image reconstruction based on the input images, respectively. To easily distinguish the outputs of the different models, the reconstructed image output by the first-type image reconstruction model is defined as the first-type reconstructed image, and the reconstructed image output by the second-type image reconstruction model is defined as the second-type reconstructed image.
[0061] S103, Determine the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image.
[0062] Because the trained first-class image reconstruction model and the second-class image reconstruction model differ, even if the input image is the same image to be detected, the reconstruction results of the two models will be different. That is, the first-class reconstructed image and the second-class reconstructed image are not exactly the same and there are differences. Based on this difference, it is easier to identify defects in the image to be detected before reconstruction. Figure 2 This is a schematic diagram illustrating how to determine defects in an image to be detected, as provided in an embodiment of this application. Figure 2 The example shows the case where the number of image reconstruction models in the second category is 1.
[0063] In other alternative implementations, multiple second-type image reconstruction models can be used for image reconstruction and to determine defects in the image to be detected. These multiple second-type image reconstruction models differ, specifically in the following ways:
[0064] 1) The model type and network parameters are different, but the training samples are the same;
[0065] 2) The model types and network parameters are the same, but the training samples are different;
[0066] 3) Different model types and network parameters will result in different training samples.
[0067] Since all second-class image reconstruction models are used as student models for easy deployment, multiple second-class image reconstruction models are trained using the first method: the model type is the same, but the network parameters and training samples are different. For example, multiple models are ResNet18 models, but their network parameters are different.
[0068] Figure 3 This is a schematic diagram illustrating another method for determining defects in an image to be detected, as provided in an embodiment of this application. Figure 3As shown, there are multiple second-class image reconstruction models, each trained under the supervision of a first-class image reconstruction model, and each model is different. In this implementation, in the previous step S102, the image to be detected is used as input to both the aforementioned first-class image reconstruction model and the multiple second-class image reconstruction models. Each of these models outputs a reconstructed image, such as... Figure 3 As shown. In this step S103, the defects in the image to be detected are determined based on the difference between the first type of reconstructed image and the second type of reconstructed image, which may specifically include:
[0069] The second-class reconstructed images output by multiple second-class image reconstruction models are compared one by one with the first-class reconstructed images to obtain multiple preliminary defect detection results of the image to be detected; then, the final defect detection result of the image to be detected is obtained based on the multiple preliminary defect detection results.
[0070] In an optional implementation, obtaining the final defect detection result of the image to be detected based on multiple preliminary defect detection results specifically includes: calculating the difference between each pair of the multiple preliminary defect detection results, and calculating the final defect detection result based on the multiple differences. Optionally, calculating the final defect detection result based on multiple differences specifically includes: accumulating the multiple differences to obtain the final defect detection result. For example, the technical solution of this application involves three second-type image reconstruction models. The three preliminary defect detection results are represented by S1, S2, and S3. For the three preliminary defect detection results, calculating the difference between each pair yields S1. 13 S 23 and S 12 Finally, the three differences are summed to obtain the final defect detection result S. final It can be expressed as the following formula:
[0071] S final =S 13 +S 23 +S 12
[0072] By obtaining multiple reconstructed images using several trained second-type image reconstruction models, and subsequently multiple preliminary defect detection results, the problem of individual second-type reconstructed images having similar reconstruction effects to first-type reconstructed images, thus affecting the accuracy of defect detection, is avoided. This further ensures the accuracy of image defect detection. It can be understood that each preliminary defect detection result represents the joint detection effect of the corresponding second-type and first-type image reconstruction models on the image to be detected. The calculation and accumulation of the differences effectively constructs a more reliable, comprehensive, and less prone-to-oversight image defect detection mechanism using multiple second-type image reconstruction models.
[0073] The above describes an implementation of an image defect detection method provided in this application. In this embodiment, different trained image reconstruction models are used to reconstruct the image. In the image defect detection method, the image to be detected is first acquired; then, the image to be detected is input into a first-type image reconstruction model and a second-type image reconstruction model, respectively, to obtain a first-type reconstructed image output by the first-type image reconstruction model and a second-type reconstructed image output by the second-type image reconstruction model; finally, the defects in the image to be detected are determined based on the differences between the first-type reconstructed image and the second-type reconstructed image. Since the second-type image reconstruction model is a student model trained under the supervision of the first-type image reconstruction model, there are certain differences between the two in terms of image reconstruction functionality.
[0074] Because of the differences between the two types of models, even if one model has a strong generalization ability and reconstructs the defect, the other model will not necessarily reconstruct the exact same defect. Therefore, the use of both types of models establishes a joint detection mechanism for image defects. Compared to schemes that use the input and output of a single image reconstruction model to detect defects, the technical solution of this application can reduce the impact of excessive model generalization ability, thus making it easier to accurately detect image defects.
[0075] The embodiments described above mention a first type of image reconstruction model and a second type of image reconstruction model. The training process of the first type of image reconstruction model and the second type of image reconstruction model will be described below with reference to the accompanying drawings and embodiments.
[0076] Based on the preceding description, both the first and second type of image reconstruction models perform image reconstruction based on defective images, aiming to obtain a more complete reconstructed image. For example, the reconstructed image should not contain the defects present in the input image. To train a model with this reconstruction capability, an image set containing defect-free images that do not require reconstruction needs to be constructed. For clarity, the image set used to train the first type of image reconstruction model is referred to as the first image set, which includes multiple normal images. Similarly, a second image set is constructed, which also includes multiple normal images. The normal images mentioned above refer to images that do not require image reconstruction.
[0077] As previously introduced, the first and second type of image reconstruction models represent a teacher model and a student model, respectively. The student model requires supervision and guidance from the teacher model's generalization ability, thus requiring only a small amount of data for training. This "small amount" refers to the comparison with the teacher model. Therefore, the second image set contains fewer images than the first. Consequently, the generalization ability of the second type of image reconstruction model trained in this way is weaker than that of the first type, trained with a much larger amount of data. This results in a difference in the generalization abilities of the two types of image reconstruction models. It should be noted that the second image set can be taken from a portion of the first image set; that is, there can be overlap between the normal images in the two sets. To improve the reconstruction performance of both the student and teacher models, new normal images that do not overlap with the first image set can also be used to construct the second image set.
[0078] In addition to obtaining an image set containing normal images, defective images are also needed to train a model with reconstruction capabilities. These defective images can be generated based on the normal images in the first and second image sets mentioned above.
[0079] In practical applications, a mask image can be generated from a multi-grid image, and then a defective image can be obtained by randomly combining a normal image with the mask image. These images can be used to construct the first training dataset and the second training dataset, respectively. The first image set and the second image set are constructed to generate defective images.
[0080] As an example, a first defective image is generated based on a first normal image, and a first training dataset is constructed based on the generated first defective image. During training, a first defective image and a first normal image with generation relationships are determined from the first training dataset and the first image set, respectively, and used as input data and label data to train a first-class image reconstruction model. The following describes an example implementation of generating a defective image from a normal image. In practice, this method can also be applied to generating a second defective image from a second normal image.
[0081] Generating a first defect image based on a first normal image includes:
[0082] To improve processing speed, the first normal image is typically cropped into multiple sub-images of the same size, such as squares. Each sub-image is then divided into multiple grids, and these grids are randomly masked to obtain a mask image. The first defect image is then randomly synthesized based on the sub-images and the previously generated mask image. In other words, the defect image is generated by masking the multi-grid image segmented from the sub-images. The masked portion constitutes the defect. As an example, the sub-image is a 900*900-pattern image.
[0083] As an example of a method for generating mask images through random masking, the following example illustrates how to obtain mask images by randomly masking a multi-grid image segmented into sub-images:
[0084] Multiple square sub-images of the same size obtained by cropping a normal image are divided into 9 regions using a nine-grid system. Figure 4 This is a schematic diagram illustrating how an image is segmented into multiple sub-images and then further divided into nine regions in an embodiment of this application. For example... Figure 4 As shown, one-hot encoding is used to encode at least one random region segmentation result from the nine region segmentation results, so as to mask the region in the at least one random region segmentation result and obtain a mask image.
[0085] In practice, three region partitioning results can be randomly selected from the nine results and added to the first batch of encoding queues. The subgraphs within these three results are then encoded using one-hot encoding. From the remaining six results, three more are randomly selected and added to the second batch of encoding queues. The subgraphs within these three results are then encoded using one-hot encoding. Finally, the remaining three results are added to the third batch of encoding queues, and the subgraphs within these three results are encoded using one-hot encoding. For example... Figure 4 The three region division results P2, P6 and P7 were added to the first batch of coding queues, P1, P5 and P9 were added to the second batch of coding queues, and P3, P4 and P8 were added to the third batch of coding queues.
[0086] By performing one-hot encoding on the region division results in the three encoding queues, three different first defect images corresponding to the first normal image can be obtained. Figure 5 This is a schematic diagram illustrating how to obtain three first defect images based on a first normal image. (Combined with...) Figure 5 It can be seen that the first defect image can be synthesized from the covered sub-image and the uncovered sub-image.
[0087] Based on the encoding effect of the sub-images in the first batch of encoding queues and the unencoded effect of the sub-images not in the first batch of encoding queues, based on the encoding effect of the sub-images in the second batch of encoding queues and the unencoded effect of the sub-images not in the second batch of encoding queues, and based on the encoding effect of the sub-images in the third batch of encoding queues and the unencoded effect of the sub-images not in the third batch of encoding queues, three different first defect images are obtained respectively.
[0088] certainly Figure 5 The diagram shown is merely a schematic representation of obtaining a first defect image. In other implementations, additional regions from the nine region division results can be selected and added to the encoding queue for encoding. For example, encoding P1 and P2 yields a first defect image, and further encoding P5 yields another first defect image. Furthermore, for images segmented into multiple sub-images, the region division method is not limited to a 9-grid layout; for example, a 16-grid layout could also be used. There is no limitation on the number of sub-grids. Data augmentation is achieved through the example above. It should be noted that the process of obtaining defect images can be performed offline or generated online.
[0089] As mentioned earlier, the training of the second-type image reconstruction model needs to be supervised and guided by the first-type image reconstruction model. Therefore, the second-type image reconstruction model can be trained under supervised guidance after the first-type model has been trained, or the first-type and second-type image reconstruction models can be trained simultaneously after the first-type model has reached a certain stage of training.
[0090] For the second type of image reconstruction model, its training requires, as described above, obtaining a second image set. A second defective image is generated based on the second normal image within this set, and a second training dataset is constructed based on the generated second defect-free image. The method for generating the second defective image is similar to that for generating the first defective image, and will not be repeated here; please refer to the previous description.
[0091] Next, a second defective image and a second normal image with a generation relationship are determined from the second training dataset and the second image set, respectively, and used as input data and label data to train the second type of image reconstruction model (for cases where the first type of image reconstruction model has been trained), or the first type of image reconstruction model and the second type of image reconstruction model are trained (for cases where the first type of image reconstruction model has not been trained).
[0092] In this embodiment, the first type of image reconstruction model can be a model with an encoder and decoder structure, such as a CNN, Transformer, or VAE. The second type of image reconstruction model can be a ResNet18 model. When training the first type of image reconstruction model, a loss function can be constructed based on multiple image quality evaluation metrics. An example loss function is as follows:
[0093] loss = 0.5 * PSNR + 0.5 * GSSIM
[0094] In the loss function expressed above, PSNR stands for Peak Signal-to-Noise Ratio, an image quality assessment metric sensitive to errors between corresponding pixels. GSSIM stands for Gradient SSIM. SSIM stands for Structural Similarity, and GSSIM is a further improvement on SSIM based on gradient information. Since the gradient map contains much important information, and local contrast and local structure are well reflected in the gradient map, using the gradient map to calculate the local contrast similarity and structural similarity in the SSIM metrics is a natural improvement. In the above loss function, both metrics are assigned a weight of 0.5. In other examples, evaluation metrics can be added or removed, or their weights in the loss function can be changed. Here, there are no restrictions on the evaluation metrics and their weights used in the loss function. During the training of the first-class image reconstruction model, the model loss is monitored until the convergence of the first-class image reconstruction model is determined based on the loss.
[0095] The teacher and student models are trained using a second image set and a second training dataset. The second defect images in the second training dataset can be generated online or offline before training. The parameters of the first and second image reconstruction models can still be optimized using the loss function mentioned earlier until both the teacher and student models converge to the optimal loss. Alternatively, during student model training, the teacher model can be used to guide the student models until they converge to the optimal loss, without training the teacher model.
[0096] When training the second type of image reconstruction model, as a manifestation of the supervisory role of the first type of image reconstruction model, the output of the first type of image reconstruction model is used as a soft label to calculate the loss and guide the output of the second type of image reconstruction model, so as to minimize the difference between the output of the second type of image reconstruction model and the normal image (label).
[0097] In the embodiments of this application, the types of the second type of image reconstruction models can be the same or different. For example, they can all be ResNet18 models.
[0098] This application provides another image defect detection method, in which the defect detection scheme mentioned above using the first type of reconstructed image and the second type of reconstructed image is used as a backup scheme when the first type of reconstructed image cannot detect defects. Figure 6A This is a flowchart of the method. For example... Figure 6A The image defect detection methods shown include:
[0099] S91. Obtain the image to be detected.
[0100] S92. Input the images to be detected into the trained first-class image reconstruction model to obtain the first-class reconstructed images output by the first-class image reconstruction model.
[0101] S93. Obtain the image subtraction result by subtracting the image to be detected from the first type of reconstructed image.
[0102] S94. Based on the image subtraction result, perform defect identification on the image to be detected, and determine whether a defect can be identified. If yes, obtain the defect detection result of the image to be detected; otherwise, proceed to S95.
[0103] S95. Input the images to be detected into the trained second-class image reconstruction model to obtain the second-class reconstructed images output by the second-class image reconstruction model.
[0104] S96. Determine the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image.
[0105] In other words, in the technical solution of this application, if the defect in the image to be detected cannot be identified based on the image subtraction result, the step of determining the defect in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image is further executed. The use of the first reconstructed image model and the subtraction operation between the first type of reconstructed image and the image to be detected serve as the first detection mechanism for identifying defects in the image to be detected. If the first detection mechanism fails to detect the defect, there are two possible reasons. One possible reason is that there is no defect in the image to be detected (specifically, the first type of reconstructed image is a normal image, the same as the image to be detected, so the subtraction result is empty, that is, a pure black image is obtained, and it is considered that the defect cannot be identified). Another possible reason is that the first type of image reconstruction model has a strong generalization ability, which causes the defect to be reconstructed in the first type of reconstructed image as well. In order to avoid missing defects, S95-S96 can be executed, using the scheme described in the previous embodiment as the second detection mechanism, relying on the difference between the images reconstructed by the two types of models to detect defects in the image to be detected.
[0106] In other words, this application proposes an alternative scheme for defect detection. This avoids the problem of missed defects due to the overly strong generalization ability of the first type of image reconstruction model. The combined use of the two detection mechanisms, especially the second detection mechanism, can successfully and accurately detect defects in the image.
[0107] The embodiments described in this application are the first to combine image reconstruction technology with teacher and student models to achieve image defect detection. Furthermore, one-hot encoding technology is introduced for the first time during the training phase of the image defect detection scheme, constructing defect images based on normal images. An N-grid algorithm, using a nine-grid algorithm as an example, processes the data stream, achieving efficient expansion of training data. A creative design involves a teacher model guiding a student model to perform multi-channel image reconstruction for image defect detection. These technical solutions represent one of the few feasible image defect detection solutions for industrial applications, and their effectiveness has been verified through practical application.
[0108] Based on the image defect detection method described in the foregoing embodiments, this application also provides an image defect detection device. The implementation of this image defect detection device will be described below with reference to embodiments and accompanying drawings.
[0109] Figure 6B This is a schematic block diagram of an image defect detection device provided in an embodiment of this application. Figure 6B The apparatus shown includes:
[0110] The image acquisition module is used to acquire the image to be detected.
[0111] The image reconstruction module is used to input the image to be detected into a trained first-class image reconstruction model and a trained second-class image reconstruction model, respectively, to obtain the first-class reconstructed image output by the first-class image reconstruction model and the second-class reconstructed image output by the second-class image reconstruction model.
[0112] The defect determination module is used to determine defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image.
[0113] The second type of image reconstruction model is an image reconstruction model trained under the supervision of the first type of image reconstruction model.
[0114] Because of the differences between the two types of models, even if one model has a strong generalization ability and reconstructs the defect, the other model will not necessarily reconstruct the exact same defect. Therefore, the use of both types of models establishes a joint detection mechanism for image defects. Compared to schemes that use the input and output of a single image reconstruction model to detect defects, the technical solution of this application can reduce the impact of excessive model generalization ability, thus making it easier to accurately detect image defects.
[0115] Optionally, the number of trained second-type image reconstruction models is multiple and different; the defect determination module specifically includes:
[0116] The preliminary detection unit is used to compare the second-type reconstructed images output by the multiple second-type image reconstruction models with the first-type reconstructed images to obtain multiple preliminary defect detection results of the image to be detected.
[0117] The final detection unit is used to obtain the final defect detection result of the image to be detected based on the multiple preliminary defect detection results.
[0118] Optionally, the final detection unit specifically includes:
[0119] The difference calculation unit is used to calculate the difference between each pair of the multiple preliminary defect detection results;
[0120] The final defect detection result acquisition unit is used to calculate the final defect detection result based on multiple differences.
[0121] Optionally, the final defect detection result acquisition unit is specifically used for:
[0122] The final defect detection result is obtained by summing the multiple differences.
[0123] Optionally, the image defect detection device further includes:
[0124] A dataset construction unit is used to train a first type of image reconstruction model using a first training dataset and to train a second type of image reconstruction model using a second training dataset; the first training dataset and the second training dataset are different training datasets;
[0125] Both the first training dataset and the second training dataset include images randomly synthesized from normal images and masked images.
[0126] Optionally, the mask image is randomly generated based on a multi-grid image using one-hot encoding.
[0127] Optionally, the defect determination module is also used for:
[0128] Before determining the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image, the image subtraction result is obtained by subtracting the image to be detected from the first type of reconstructed image;
[0129] Defects in the image to be detected are identified based on the image subtraction result;
[0130] If the defects in the image to be detected cannot be identified based on the image subtraction result, then the step of determining the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image is further performed.
[0131] Based on the image defect detection method and image defect detection device introduced above, this application provides an image defect detection device and a computer-readable storage medium.
[0132] Image defect detection equipment includes: a processor and memory.
[0133] The memory is used to store program code and transfer program code to the processor.
[0134] The processor is used to execute the steps of the image defect detection method described above according to the instructions in the program code.
[0135] A computer-readable storage medium is used to store program code that performs the steps of the image defect detection method described in the foregoing embodiments.
[0136] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0137] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An image defect detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into a trained first-type image reconstruction model and a trained second-type image reconstruction model, respectively, to obtain a first-type reconstructed image output by the first-type image reconstruction model and a second-type reconstructed image output by the second-type image reconstruction model; the generalization ability of the second-type image reconstruction model is weaker than that of the first-type image reconstruction model. Defects in the image to be detected are determined based on the differences between the first type of reconstructed image and the second type of reconstructed image; The second type of image reconstruction model is an image reconstruction model trained under the supervision of the first type of image reconstruction model; The number of trained second-type image reconstruction models is multiple and different; the step of determining the defects in the image to be detected based on the differences between the first-type reconstructed image and the second-type reconstructed image specifically includes: The second-class reconstructed images output by the multiple second-class image reconstruction models are compared one by one with the first-class reconstructed images to obtain multiple preliminary defect detection results of the image to be detected. Calculate the differences between each pair of the preliminary defect detection results; The final defect detection result is obtained by summing the multiple differences.
2. The image defect detection method according to claim 1, characterized in that, Also includes: The first type of image reconstruction model is trained using a first training dataset, and the second type of image reconstruction model is trained using a second training dataset; the first training dataset and the second training dataset are different training datasets. Both the first training dataset and the second training dataset include images randomly synthesized from normal images and masked images.
3. The image defect detection method according to claim 2, characterized in that, The mask image is randomly generated based on a multi-grid image using one-hot encoding.
4. The image defect detection method according to any one of claims 1-3, characterized in that, Before determining the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image, the method further includes: The image subtraction result is obtained by subtracting the image to be detected from the first type of reconstructed image; Defects in the image to be detected are identified based on the image subtraction result; If the defects in the image to be detected cannot be identified based on the image subtraction result, then the step of determining the defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image is further performed.
5. An image defect detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected. The image reconstruction module is used to input the image to be detected into a trained first-type image reconstruction model and a trained second-type image reconstruction model, respectively, to obtain a first-type reconstructed image output by the first-type image reconstruction model and a second-type reconstructed image output by the second-type image reconstruction model; the generalization ability of the second-type image reconstruction model is weaker than that of the first-type image reconstruction model. A defect determination module is used to determine defects in the image to be detected based on the difference between the first type of reconstructed image and the second type of reconstructed image; The second type of image reconstruction model is an image reconstruction model trained under the supervision of the first type of image reconstruction model; The number of trained second-type image reconstruction models is multiple and different; the defect determination module specifically includes: The preliminary detection unit is used to compare the second-type reconstructed images output by the multiple second-type image reconstruction models with the first-type reconstructed images to obtain multiple preliminary defect detection results of the image to be detected. The difference calculation unit is used to calculate the difference between each pair of the multiple preliminary defect detection results; The final defect detection result acquisition unit is used to accumulate the multiple differences to obtain the final defect detection result.
6. An image defect detection device, characterized in that, include: Processor and memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the steps of the image defect detection method according to any one of claims 1 to 4, based on the instructions in the program code.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the steps of the image defect detection method according to any one of claims 1 to 4.