Training method, device and computer equipment for image defect detection model

Generating label-free images through feature enhancement models, clustering and adversarial model training, the problem of insufficient samples in micron-level image detection is solved, the accuracy and recall of defect detection are improved, and the generalization ability of the model is enhanced.

CN113610787BActive Publication Date: 2024-11-08GUANGDONG SCI & TECH INFRASTRUCTURE CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110852005.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-27
Publication Date
2024-11-08
Estimated Expiration
2041-07-27

AI Technical Summary

Technical Problem

In micron-scale image detection, the number of samples is small and uneven, resulting in overfitting of the defect detection model, reducing the accuracy of micron-scale defect detection.

Method used

By obtaining the pre-trained feature enhancement model, generating label-free images, and performing clustering processing, building a category adversarial model, using random variable data to generate category feature images, building a defect detection model, and conducting multi-category feature image training to avoid over-conception Together, improve generalization ability.

Benefits of technology

It significantly improves the accuracy and recall of micron-level defect detection, alleviates the problem of insufficient sample size, and enhances the generalization ability of defect detection models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113610787B_ABST
    Figure CN113610787B_ABST
Patent Text Reader

Abstract

The present application relates to a training method, device, computer equipment and storage medium for an image defect detection model. The method includes: a plurality of unlabeled generated images belonging to a target domain can be generated by a first generator in a trained feature enhancement model. In this way, a generated image with greatly enhanced features can be obtained, and then the generated image is clustered to obtain the categories corresponding to each generated image; each category adversarial model is constructed, and each trained category adversarial model is obtained based on the generated image of the same category; the random variable data and the noise data are obtained, and superimposed and input into the second generator in each category adversarial model, which greatly generates category feature images of each category, greatly alleviates the situation of insufficient sample size, can avoid overfitting, and then obtains a trained defect detection model with improved generalization ability based on the feature images of each category, thereby greatly improving the accuracy and recall rate of micron-level defect detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a training method, apparatus, computer equipment and storage medium for an image defect detection model. Background Art

[0002] With the development of image detection technology, when performing defect detection on the surface of products in the fields of new energy, semiconductors, panel displays, circuit boards, etc., image processing algorithms can be used to extract the features of product images and identify image defects based on the feature information.

[0003] However, when performing micron-level image inspection on images, the number of samples with micron-level defects is small and unbalanced, which causes the trained defect detection model to overfit, thereby greatly reducing the accuracy of micron-level defect detection. Summary of the invention

[0004] Based on this, it is necessary to provide a training method, device, computer equipment and storage medium for an image defect detection model to address the above technical problems.

[0005] A method for training an image defect detection model, the method comprising:

[0006] A first generator in a trained feature enhancement model is obtained; wherein the feature enhancement model is pre-trained based on source domain sample images and noise data, and then re-trained based on target domain sample images and noise data; the noise data is input into the first generator to obtain a plurality of unlabeled generated images; the generated images are clustered to obtain the categories corresponding to the generated images; a category adversarial model corresponding to each category is constructed respectively, and the category adversarial model corresponding to the corresponding category is trained based on the generated images of the same category to obtain trained category adversarial models corresponding to each category; random variable data is obtained, and the noise data and the random variable data are superimposed and input into the second generator in each trained category adversarial model to obtain category feature images corresponding to each category; a defect detection model is constructed, and the defect detection model is trained based on a plurality of category feature images to obtain a trained defect detection model, and the trained defect detection model is used to perform defect detection on the image to be detected in the target domain.

[0007] In one embodiment, the feature enhancement model includes a first generator and a first discriminator, and the pre-training step of the feature enhancement model includes:

[0008] Acquire a source domain sample image and noise data; perform feature extraction on the current source domain sample image to obtain the current source domain feature map; generate the current source domain training generated image based on the noise data and the source domain sample hidden features obtained in the previous training iteration through the first generator; input the current source domain feature map and the current source domain training generated image into the first discriminator to obtain a first discrimination result and the current source domain sample hidden features; adjust the model parameters of the feature enhancement model based on the first discrimination result, enter the next loop iteration, and use the current source domain sample hidden features as the source domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, return to the current source domain sample image for feature extraction, and continue to execute the step of obtaining the current source domain feature map until the first training stop condition is reached to obtain a pre-trained feature enhancement model.

[0009] In one embodiment, the retraining step of the feature enhancement model includes:

[0010] Acquire a target domain sample image and noise data; perform feature extraction on the target domain sample image of the current time to obtain a target domain feature map of the current time; generate the target domain training generated image of the current time based on the noise data and the target domain sample hidden features obtained in the previous training iteration and through the first generator obtained by pre-training; input the target domain feature map of the current time and the target domain training generated image of the current time into the first discriminator obtained by pre-training to obtain a second discrimination result and the target domain sample hidden features of the current time; adjust the model parameters of the feature enhancement model obtained by pre-training based on the second discrimination result, enter the next loop iteration, and use the target domain sample hidden features of the current time as the target domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, return to the step of performing feature extraction on the target domain sample image of the current time, and obtain the target domain feature map of the current time, and continue to execute until the second training stop condition is reached to stop and obtain a trained feature enhancement model.

[0011] In one embodiment, the feature extraction is performed on the target domain sample image to obtain the target domain feature map, including:

[0012] The target area in the target domain sample image of the current time is extracted to obtain a target area image; and features are extracted from the target area image to obtain a target domain feature map of the current time.

[0013] In one embodiment, the category adversarial model includes a second generator and a second discriminator, and the generated images based on the same category are trained with the category adversarial model of the corresponding category to obtain the trained category adversarial models corresponding to each category, including:

[0014] The generated image of the same category is obtained, and feature extraction is performed on the generated image of the current time to obtain the feature extraction map of the current time; scene sample noise data and random variable data are obtained, and based on the scene noise data, random variable data and the semi-supervised sample obtained in the previous training iteration, the feature image of the training category of the current time is generated by the second generator; the feature extraction map of the current time and the feature image of the training category of the current time are input into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time; the model parameters of the category adversarial model are adjusted based on the third discrimination result, and the next loop iteration is entered, and the semi-supervised sample of the current time is used as the semi-supervised sample obtained in the previous training iteration corresponding to the next training iteration, and the step of performing feature extraction on the generated image of the current time and obtaining the feature extraction map of the current time is continued until the third training stop condition is reached, and the trained category adversarial model corresponding to the corresponding category is obtained.

[0015] In one embodiment, obtaining scene sample noise data includes:

[0016] Perform scene prejudgment on multiple feature-generated images to determine the target scene corresponding to each of the feature-generated images; and obtain scene sample noise data corresponding to the target scene.

[0017] In one embodiment, the feature extraction image and the training category feature image are input into the second discriminator to obtain a third discrimination result and the semi-supervised sample, including:

[0018] The feature extraction image of the current time and the training category feature image of the current time are input into the second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; when the intermediate semi-supervised sample meets the preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample of the current time.

[0019] A training device for an image defect detection model, the device comprising:

[0020] An acquisition module, used to acquire a first generator in a trained feature enhancement model; wherein the feature enhancement model is pre-trained based on a source domain sample image and noise data, and then re-trained based on a target domain sample image and noise data;

[0021] A first generating module, used for inputting the noise data into the first generator to obtain a plurality of unlabeled generated images;

[0022] A clustering module is used to perform clustering processing on the generated images to obtain the categories corresponding to each generated image;

[0023] The first training module is used to construct a category adversarial model corresponding to each category, and train the category adversarial model of the corresponding category based on the generated images of the same category to obtain the trained category adversarial models corresponding to each category;

[0024] A second generation module is used to obtain random variable data, and superimpose the noise data and the random variable data, and input them into the second generator in each category adversarial model after training, so as to obtain category feature images corresponding to each category;

[0025] The second training module is used to build a defect detection model and train the defect detection model based on multiple category feature images to obtain a trained defect detection model. The trained defect detection model is used to perform defect detection on the image to be detected in the target domain.

[0026] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any of the above-mentioned methods for training an image defect detection model.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements any of the above-mentioned methods for training an image defect detection model.

[0028] The training method, apparatus, computer equipment and storage medium of the above-mentioned image defect detection model can generate multiple unlabeled generated images belonging to the target domain through the first generator in the trained feature enhancement model, wherein the feature enhancement model is pre-trained based on source domain sample images and noise data, and then re-trained based on target domain sample images and noise data. In this way, a generated image with greatly enhanced features can be obtained, and then the generated image is clustered to obtain the categories corresponding to each generated image; a category adversarial model corresponding to each category is constructed respectively, and the category adversarial model with the corresponding category is trained based on the generated image of the same category to obtain the trained category adversarial model corresponding to each category; random variable data is obtained, and the noise data and the random variable data are superimposed and input into the second generator in each trained category adversarial model to obtain the category feature image corresponding to each category. In this way, the images of different categories used for defect detection can be further increased, which greatly alleviates the problem of insufficient sample size, thereby avoiding overfitting. The defect detection model is then constructed, and the defect detection model is trained based on multiple category feature images to obtain a trained defect detection model. In this way, the generalization ability of the defect detection model can be improved. Therefore, defect detection is performed on the image to be detected in the target domain based on the trained defect detection model, which can greatly improve the accuracy and recall rate of micron-level defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A diagram showing an application environment of a training method for an image defect detection model in one embodiment;

[0030] Figure 2 A schematic diagram of a flow chart of a method for training an image defect detection model in one embodiment;

[0031] Figure 3 A schematic diagram of a flow chart of a pre-training step of a feature enhancement model in one embodiment;

[0032] Figure 4 A schematic diagram of a flow chart of a retraining step of a feature enhancement model in one embodiment;

[0033] Figure 5 A schematic diagram of a flow chart of steps for generating a feature enhancement model in one embodiment;

[0034] Figure 6 A schematic diagram of a flow chart of steps for obtaining trained adversarial models of various categories in one embodiment;

[0035] Figure 7 A schematic diagram of a flow chart of steps for generating a category adversarial model corresponding to each category in one embodiment;

[0036] Figure 8 is a flowchart of a method for training an image defect detection model in another embodiment;

[0037] Fig. 9 is a structural block diagram of a training device for an image defect detection model in one embodiment;

[0038] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] The training method of the image defect detection model provided in this application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 via a network. The terminal 102 and the server 104 can be used alone to execute the training method of the image defect detection model, or can be used together to execute the training method of the image defect detection model. Taking the training method of the image defect detection model executed by the server alone as an example, the server 104 obtains the first generator in the trained feature enhancement model; wherein, after the feature enhancement model is pre-trained based on the source domain sample image and noise data, the server 104 retrains it based on the target domain sample image and noise data; the server 104 inputs the noise data into the first generator to obtain a plurality of unlabeled generated images; the server 104 performs clustering processing on the generated images to obtain the categories corresponding to each generated image; the server 104 constructs a category adversarial model corresponding to each category respectively, and trains the category adversarial model corresponding to the corresponding category based on the generated images of the same category to obtain the trained category adversarial models corresponding to each category; the server 104 obtains random variable data, and superimposes the noise data and the random variable data and inputs them into the second generator in each trained category adversarial model to obtain the category feature images corresponding to each category; the server 104 constructs a defect detection model, and trains the defect detection model based on a plurality of category feature images to obtain a trained defect detection model, which is used to perform defect detection on the image to be detected in the target domain. The terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0041] In one embodiment, Figure 2 As shown, a training method for an image defect detection model is provided, and the method is applied to a computer device as an example for explanation. The computer device may be specifically Figure 1 The terminal or server in the image defect detection model includes the following steps:

[0042] Step S202, obtaining a first generator in a trained feature enhancement model; wherein the feature enhancement model is pre-trained based on source domain sample images and noise data, and then re-trained based on target domain sample images and noise data.

[0043] The source domain is the knowledge that has been learned, and the target domain is the new knowledge to be learned. The noise data is noise data carrying multiple task labels, for example, the noise data carries task labels such as incomplete etching, residual copper, needle puncture, small burrs on the line, thin lines, and small gaps on the line.

[0044] Specifically, the computer device constructs a feature enhancement model based on a generative adversarial network, and pre-trains and re-trains the feature enhancement model to obtain a trained feature enhancement model, and based on the trained feature enhancement model, obtains a first generator in the trained feature enhancement model. The feature enhancement model includes a first generator and a first discriminator, the first generator is pre-trained based on a source domain sample image and noise data, and then re-trained based on a target domain sample image and noise image, wherein the first generator can be a deep convolutional network, which can enhance the details of the high-resolution image generated by the adversarial network.

[0045] For example, a computer device constructs a first generator and a first discriminator based on an adversarial training model, constructs a feature enhancement model based on the first generator and the first discriminator, and trains the feature enhancement model based on source domain data of a display panel and target domain data of a circuit board to obtain a trained feature enhancement model, and based on the trained feature enhancement model, obtains a first generator in the trained feature enhancement model, wherein the feature enhancement model includes a first generator and a first discriminator, and the first generator is pre-trained based on source domain sample images and noise data, and then re-trained based on target domain sample images and noise images.

[0046] Step S204: input the noise data into the first generator to obtain a plurality of unlabeled generated images.

[0047] Specifically, the computer device obtains the first generator in the trained feature enhancement model, and inputs the noise data into the first generator to perform feature generation, thereby obtaining a plurality of unlabeled generated images. For example, the computer device obtains the first generator in the trained feature enhancement model, and the first generator may be based on a deep convolutional network, and the noise data carrying task labels such as needle pricks, small burrs on the line, and thin lines are passed through the deep convolutional network to perform feature generation, thereby obtaining a plurality of unlabeled generated images.

[0048] Step S206: clustering the generated images to obtain categories corresponding to the generated images.

[0049] Among them, clustering is the process of learning the feature representation of data and clustering the data based on the feature representation, that is, according to a certain evaluation criterion, the objects in the data are divided into multiple classes according to the similarity of each object in the data. Objects in the same cluster have high correlation and are similar to each other, while objects in different clusters have low correlation and are different from each other. Clustering can be achieved by using SOM (SelfOrganizing Maps, Self-Organizing Map Neural Network) algorithm, K-means clustering algorithm, ROCK algorithm, etc.

[0050] Specifically, the computer device obtains a plurality of unlabeled generated images, and performs clustering processing on the plurality of unlabeled generated images, taking images with the same features as a category, and obtaining the categories corresponding to each generated image. For example, the computer device obtains a plurality of unlabeled generated images, and performs clustering processing using a SOM (Self Organizing Maps) clustering algorithm, and inputs the plurality of unlabeled generated images into the input layer of the SOM, and for each generated image, finds a node in the hidden layer of the SOM that best matches the generated image, and clusters them to obtain the categories in each input generated image.

[0051] Step S208, constructing a category adversarial model corresponding to each category respectively, and training the category adversarial model of the corresponding category based on the generated images of the same category, to obtain a trained category adversarial model corresponding to each category.

[0052] Among them, the type of adversarial model is Generative Adversarial Networks (GAN), which is a deep learning model used for image generation.

[0053] Specifically, the computer device determines the number of categories based on the categories corresponding to each generated image, constructs a category adversarial model corresponding to each category based on the number of categories, and trains the category adversarial model of the corresponding category based on the generated images of the same category to obtain a trained category adversarial model corresponding to each category. The trained category adversarial model can generate an image with high authenticity based on the input noise data.

[0054] For example, the computer device obtains categories of each generated image according to clustering processing. For example, there are three categories, namely residual copper, burrs, and notches. Based on the categories, a residual copper category adversarial model corresponding to the residual copper category, a burr category adversarial model corresponding to the burr category, and a notch category adversarial model corresponding to the notch category are respectively constructed. For the residual copper category adversarial model corresponding to the residual copper category, the generated image corresponding to the residual copper is input for training, and a trained residual copper category adversarial model corresponding to the residual copper category is obtained. The trained residual copper category adversarial model can generate an image with residual copper based on the input noise.

[0055] Step S210, obtaining random variable data, and superimposing the noise data and the random variable data, and inputting them into the second generator in each category adversarial model after training, to obtain category feature images corresponding to each category.

[0056] The random variable data is data related to noise, and the random variable data can be any random noise; the noise data is noise data carrying multiple task labels, for example, the noise data carries task labels such as incomplete etching, residual copper, needle puncture, small burrs on the line, thin lines, and small gaps on the line. The second generator in each category adversarial model adopts a multi-layer convolutional network.

[0057] Specifically, the computer device obtains random variable data, and superimposes the noise data and the random variable data and inputs them together into the second generator in each category adversarial model that has been trained. The second generator uses a multi-scale convolution kernel to perform a convolution operation on the image to generate category feature images corresponding to each category.

[0058] For example, a computer device obtains random variable data and obtains a trained residual copper category adversarial model, a trained burr category adversarial model, and a trained gap category adversarial model. For each category adversarial model, the random variable data and noise data carrying task labels such as needle pricks, small burrs on the line, thin lines, etc. are superimposed and input together into the second generator in the trained category adversarial model corresponding to the category. For example, the random variable data and noise data carrying task labels such as needle pricks, small burrs on the line, thin lines, etc. are superimposed and input together into the second generator in the trained residual copper category adversarial model to obtain a residual copper feature image.

[0059] Step S212, constructing a defect detection model, and training the defect detection model based on multiple category feature images to obtain a trained defect detection model, and the trained defect detection model is used to perform defect detection on the image to be detected in the target domain.

[0060] Specifically, the computer device builds a defect detection model based on a convolutional neural network, and obtains multiple category feature images corresponding to each category. The computer device trains the defect detection model based on the multiple category feature images to obtain a trained defect detection model. The trained defect detection model is used to perform defect detection on the image to be detected in the target domain.

[0061] The training method of the above-mentioned image defect detection model can generate multiple unlabeled generated images belonging to the target domain through the first generator in the trained feature enhancement model, wherein the feature enhancement model is pre-trained based on the source domain sample image and noise data, and then re-trained based on the target domain sample image and noise data. In this way, a generated image with greatly enhanced features can be obtained, and then the generated image is clustered to obtain the categories corresponding to each generated image; a category adversarial model corresponding to each category is constructed respectively, and the category adversarial model with the corresponding category is trained based on the generated image of the same category to obtain the trained category adversarial model corresponding to each category; random variable data is obtained, and the noise data and the random variable data are superimposed and input into the second generator in each trained category adversarial model to obtain the category feature image corresponding to each category. In this way, the images of different categories used for defect detection can be further increased, which greatly alleviates the problem of insufficient sample size, thereby avoiding overfitting. The defect detection model is then constructed, and the defect detection model is trained based on multiple category feature images to obtain a trained defect detection model. In this way, the generalization ability of the defect detection model can be improved. Therefore, defect detection is performed on the image to be detected in the target domain based on the trained defect detection model, which can greatly improve the accuracy and recall rate of micron-level defect detection.

[0062] In one embodiment, Figure 3 As shown, the feature enhancement model includes a first generator and a first discriminator, and the pre-training steps of the feature enhancement model include:

[0063] Step S302: Acquire source domain sample images and noise data.

[0064] The noise data is noise data carrying multiple task labels, for example, the noise data carries task labels such as incomplete etching, residual copper, needle puncture, small burrs on the line, thin lines, and small gaps on the line.

[0065] Specifically, the computer device obtains source domain sample images and noise data carrying multi-task labels, wherein the source domain sample images may be knowledge domain sample images based on the fields of semiconductors, panel displays, circuit boards, etc.

[0066] Step S304, extracting features from the current source domain sample image to obtain the current source domain feature map.

[0067] Specifically, the computer device performs feature extraction based on the acquired source domain sample image to obtain the current source domain feature map. For example, the computer device cuts the area to be extracted in the current source domain sample image corresponding to the panel display to obtain the image of the area to be extracted, and the computer device performs feature extraction on the image of the area to be extracted to obtain the current source domain feature map.

[0068] Step S306: Based on the noise data and the hidden features of the source domain samples obtained in the previous training iteration, the source domain training generated image is generated through the first generator.

[0069] Among them, latent features can be understood as latent variables, which are unobservable variables that will affect the state of the system and the observable output.

[0070] Specifically, the computer device obtains noise data carrying multiple task labels and hidden features of source domain samples obtained in the previous training iteration, inputs the noise data and hidden features obtained in the previous training iteration into the first generator, and generates the source domain training generated image of the current time through the first generator. For example, the computer device obtains noise data carrying task labels such as incomplete etching, residual copper, needle piercing, and small burrs on the line, and the noise data can be matrix data, and obtains hidden features of source domain samples corresponding to the panel display obtained in the previous training iteration, and superimposes the noise data and the hidden features of the source domain samples and inputs them into the first generator, and generates the source domain training generated image corresponding to the panel display of the current time through the first generator, wherein the first generator can be constructed based on a deep convolutional network.

[0071] Step S308: input the current source domain feature map and the current source domain training generated image into the first discriminator to obtain the first discrimination result and the current source domain sample hidden feature.

[0072] Among them, the activation function of the first discriminator can be LReLU (G) or Sigmoid (D), and fractional strided convolution is adopted. In addition, the first discriminator realizes batch normalization by normalizing each batch of data, which can avoid the problem of gradient disappearance or gradient explosion.

[0073] Specifically, the computer device obtains the current source domain feature map and the current source domain training generated map, and inputs the current source domain feature map and the current source domain training generated map into the first discriminator to obtain a first discrimination result and the current source domain sample hidden features, wherein the first discrimination result represents the ability of the first generator to generate images, that is, the probability of judging whether the generated image is real. For example, the computer device inputs the current source domain feature map and the current source domain training generated map into the first discriminator for fractional strided convolution operation to obtain the first discrimination result and the current source domain sample hidden features. If the first discrimination result is 1, it represents that the image generated by the first generator is the same as the real image. If the first discriminator is 0, it represents that the image generated by the first generator is different from the real image, and the first generator needs to be trained.

[0074] Step S310, based on the first discrimination result, the model parameters of the feature enhancement model are adjusted, and the next loop iteration is entered, and the hidden features of the source domain samples of the current time are used as the hidden features of the source domain samples obtained in the previous training iteration corresponding to the next training iteration, and the feature extraction of the source domain sample images of the current time is returned to obtain the step of the source domain feature map of the current time, and the step is continued until the first training stop condition is reached to obtain the pre-trained feature enhancement model.

[0075] Specifically, the computer device obtains the first discrimination result. When the first discrimination result is not 0, the computer device adjusts the model parameters of the feature enhancement model based on the first discrimination result, enters the next loop iteration, and uses the hidden features of the source domain samples of the current time as the hidden features of the source domain samples obtained in the previous training iteration corresponding to the next training iteration, returns to the step of performing feature extraction on the source domain sample image of the current time, and continues to execute the step of obtaining the source domain feature image of the current time until the first training stop condition is reached, and stops to obtain the pre-trained feature enhancement model. The first training stop condition may be that the number of iterations reaches a first predetermined number of iterations, or that the iteration time reaches a first predetermined iteration time.

[0076] For example, the computer device obtains the first discrimination result. When the first discrimination result is not 0, the computer device adjusts the parameters of the first generator and the first discriminator based on the first discrimination result, sets the first predetermined number of iterations, performs the next loop iteration, and uses the hidden features of the source domain samples of the current time as the source domain sample features obtained in the previous training iteration corresponding to the next training iteration, returns the step of performing feature extraction on the source domain sample image of the current time, and continues to execute the step of obtaining the source domain feature image of the current time until the number of iterations reaches the first predetermined number of iterations, and obtains the pre-trained feature enhancement model.

[0077] In this embodiment, a source domain sample image and noise data are obtained; feature extraction is performed on the current source domain sample image to obtain the current source domain feature map; based on the noise data and the source domain sample hidden features obtained in the previous training iteration, the current source domain training generated image is generated through the first generator; the current source domain feature map and the current source domain training generated image are input into the first discriminator to obtain the first discrimination result and the current source domain sample hidden features; based on the first discrimination result, the model parameters of the feature enhancement model are adjusted, and the next loop iteration is entered, and the current source domain sample hidden features are used as the source domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, and the step of performing feature extraction on the current source domain sample image and obtaining the current source domain feature map is returned and continued to be executed until the first training stop condition is reached, and the pre-trained feature enhancement model is obtained. In this way, the first generator in the pre-trained feature enhancement model can be used for knowledge transfer in the target domain, so that a feature enhancement model that enhances micron-level defect features can be generated.

[0078] In one embodiment, Figure 4 As shown, the retraining steps of the feature enhancement model include:

[0079] Step S402, obtaining a target domain sample image and noise data.

[0080] The noise data is noise data carrying multiple task labels, for example, the noise data carries task labels such as incomplete etching, residual copper, needle puncture, small burrs on the line, thin lines, and small gaps on the line.

[0081] Specifically, the computer device obtains a target domain sample image and noise data carrying multiple task labels, wherein the target domain sample image may be a knowledge domain sample image to be learned based on the fields of semiconductors, panel displays, circuit boards, etc.

[0082] Step S404, extracting features from the current target domain sample image to obtain the current target domain feature map.

[0083] Specifically, the computer device performs feature extraction based on the acquired target domain sample image to obtain the target domain feature map of the current time. For example, the computer device cuts the area to be extracted in the target domain sample image corresponding to the circuit board of the current time to obtain the image of the area to be extracted, and the computer device performs feature extraction on the image of the area to be extracted to obtain the target domain feature map of the current time.

[0084] Step S406, based on the noise data and the target domain sample hidden features obtained in the previous training iteration, the target domain training generated image is generated by the first generator obtained by pre-training.

[0085] Among them, latent features can be understood as latent variables, which are unobservable variables that will affect the state of the system and the observable output.

[0086] Specifically, a computer device obtains noise data carrying multiple task labels and hidden features of target domain samples obtained in the previous training iteration. The hidden features of the target domain samples are learned by the first discriminator in the previous iterative training to distinguish between suitable feature blocks and inappropriate synthetic feature blocks in the source domain and the target domain, and determine the domain invariant features of the previous training iteration. The domain invariant features are the hidden features of the target domain samples obtained in the previous training iteration. The computer device inputs the noise data and the hidden features obtained in the previous training iteration into the first generator, and generates the target domain training generated image of this time through the first generator, so as to further enhance the tiny features.

[0087] For example, a computer device obtains noise data carrying task labels such as incomplete etching, residual copper, needle punctures, and small burrs on the line. The noise data can be matrix data, and obtains the target domain sample hidden features corresponding to the circuit board obtained in the previous training iteration. The target domain sample hidden features are the first discriminator in the previous iterative training that learns to distinguish between suitable feature blocks and inappropriate synthetic feature blocks in the source domain and the target domain, and determines the domain invariant features of the previous training iteration. The computer device finds the cross-domain hidden feature space through data mapping and transfers the domain invariant features based on transfer learning. The domain invariant features are the target domain sample hidden features obtained in the previous training iteration. The computer device inputs the noise data and the hidden features obtained in the previous training iteration into the first generator, and superimposes the noise data and the target domain sample hidden features and inputs them into the first generator. The target domain training generated image corresponding to the circuit board is generated by the first generator.

[0088] Step S408, input the current target domain feature map and the current target domain training generated image into the first discriminator obtained by pre-training, and obtain the second discrimination result and the current target domain sample hidden feature.

[0089] The activation function of the first discriminator can be LReLU or Sigmoid, and fractional strided convolution is used. In addition, the first discriminator implements batch normalization by normalizing each batch of data, which can avoid the problem of gradient disappearance or gradient explosion.

[0090] The computer device obtains the current target domain feature map and the current target domain training generated map, and inputs the current target domain feature map and the current target domain training generated map into the first discriminator to obtain the first discrimination result and the current target domain sample hidden feature, the current target domain sample feature is the updated domain invariant feature, that is, the current target domain sample hidden feature is obtained by updating the previous target domain sample feature, wherein the first discrimination result represents the ability of the first generator to generate images, that is, the probability of judging whether the generated image is real. For example, the computer device inputs the current target domain feature map and the current target domain training generated map into the first discriminator for fractional strided convolution operation to obtain the first discrimination result and the current target domain sample hidden feature, if the first discrimination result is 1, it represents that the image generated by the first generator is the same as the real image, if the first discriminator is 0, it represents that the image generated by the first generator is different from the real image, and the first generator needs to be trained.

[0091] Step S410, based on the second discrimination result, the model parameters of the pre-trained feature enhancement model are adjusted, and the next loop iteration is entered, and the target domain sample latent features of the current time are used as the target domain sample latent features obtained in the previous training iteration corresponding to the next training iteration, and the step of performing feature extraction on the target domain sample image of the current time is returned to obtain the target domain feature map of the current time, and the step is continued until the second training stop condition is reached to obtain the trained feature enhancement model.

[0092] Specifically, the computer device obtains the first discrimination result. When the first discrimination result is not 0, the computer device adjusts the model parameters of the pre-trained feature enhancement model based on the first discrimination result, enters the next loop iteration, and uses the target domain sample latent features of the current time as the target domain sample latent features obtained in the previous training iteration corresponding to the next training iteration, returns to the step of performing feature extraction on the target domain sample image of the current time, and continues to execute the step of obtaining the target domain feature image of the current time until the second training stop condition is reached, and the trained feature enhancement model is obtained. The second training stop condition may be that the number of iterations reaches a second predetermined number of iterations, or that the iteration time reaches a second predetermined iteration time.

[0093] For example, the computer device obtains the first discrimination result. When the first discrimination result is not 0, the computer device adjusts the parameters of the first generator and the first discriminator based on the first discrimination result, sets a second predetermined number of iterations, performs the next loop iteration, and uses the target domain sample latent features of the current time as the target domain sample features obtained in the previous training iteration corresponding to the next training iteration, returns the step of performing feature extraction on the target domain sample image of the current time, and continues to execute the step of obtaining the target domain feature image of the current time until the number of iterations reaches the second predetermined number of iterations, and obtains the trained feature enhancement model.

[0094] In this embodiment, a target domain sample image and noise data are obtained, so that the generated features can be made more noise-resistant by adding noise data, and features are extracted from the target domain sample image of the current time to obtain the target domain feature map of the current time; based on the noise data and the target domain sample hidden features obtained in the previous training iteration, the target domain training generated image of the current time is generated by the first generator obtained by pre-training; the target domain feature map of the current time and the target domain training generated image of the current time are input into the first discriminator obtained by pre-training to obtain a second discriminant result and the target domain sample hidden features of the current time; based on the second discriminant result, the model parameters of the feature enhancement model obtained by pre-training are adjusted. Adjustment is performed, and the next loop iteration is entered. The hidden features of the target domain samples of the current time are used as the hidden features of the target domain samples obtained in the previous training iteration corresponding to the next training iteration. The step of extracting features of the target domain sample images of the current time and obtaining the target domain feature map of the current time is continued until the second training stop condition is reached, and a trained feature enhancement model is obtained. Therefore, by feeding back the first discrimination result obtained by the first discriminator to the first generator, the features can be further enhanced, thereby generating more refined image features, and it is helpful to overcome the problem of difficulty in convergence when finding Nash equilibrium in traditional generative adversarial networks, so that the training is carried out in the direction of convergence.

[0095] In order to more clearly understand the generation of feature enhancement models, such as Figure 5As shown, a more detailed embodiment is provided for description. The computer device obtains a target domain sample image and noise data carrying multiple task labels, and inputs the noise data into the first generator of the feature enhancement model constructed by the deep convolutional network to obtain the target domain training generated image. The computer device extracts features from the target domain sample image to obtain a target feature map, and inputs the target feature map and the target domain training generated image into the first discriminator in the feature enhancement model. The first discriminator learns to distinguish between suitable feature blocks and inappropriate synthetic feature blocks in the source domain and the target domain, determines domain invariant features, and uses the domain invariant features as target domain sample hidden features, and obtains a second discrimination result. The computer device inputs the target domain sample hidden features and the noise data into the first generator to start the next iteration, and continuously updates the target domain sample hidden features through iteration until the second training stop condition is reached, thereby obtaining a trained feature enhancement model.

[0096] In this embodiment, the noise data carrying multiple task labels is passed through the first generator of the deep convolutional network to generate a target domain training generated image with finer features. Based on the target domain training generated image and the target domain sample image, feature extraction is performed on the target domain sample image to obtain a target feature map. The domain invariant features are determined by the first discriminator, and the domain invariant features are fed back to the first generator as latent features of the target domain samples through transfer learning. The trained feature enhancement model is finally obtained through iterative training, which can generate finer features, thereby enabling defect detection of micron-level images.

[0097] In one embodiment, the feature extraction is performed on the target domain sample image to obtain the target domain feature map, including:

[0098] The target area in the target domain sample image of the current time is extracted to obtain a target area image; and features are extracted from the target area image to obtain a target domain feature map of the current time.

[0099] Specifically, the computer device obtains the target domain image of the time and determines the target area. The computer device cuts and extracts the target area in the target domain sample image of the time to obtain the target domain area image. The computer device extracts features of the target domain area image and converts high-dimensional pixel features into low-dimensional features to obtain the target domain feature map of the time.

[0100] In this embodiment, the target area in the target domain sample image of the current time is extracted to obtain the target area image; the feature extraction is performed on the target area image to obtain the target domain feature map of the current time, so that the target domain feature map of the current time is input into the first discriminator to obtain more representative features.

[0101] In one embodiment, Figure 6As shown, the category adversarial model includes a second generator and a second discriminator, and the generated images based on the same category are trained with the category adversarial model of the corresponding category to obtain the trained category adversarial models corresponding to each category, including:

[0102] Step S602, obtaining a generated image of the same category, and performing feature extraction on the generated image of the current time to obtain a feature extraction graph of the current time.

[0103] Specifically, the computer device obtains the category of each generated image obtained through clustering processing, obtains a generated image of the same category from multiple generated images, and performs feature extraction on the generated image at that time to obtain a feature extraction map at that time. For example, the computer device obtains a generated image of the category of burrs from multiple generated images, and performs feature extraction on each generated image of the category of burrs to obtain a feature extraction map at that time.

[0104] Step S604, obtaining scene sample noise data and random variable data, and generating the current training category feature image through the second generator based on the scene noise data, random variable data and the semi-supervised samples obtained in the previous training iteration.

[0105] Among them, the scene sample noise data is noise data related to the scene and carries multiple task labels, the random variable data is data related to the noise, and the random variable data can be any random noise, and the semi-supervised sample is a hidden code c. Referring to the Info GAN structure framework, and considering that the hidden code c of the Info GAN model is correlated with the image features, the model can be controlled to generate different types of pictures by controlling the difference in hidden code c. The second generator is a multi-layer convolutional network, and each layer of the convolutional network uses transposed convolution operations (G) and convolution operations (D). The activation functions use ReLU (G) and LeakyReLU (D). The output layer of the generator uses convolution operations and the activation functions use Tan (G) and Softmax (D).

[0106] Specifically, the computer device obtains scene sample noise data and random variable data related to the scene and carrying multiple task labels, and obtains a semi-supervised sample obtained in the previous training iteration. The computer device performs a convolution operation through a multi-layer convolutional network of a second generator based on the scene noise data, random variable data and the semi-supervised sample obtained in the previous training iteration to generate a feature image of the training category at that time.

[0107] For example, the computer device obtains scene sample noise data and random variable data of a circuit board scene, and obtains a semi-supervised sample obtained in the previous training iteration. The semi-supervised sample is the adversarial domain adaptation learning of the second discriminator in the previous iterative training, and the difference between the target sample and the source domain sample is judged by the domain discriminator, and the computer device performs transfer learning according to the correlation between cross-domain tasks and task covariance to obtain a semi-supervised sample. The computer device performs convolution operation based on the scene noise data, random variable data and the semi-supervised sample obtained in the previous training iteration through the multi-layer convolution network of the second generator to generate the training category feature image of the current time. Among them, the second discriminator is composed of multi-layer convolution and multi-layer full connection, and the activation function is LReLU, while the activation function of the second generator is ReLU, which can effectively avoid overfitting.

[0108] Step S606: input the feature extraction image of the current time and the training category feature image of the current time into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time.

[0109] Among them, the second discriminator is composed of multi-layer convolution and multi-layer full connection, the activation function adopts LReLU and Sigmoid, and the fractional step convolution is adopted. In addition, the second discriminator realizes batch normalization by normalizing each batch of data, which can avoid the problem of gradient disappearance or gradient explosion.

[0110] Specifically, the computer device obtains the feature extraction graph of the current time and the feature image of the training category of the current time, and inputs the feature extraction graph of the current time and the feature image of the training category of the current time into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time, wherein the semi-supervised sample of the current time is the semi-supervised sample obtained in the previous training iteration after the update, wherein the third discrimination result characterizes the ability of the second generator to generate images, that is, the probability of judging that the generated image is close to the real image. For example, the computer device inputs the feature extraction graph of the current time and the feature image of the training category of the current time into the second discriminator, and obtains the second discrimination result and the semi-supervised sample of the current time through fractional strided convolution operation based on the activation function of LReLU and Sigmoid.

[0111] Step S608, based on the third discrimination result, the model parameters of the category adversarial model are adjusted, and the next loop iteration is entered, and the semi-supervised sample of the current time is used as the semi-supervised sample obtained in the previous training iteration corresponding to the next training iteration, and the feature extraction of the generated image of the current time is returned to obtain the step of the feature extraction graph of the current time, and the training is stopped when the third training stop condition is reached, and the category adversarial model corresponding to the corresponding category is obtained.

[0112] Specifically, the computer device adjusts the model parameters of the category adversarial model based on the third discrimination result, performs the next loop iteration, and uses the semi-supervised sample of the current time as the semi-supervised sample obtained in the current training iteration corresponding to the next training iteration, returns to the step of performing feature extraction on the generated image of the current time, and continues to execute the step of obtaining the feature extraction map of the current time until the third training stop condition is reached, and obtains the trained category adversarial model corresponding to the corresponding category, integrates the various category adversarial models, and obtains a classification decision model for multiple categories. The third training stop condition may be that the number of iterations reaches a third predetermined number of iterations, or that the iteration time reaches a third predetermined iteration time.

[0113] For example, the computer device obtains the third discrimination result, and adjusts the model parameters of the category adversarial model based on the third discrimination result, performs the next loop iteration, and uses the semi-supervised sample of the current time as the semi-supervised sample obtained in the current training iteration corresponding to the next training iteration, returns to the step of performing feature extraction on the generated image of the current time, and continues to execute the step of obtaining the feature extraction map of the current time until the number of iterations reaches the third predetermined number of iterations, and obtains the trained category adversarial model corresponding to the corresponding category. If the category of each generated image is determined to be residual copper, burr, or notch through clustering processing, the computer device obtains the trained residual copper category adversarial model, the trained burr category adversarial model, and the trained notch category adversarial model, and integrates the three category adversarial models to obtain a classification decision model corresponding to the three categories.

[0114] In this embodiment, a generated image of the same category is obtained, and feature extraction is performed on the generated image of the current time to obtain the feature extraction map of the current time; scene sample noise data and random variable data are obtained, and based on the scene noise data, random variable data and the semi-supervised sample obtained in the previous training iteration, the feature image of the training category of the current time is generated by the second generator; the feature extraction map of the current time and the feature image of the training category of the current time are input into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time; based on the third discrimination result, the model parameters of the category adversarial model are adjusted to enter the next loop iteration, and the semi-supervised sample of the current time is used as the semi-supervised sample obtained in the previous training iteration corresponding to the next training iteration, and the step of performing feature extraction on the generated image of the current time and obtaining the feature extraction map of the current time is continued until the third training stop condition is reached, and the trained category adversarial model corresponding to the corresponding category is obtained. In this way, images of different categories used for defect detection can be further increased, thereby improving the generalization ability of the defect detection model.

[0115] In order to more clearly understand the generation process of the category adversarial model corresponding to each category, take the generation process of the category adversarial model of category A as an example. Figure 7 As shown, a more detailed embodiment is provided for description. The computer device obtains a generated image of category A, and scene sample noise data and random variable data carrying multiple task labels, and inputs the scene sample noise data and random variable data into the second generator of the category adversarial model of category A to obtain a training category feature image of category A. The computer device extracts features from the generated image of category A to obtain a feature extraction map of category A, and inputs the feature extraction map of category A and the training category feature image of category A into the second discriminator of the category adversarial model of category A to obtain a third discrimination result, and the second discriminator performs adversarial domain adaptation learning, and judges the difference between the target sample and the source domain sample through the domain discriminator, and the computer device performs transfer learning according to the correlation and task covariance between cross-domain tasks to obtain a semi-supervised sample, and the computer device inputs the semi-supervised sample, the scene sample noise data, and the random variable data into the second generator of the category adversarial model of category A to start the next iteration, and continuously updates the semi-supervised sample through iteration until the third training stop condition is reached, and the trained category adversarial model of category A is obtained.

[0116] In this embodiment, scene sample noise data, random variable data, and semi-supervised samples carrying multiple task labels are input into the second generator in the category adversarial model to obtain a training category feature image corresponding to the category, and based on the generated image corresponding to the category adversarial model, a feature extraction map is obtained, and a semi-supervised sample is determined based on the feature extraction map and the training category feature image, and the semi-supervised sample is fed back to the second generator, and a trained category adversarial model corresponding to the category is obtained through iterative training, so that category adversarial models of multiple categories can be obtained, and then the different categories of images used for defect detection can be further increased, thereby improving the generalization ability of the defect detection model.

[0117] In one embodiment, obtaining scene sample noise data includes:

[0118] Perform scene prejudgment on multiple feature-generated images to determine the target scene corresponding to each of the feature-generated images; and obtain scene sample noise data corresponding to the target scene.

[0119] Specifically, the computer device obtains multiple feature-generated images, and performs scene prejudgment on the multiple feature-generated images to determine the target scene corresponding to each of the feature-generated images. Based on the target scene, the computer device obtains scene sample noise data corresponding to the target scene and carrying multiple task labels. For example, the computer device obtains multiple feature-generated images, and performs scene prejudgment on the multiple feature-generated images based on the scene prejudgment model to determine the target scene corresponding to the distribution of each of the feature-generated images. For example, when the target scene is a circuit board scene, the computer device obtains circuit board sample noise carrying multiple task noises in the circuit board scene.

[0120] In this implementation, scene prediction is performed on multiple feature-generated images to determine the target scenes corresponding to each of the feature-generated images; scene sample noise data corresponding to the target scene is obtained, so that the noise data corresponding to the target scene can be quickly determined, which can improve the accuracy of the category adversarial model, thereby improving the detection efficiency and accuracy of the defect detection model.

[0121] In one embodiment, the feature extraction graph and the training category feature image are input into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the time, including:

[0122] The feature extraction image of the current time and the training category feature image of the current time are input into the second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; when the intermediate semi-supervised sample meets the preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample of the current time.

[0123] Specifically, the computer device obtains the feature extraction image of the current time and the feature image of the training category of the current time, and inputs the feature extraction image of the current time and the feature image of the training category of the current time into the second discriminator to obtain the third discrimination result and the intermediate semi-supervised sample. The computer device determines the preset threshold, for example, the preset threshold is selected as 0.75-0.85, and the semi-supervised sample is extracted according to the preset threshold. When the intermediate semi-supervised sample meets the preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample of the current time. Among them, if the preset threshold is too high, it is easy to cause the samples generated by the generator to cause overfitting after training, and the generalization ability is weak. If it is too low, it will have little effect on the accuracy, but will cause the calculation speed to decrease.

[0124] In this embodiment, the feature extraction image of the time and the training category feature image of the time are input into the second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; when the intermediate semi-supervised sample meets the preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample of the time. In this way, the category adversarial model can be trained based on more suitable semi-supervised samples. Therefore, the trained category adversarial model can help improve the generalization ability of the defect detection model.

[0125] In order to have a clearer understanding of the training of image defect detection models, Figure 8 As shown, a more detailed embodiment is provided for description. First, pre-training is performed based on source domain sample images and noise data to obtain a trained pre-trained feature enhancement model, and then re-training is performed based on target domain sample images and noise data, and a trained feature enhancement model is obtained through transfer learning (i.e., knowledge transfer). Then, noise data carrying multiple task labels is input into the first generator in the feature enhancement model to obtain multiple unlabeled generated images with feature enhancement, and clustering is performed on the generated images to obtain the categories corresponding to each generated image, and a category adversarial model corresponding to each category is constructed to obtain a multi-category category adversarial model. For each category adversarial model, a generated image of the same category is obtained, and a scene is pre-judged for the generated image to determine the target scene, and scene sample noise data is determined based on the target scene. The computer device inputs the scene sample noise data carrying multiple task labels (i.e., multi-task source) and random variable data into the second generator in the category adversarial model for training, and through transfer learning of each category (multi-task recognition transfer), the trained category adversarial models corresponding to each category are obtained, and each category adversarial model is integrated to obtain a multi-category classification decision model, i.e., a multi-category category adversarial model. In the process of training each category adversarial model, the generated semi-supervised samples are fed back to the second generator to adjust the generated image features. The computer device obtains random variable data, and inputs the noise data and the random variable data into the second generator in each trained category adversarial model to obtain category feature images corresponding to each category; and constructs a defect detection model. The computer device trains the defect detection model based on multiple category feature images to obtain a trained defect detection model. In addition, the detection process is guided based on the loss function, so that the results of the defect detection model are more accurate.

[0126] In this embodiment, the features of the generated image can be greatly enhanced by the feature enhancement model, which is helpful to realize the extraction of features with strong separability and migration adaptability of micron-level defects, and the category adversarial model corresponding to each category is trained based on the feature-enhanced generated image to obtain each trained category adversarial model. Based on the each category adversarial model, images of different categories for defect detection can be further increased, thereby avoiding overfitting. In which, scene sample noise data carrying multiple task labels is used as the input variable of the second generator, which can reduce the damage to the overall information, that is, after the two-level generative adversarial network and transfer learning of the feature enhancement model for feature extraction and the multi-category category adversarial model for each category, a defect detection model is constructed. In which, the two levels of the feature enhancement model for feature extraction and the multi-category category adversarial model for each category are connected, and at the same time, a multi-level and multi-task defect detection model is constructed through the scene sample noise data (that is, multi-task source) and transfer learning of each category (multi-task recognition migration). The defect detection model is then trained based on multiple category feature images to obtain a trained defect detection model. This can form a low-rank space shared by multiple categories (multiple tasks), thereby improving the generalization ability of the defect detection model. Therefore, defect detection is performed on the images to be inspected in the target domain based on the trained defect detection model, which can greatly improve the accuracy and recall rate of micron-level defect detection.

[0127] It should be understood that although Figures 2 to 4 , Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figures 2 to 4 , Figure 6 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0128] In one embodiment, Fig. 9 As shown, a training device for an image defect detection model is provided, comprising: an acquisition module 902, a first generation module 904, a clustering module 906, a first training module 908, a second generation module 910, and a second training module 912, wherein:

[0129] The acquisition module 902 is used to acquire the first generator in the trained feature enhancement model; wherein the feature enhancement model is pre-trained based on the source domain sample image and noise data, and then re-trained based on the target domain sample image and noise data.

[0130] The first generating module 904 is used to input the noise data into the first generator to obtain a plurality of unlabeled generated images.

[0131] The clustering module 906 is used to perform clustering processing on the generated images to obtain the categories corresponding to each generated image.

[0132] The first training module 908 is used to construct a category adversarial model corresponding to each category, and train the category adversarial model of the corresponding category based on the generated images of the same category to obtain the trained category adversarial model corresponding to each category.

[0133] The second generation module 910 is used to obtain random variable data, and superimpose the noise data and the random variable data, and input them into the second generator in each category adversarial model after training, so as to obtain category feature images corresponding to each category.

[0134] The second training 912 is used to build a defect detection model and train the defect detection model based on multiple category feature images to obtain a trained defect detection model. The trained defect detection model is used to perform defect detection on the image to be detected in the target domain.

[0135] In one embodiment, the acquisition module 902 is used to obtain a source domain sample image and noise data; perform feature extraction on the current source domain sample image to obtain the current source domain feature map; based on the noise data and the source domain sample hidden features obtained in the previous training iteration, generate the current source domain training generated image through the first generator; input the current source domain feature map and the current source domain training generated image into the first discriminator to obtain a first discrimination result and the current source domain sample hidden features; adjust the model parameters of the feature enhancement model based on the first discrimination result, enter the next loop iteration, and use the current source domain sample hidden features as the source domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, return to the current source domain sample image to perform feature extraction, and continue to execute the step of obtaining the current source domain feature map until the first training stop condition is reached to obtain a pre-trained feature enhancement model.

[0136] In one embodiment, the acquisition module 902 is used to obtain a target domain sample image and noise data; perform feature extraction on the target domain sample image of the current time to obtain the target domain feature map of the current time; generate the target domain training generated image of the current time based on the noise data and the target domain sample hidden features obtained in the previous training iteration and through the first generator obtained by pre-training; input the target domain feature map of the current time and the target domain training generated image of the current time into the first discriminator obtained by pre-training to obtain a second discrimination result and the target domain sample hidden features of the current time; adjust the model parameters of the feature enhancement model obtained by pre-training based on the second discrimination result, enter the next loop iteration, and use the target domain sample hidden features of the current time as the target domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, return to the step of performing feature extraction on the target domain sample image of the current time, and obtain the target domain feature map of the current time, and continue to execute until the second training stop condition is reached to stop and obtain a trained feature enhancement model.

[0137] In one embodiment, the acquisition module 902 is used to extract the target area in the current target domain sample image to obtain a target area image; and perform feature extraction on the target area image to obtain a current target domain feature map.

[0138] In one embodiment, the first training module 908 is used to obtain a generated image of the same category, and perform feature extraction on the generated image of the current time to obtain the feature extraction map of the current time; obtain scene sample noise data and random variable data, and generate the feature image of the training category of the current time through the second generator based on the scene noise data, random variable data and the semi-supervised sample obtained in the previous training iteration; input the feature extraction map of the current time and the feature image of the training category of the current time into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time; adjust the model parameters of the category adversarial model based on the third discrimination result, enter the next loop iteration, and use the semi-supervised sample of the current time as the semi-supervised sample obtained in the previous training iteration corresponding to the next training iteration, return to the step of performing feature extraction on the generated image of the current time, and obtain the feature extraction map of the current time, and continue to execute until the third training stop condition is reached to stop, and obtain a trained category adversarial model corresponding to the corresponding category.

[0139] In one embodiment, the first training module 908 is used to perform scene pre-determination on a plurality of feature-generated images, determine a target scene corresponding to each of the feature-generated images, and obtain scene sample noise data corresponding to the target scene.

[0140] In one embodiment, the second training module 908 is used to input the feature extraction image of the current time and the training category feature image of the current time into the second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; when the intermediate semi-supervised sample meets a preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample of the current time.

[0141] For the specific definition of the training device of the image defect detection model, please refer to the definition of the training method of the image defect detection model above, which will not be repeated here. The various modules in the above-mentioned training device of the image defect detection model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0142] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training data of the image defect detection model. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for an image defect detection model is implemented.

[0143] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0146] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0147] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A training method for an image defect detection model, characterized in that: The method comprises: Obtaining a first generator in a trained feature enhancement model; wherein the feature enhancement model is pre-trained based on source domain sample images and noise data, and then re-trained based on target domain sample images and noise data; The feature enhancement model includes a first generator and a first discriminator; the pre-training step of the feature enhancement model includes: obtaining a source domain sample image and noise data; performing feature extraction on the current source domain sample image to obtain the current source domain feature map; generating the current source domain training generated image based on the noise data and the source domain sample hidden features obtained in the previous training iteration through the first generator; inputting the current source domain feature map and the current source domain training generated image into the first discriminator to obtain a first discrimination result and the current source domain sample hidden features; adjusting the model parameters of the feature enhancement model based on the first discrimination result, entering the next loop iteration, and using the current source domain sample hidden features as the source domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the current source domain sample image to obtain the current source domain feature map, and continuing to execute until the first training stop condition is reached, thereby obtaining a pre-trained feature enhancement model; The retraining step of the feature enhancement model includes: obtaining a target domain sample image and noise data; performing feature extraction on the target domain sample image of the current time to obtain the target domain feature map of the current time; generating the target domain training generated image of the current time based on the noise data and the target domain sample latent features obtained in the previous training iteration and through the first generator obtained by pre-training; inputting the target domain feature map of the current time and the target domain training generated image of the current time into the first discriminator obtained by pre-training to obtain a second discrimination result and the target domain sample latent features of the current time; adjusting the model parameters of the feature enhancement model obtained by pre-training based on the second discrimination result, entering the next loop iteration, and using the target domain sample latent features of the current time as the target domain sample latent features obtained in the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the target domain sample image of the current time to obtain the target domain feature map of the current time and continuing to execute until the second training stop condition is reached, thereby obtaining a trained feature enhancement model; Inputting the noise data into the first generator to obtain a plurality of unlabeled generated images; Performing clustering processing on the generated images to obtain categories corresponding to the generated images; Constructing a category adversarial model corresponding to each category respectively, wherein the category adversarial model includes a second generator and a second discriminator; Obtain the generated images of the same category, and perform feature extraction on the generated images of the current time to obtain the feature extraction graph of the current time; Obtaining scene sample noise data and random variable data, and generating a training category feature image for the current time through the second generator based on the scene sample noise data, the random variable data, and the semi-supervised sample obtained in the previous training iteration; Inputting the feature extraction image of the current time and the training category feature image of the current time into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time; Adjusting the model parameters of the class adversarial model based on the third discrimination result, entering the next loop iteration, and using the current semi-supervised sample as the semi-supervised sample obtained in the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the current generated image to obtain the current feature extraction graph, and continuing until the third training stop condition is reached, thereby obtaining a trained class adversarial model corresponding to the corresponding class; Obtain random variable data, and superimpose the noise data and the random variable data, inputting them into the second generator of each trained category adversarial model to obtain category feature images corresponding to each category; A defect detection model is constructed and trained based on multiple category feature images to obtain a trained defect detection model, which is used to perform defect detection on images to be detected in a target domain.

2. The method according to claim 1, characterized in that The feature extraction of the target domain sample image is performed to obtain the target domain feature map, including: Extract the target area in the target domain sample image to obtain the target area image; Feature extraction is performed on the target area image to obtain the target domain feature map of the current time.

3. The method according to claim 1, characterized in that The acquiring of scene sample noise data includes: Performing scene pre-judgment on a plurality of feature-generated images to determine a target scene corresponding to each of the feature-generated images; Acquire scene sample noise data corresponding to the target scene.

4. The method according to claim 1, wherein The step of inputting the feature extraction graph and the training category feature image into the second discriminator to obtain a third discrimination result and the semi-supervised sample includes: Inputting the feature extraction image of the current time and the training category feature image of the current time into the second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; When the intermediate semi-supervised sample meets the preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample for that time.

5. A training device for an image defect detection model, characterized in that: The device comprises: An acquisition module is used to acquire the first generator in the trained feature enhancement model; wherein, the feature enhancement model is pre-trained based on the source domain sample image and noise data, and then re-trained based on the target domain sample image and noise data; the feature enhancement model includes a first generator and a first discriminator; the acquisition module is also used to acquire the source domain sample image and noise data; feature extraction is performed on the current source domain sample image to obtain the current source domain feature map; based on the noise data and the source domain sample hidden features obtained in the previous training iteration, the current source domain training generated image is generated through the first generator; the current source domain feature map and the current source domain training generated image are input into the first discriminator to obtain the first discrimination result and the current source domain sample hidden features; based on the first discrimination result, the model parameters of the feature enhancement model are adjusted to enter the next loop iteration, and the current source domain sample hidden features are used as the source domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, and the feature extraction of the current source domain sample image is returned to obtain the current source domain features The steps of the figure continue to be executed until the first training stop condition is reached, and a pre-trained feature enhancement model is obtained; the acquisition module is also used to obtain a target domain sample image and noise data; feature extraction is performed on the target domain sample image of the current time to obtain the target domain feature map of the current time; based on the noise data and the target domain sample hidden features obtained in the previous training iteration, the target domain training generated image of the current time is generated by the first generator obtained by pre-training; the target domain feature map of the current time and the target domain training generated image of the current time are input into the first discriminator obtained by pre-training to obtain a second discrimination result and the target domain sample hidden features of the current time; based on the second discrimination result, the model parameters of the feature enhancement model obtained by pre-training are adjusted, and the next loop iteration is entered, and the target domain sample hidden features of the current time are used as the target domain sample hidden features obtained in the previous training iteration corresponding to the next training iteration, and the step of performing feature extraction on the target domain sample image of the current time to obtain the target domain feature map of the current time is returned to continue to be executed until the second training stop condition is reached, and a trained feature enhancement model is obtained; a first generating module, configured to input the noise data into the first generator to obtain a plurality of unlabeled generated images; A clustering module, configured to perform clustering processing on the generated images to obtain categories corresponding to the generated images; A first training module is used to construct a category adversarial model corresponding to each category, wherein the category adversarial model includes a second generator and a second discriminator; The first training module is used to obtain generated images of the same category and perform feature extraction on the generated images to obtain a feature extraction graph of the current time; Obtaining scene sample noise data and random variable data, and generating a training category feature image for the current time through the second generator based on the scene sample noise data, the random variable data, and the semi-supervised sample obtained in the previous training iteration; Inputting the feature extraction image of the current time and the training category feature image of the current time into the second discriminator to obtain a third discrimination result and the semi-supervised sample of the current time; Adjusting the model parameters of the class adversarial model based on the third discrimination result, entering the next loop iteration, and using the current semi-supervised sample as the semi-supervised sample obtained in the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the current generated image to obtain the current feature extraction graph, and continuing until the third training stop condition is reached, thereby obtaining a trained class adversarial model corresponding to the corresponding class; A second generation module is configured to obtain random variable data, and superimpose the noise data and the random variable data, and input the superimposed noise data and the random variable data into the second generator of each trained adversarial model to obtain a category feature image corresponding to each category; The second training module is used to build a defect detection model and train the defect detection model based on multiple category feature images to obtain a trained defect detection model. The trained defect detection model is used to perform defect detection on the image to be detected in the target domain.

6. The device according to claim 5, characterized in that The acquisition module is used to extract the target area in the current target domain sample image to obtain the target area image; and perform feature extraction on the target area image to obtain the current target domain feature map.

7. The device according to claim 5, characterized in that The first training module is used to perform scene pre-judgment on multiple feature-generated images, determine the target scene corresponding to each of the feature-generated images, and obtain scene sample noise data corresponding to the target scene.

8. The device according to claim 5, characterized in that The first training module is used to input the feature extraction image of the current time and the training category feature image of the current time into the second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; when the intermediate semi-supervised sample meets the preset threshold range, the intermediate semi-supervised sample is used as the semi-supervised sample of the current time.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.