Defect detection method and apparatus, and model transfer method and apparatus

Abstract

A defect detection method and a model transfer method. The model transfer method comprises: acquiring a source-domain image and a corresponding annotation, and a baseline model obtained from a source domain; acquiring target-domain images, which are fully annotated, partially annotated, or unannotated; inputting the annotation of the source-domain image into a denoiser to obtain a denoised annotation; inputting the unannotated target-domain images into an initial baseline model to obtain second target-domain prediction results, performing data augmentation on the unannotated target-domain images to obtain second target-domain augmented images, and using the second target-domain prediction results as pseudo-annotations of the second target-domain augmented images; and using the source-domain image and the corresponding denoised annotation, the annotated target-domain images and the corresponding annotations, and / or the second target-domain augmented images and the corresponding pseudo-annotations to train the baseline model, so as to obtain a transfer model. The method can complete training by using a small number of annotated target-domain images, thereby solving the problem of model transfer performance being poor in the case of insufficient target-domain images.

Description

Defect detection method and device, model migration method and device TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a defect detection method and device, a model migration method and device. BACKGROUND

[0002] In the field of machine vision, detection techniques based on deep learning are widely used. When using deep learning technology to detect defects in industrial images, it is necessary to collect sufficient defect samples of various products produced and manually label them accordingly, and train the model with these defect samples to achieve the purpose of detection. However, data collection is usually difficult, and labeling costs are high (requiring a large amount of manpower), resulting in high data costs for production enterprises. In real industrial production scenarios, for the same product or similar products, due to differences in imaging equipment, shooting environment, lighting conditions, or slight changes in the product itself, the trained model cannot be migrated or quickly migrated, which is a pain point for production and manufacturing enterprises using deep detection technology, mainly in the following two aspects:

[0003] (1) Same product: In actual industrial production, the same product can be produced by different production lines, but due to changes in lighting, imaging environment, etc., there are slight differences in the data distribution of product images on different production lines (i.e., the "domain shift" problem), and the data cannot be generalized, forming a strict correspondence mode of "production line-data-model", for example: the data collected on A production line and the model trained cannot be migrated to B production line or other production lines, especially when the enterprise expands production, the data and advanced models obtained by the initial production cannot be migrated or quickly migrated to the new production line, resulting in waste of resources and increase in production costs;

[0004] (2) Similar products: mainly reflected in that even if the products produced have slight differences, data and model migration or quick migration cannot be performed, for example, the surface color of the product is slightly different, but the rest is the same, but there is a "domain shift" problem between the image data of different color products, and the data and model cannot achieve good migration performance. In actual production, there are usually multiple similar products produced to meet the needs of consumers, but at present, in the production process of similar products, the data cannot be aligned, resulting in that the data cannot be fully utilized and the model migration performance is poor. SUMMARY

[0005] The technical problem solved by the present application is that the machine vision model migration capability is poor and cannot adapt to different image data.

[0006] According to a first aspect, in one embodiment, a defect detection method is provided, comprising:

[0007] obtaining a to-be-detected image of a target domain;

[0008] inputting the to-be-detected image into a pre-trained defect detection model for defect detection to obtain a defect detection result of the to-be-detected image;

[0009] The defect detection model is obtained by the following method:

[0010] obtaining source domain images and corresponding defect labels, and obtaining a baseline model trained by the source domain images and the corresponding defect labels;

[0011] obtaining target domain images, wherein the source domain and the target domain are different imaging scenes, and the different imaging scenes refer to different measured objects and / or different imaging conditions, the different measured objects can be the same measured object of different sources (for example, the same product produced by different production lines) or similar measured objects (for example, measured objects that are mostly the same except for individual parameters); part or all of the target domain images have corresponding defect labels, or none of the target domain images have corresponding defect labels, if part or all of the target domain images have corresponding defect labels, obtaining the defect labels of the target domain images with corresponding defect labels;

[0012] inputting the defect labels of the source domain images into a learnable denoiser to obtain denoised labels;

[0013] if there are target domain images without corresponding defect labels, inputting the target domain images without corresponding defect labels into an initial baseline model to obtain a second target domain prediction result, and performing a data augmentation operation on the target domain images without corresponding defect labels to obtain second target domain augmented images, and using the second target domain prediction result as pseudo labels corresponding to the second target domain augmented images;

[0014] training the baseline model and the denoiser by using the source domain images and the corresponding denoised labels, the target domain images with corresponding defect labels and the corresponding defect labels, and / or the second target domain augmented images and the corresponding pseudo labels; the defect detection model is a baseline model obtained at the end of training; and the initial baseline model is a baseline model whose parameters are not updated before training.

[0015] In some embodiments, the training of the baseline model and the denoiser by using the source domain images and the corresponding denoised labels, the target domain images with corresponding defect labels and the corresponding defect labels, and / or the second target domain augmented images and the corresponding pseudo labels comprises:

[0016] inputting the source domain images into the baseline model to obtain source domain prediction results;

[0017] If there is a target domain image with corresponding defect label, the target domain image is input into the baseline model to obtain a first target domain prediction result;

[0018] If there is a target domain image without corresponding defect label, the second target domain augmented image obtained by performing a data augmentation operation on the target domain image without corresponding defect label is input into the baseline model to obtain a second target domain augmented prediction result;

[0019] The baseline model and the denoiser are trained according to a total loss function Loss;

[0020] The total loss function Loss is determined by a first loss function L1, a second loss function L2 and / or a third loss function L3, the first loss function L1 is determined according to a difference between the source domain prediction result and the corresponding denoising label, the second loss function L2 is determined at least according to a difference between the first target domain prediction result and the corresponding defect label, and the third loss function L3 is determined according to a difference between the second target domain augmented prediction result and the corresponding pseudo label.

[0021] According to a second aspect, an embodiment provides a model migration method, characterized by being used for migrating a baseline model obtained from a source domain to a target domain to obtain a migration model, and the model migration method comprises:

[0022] Obtaining source domain images and corresponding labels, and obtaining the baseline model trained by the source domain images and the corresponding labels;

[0023] Obtaining target domain images, some or all of the target domain images have corresponding labels, or none of the target domain images has corresponding labels, if some or all of the target domain images have corresponding labels, obtaining labels of the target domain images with corresponding labels;

[0024] Inputting the labels of the source domain images into a learnable denoiser to obtain denoising labels;

[0025] If there is a target domain image without corresponding label, inputting the target domain image into an initial baseline model to obtain a second target domain prediction result, performing a data augmentation operation on the target domain image without corresponding label to obtain a second target domain augmented image, and taking the second target domain prediction result as a pseudo label corresponding to the second target domain augmented image;

[0026] The baseline model and the denoiser are trained using the source domain image and the corresponding denoising label, and the target domain image with the corresponding label and / or the second target domain augmented image and the corresponding pseudo label; the transfer model is the baseline model obtained at the end of training; and the initial baseline model is the baseline model obtained before training.

[0027] According to a third aspect, an embodiment of a defect detection device is provided, comprising:

[0028] An image to be detected acquisition module is configured to acquire an image to be detected in a target domain;

[0029] A detection module is connected to the image to be detected acquisition module and configured to input the image to be detected into a pre-trained defect detection model for defect detection to obtain a defect detection result of the image to be detected;

[0030] A model training module is connected to the detection module and configured to train the defect detection model, comprising:

[0031] A source domain data acquisition unit is configured to acquire a source domain image and a corresponding defect label, and acquire a baseline model trained by the source domain image and the corresponding defect label;

[0032] A target domain data acquisition unit is configured to acquire a target domain image, wherein the source domain and the target domain are different imaging scenes, and the different imaging scenes refer to different measured objects and / or different imaging conditions; part or all of the target domain images have corresponding defect labels, or none of the target domain images have corresponding defect labels; if part or all of the target domain images have corresponding defect labels, the defect labels of the target domain images with corresponding defect labels are acquired;

[0033] A denoising unit is connected to the source domain data acquisition unit and configured to input the defect label of the source domain image into a learnable denoiser to obtain a denoising label;

[0034] A pseudo label generation unit is connected to the target domain data acquisition unit and configured to, if there is a target domain image without a corresponding defect label, input the target domain image into an initial baseline model to obtain a second target domain prediction result, perform a data augmentation operation on the target domain image without the corresponding defect label to obtain a second target domain augmented image, and take the second target domain prediction result as a pseudo label corresponding to the second target domain augmented image;

[0035] a training unit, connected with the source domain data acquisition unit, the target domain data acquisition unit, the denoising unit and the pseudo-label generation unit, configured to train the baseline model and the denoiser by using the source domain images and corresponding denoising labels, and the target domain images with corresponding labels and corresponding labels and / or the second target domain augmented images and corresponding pseudo-labels;

[0036] The defect detection model is a baseline model obtained at the end of training; and the initial baseline model is a baseline model with parameters not updated before training.

[0037] According to a fourth aspect, an embodiment provides a model migration device for migrating a baseline model obtained from a source domain to a target domain to obtain a migration model, the model migration device comprising:

[0038] a source domain data acquisition unit configured to acquire source domain images and corresponding labels, and acquire the baseline model trained by the source domain images and corresponding labels;

[0039] a target domain data acquisition unit configured to acquire target domain images, some or all of the target domain images having corresponding labels, or none of the target domain images having corresponding labels, if some or all of the target domain images have corresponding labels, acquiring labels of the target domain images with corresponding labels;

[0040] a denoising unit connected with the source domain data acquisition unit, configured to input the labels of the source domain images into a learnable denoiser to obtain denoising labels;

[0041] a pseudo-label generation unit connected with the target domain data acquisition unit, configured to, if there are target domain images without corresponding labels, input the target domain images into an initial baseline model to obtain second target domain prediction results, and perform a data augmentation operation on the target domain images without corresponding labels to obtain second target domain augmented images, and take the second target domain prediction results as corresponding pseudo-labels of the second target domain augmented images;

[0042] a training unit, connected with the source domain data acquisition unit, the target domain data acquisition unit, the denoising unit and the pseudo-label generation unit, configured to train the baseline model and the denoiser by using the source domain images and corresponding denoising labels, and the target domain images with corresponding labels and corresponding labels and / or the second target domain augmented images and corresponding pseudo-labels;

[0043] The migration model is a baseline model obtained at the end of training; and the initial baseline model is a baseline model with parameters not updated before training.

[0044] According to a fifth aspect, a computer readable storage medium is provided in an embodiment, on which a program is stored, the program being executable by a processor to implement the defect detection method of any of the above embodiments or the model migration method of any of the above embodiments.

[0045] According to a sixth aspect, a computer program product is provided in an embodiment, comprising a computer program and / or instructions, which, when executed by a processor, implement the defect detection method of any of the above embodiments or the model migration method of any of the above embodiments.

[0046] The technical solution of the present application combines source domain images and target domain images to perform migration training of a baseline model from a source domain to a target domain, wherein the baseline model is trained in the source domain; the source domain images have corresponding labels, and the labels of the source domain images are input into a learnable denoiser to obtain denoised labels; the target domain images can all have corresponding labels or some have corresponding labels or all do not have corresponding labels; for the target domain images without labels, pseudo labels are generated by using the baseline model; the baseline model and the denoiser are trained by comprehensively using the source domain images and the corresponding denoised labels, and the target domain images and the corresponding labels / pseudo labels, so as to realize migration of the baseline model to the target domain, complete training with a small amount of labeled target domain images, realize rapid migration of the model, and solve the problem of poor model migration performance in the case of insufficient target domain images. On the other hand, the present application regards the domain adaptation process from the source domain to the target domain as a "denoising" problem to be solved, models the source domain images as noise samples of the target domain images, and "denoises" the labels of the source domain images by designing a label denoiser, so that the source domain images match the target domain images. Moreover, the technical solution of the present application generates corresponding pseudo labels for the unlabeled target domain images, can adapt to the case that all target domain images are labeled, some target domain images are labeled, and all target domain images are unlabeled, can adaptively use the target domain images and the corresponding labels / pseudo labels for model migration training according to the label situation of the input target domain images, and improves the flexibility and widens the use scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0047] FIG. 1 is a schematic diagram of the overall processing flow of the model migration method according to an embodiment;

[0048] FIG. 2 is a flowchart of the model migration method according to an embodiment;

[0049] FIG. 3 is a flowchart of training the baseline model and the denoiser by using the source domain images and the corresponding denoised labels, and the target domain images with corresponding labels and the corresponding labels and / or the second target domain expansion images and the corresponding pseudo labels according to an embodiment;

[0050] FIG. 4 is a flowchart of the model migration method according to another embodiment;

[0051] FIG. 5 is a flowchart of training a baseline model and a denoiser using source domain images and corresponding denoising labels, target domain images with corresponding defect labels and corresponding defect labels and / or second target domain augmented images and corresponding pseudo labels in an embodiment;

[0052] FIG. 6 is a flowchart of a defect detection method in an embodiment;

[0053] FIG. 7 is a schematic diagram of a result of defect detection on a to-be-detected image using a defect detection model in an embodiment;

[0054] FIG. 8 is a schematic diagram of a structure of a model migration device in an embodiment;

[0055] FIG. 9 is a schematic diagram of a structure of a defect detection device in an embodiment. DETAILED DESCRIPTION

[0056] The application will be further described in details through specific embodiments and the accompanying drawings. In different embodiments, similar elements are associated with similar element labels. In the following embodiments, many details are described in order to make the application better understood. However, those skilled in the art can easily recognize that some features can be omitted or replaced by other elements, materials or methods in different cases. In some cases, some operations related to the application are not shown or described in the specification in order to avoid the core part of the application being overwhelmed by too much description, and it is not necessary to describe these related operations in detail for those skilled in the art according to the description in the specification and general technical knowledge in the art.

[0057] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate way to form various embodiments. At the same time, the steps or actions in the method description can also be sequentially adjusted or adjusted in a manner that is obvious to those skilled in the art. Therefore, the order in the specification and the drawings is only for the purpose of clearly describing a certain embodiment, and does not mean that it is the necessary order, unless otherwise stated that a certain order must be followed.

[0058] The serial numbers of components in this paper, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any order or technical meaning. Unless otherwise specified, "connection" and "coupling" in this application include direct and indirect connection (coupling).

[0059] The two problems described in the background are the pain points encountered by current production enterprises in the use of deep learning technology in industrial detection. In order to enable the trained model to be used on data with "domain shift", the transfer learning technology is usually used to migrate the model to the target domain for use.

[0060] The purpose of transfer learning is to enable the knowledge or patterns learned from the source domain to be effectively transferred to the target domain to improve the performance of the task in the target domain. The source domain and the target domain are two key concepts in transfer learning, which respectively refer to the dataset or field on which the model is pre-trained and the new dataset or field to which the model will be applied. In the source domain, there are usually a large number of labeled samples available for learning, which are used to train and build the initial model, also known as the baseline model. The source domain can be a task, a field or a dataset, depending on the specific application scenario. For example, in the image classification task, the ImageNet dataset can be regarded as a source domain, which contains a large number of images and corresponding labels, and can be used to train a neural network. In the target domain, there are usually fewer labeled samples available for learning, so the model needs to use transfer learning to utilize the knowledge and features learned from the source domain in order to achieve good performance on the target task.

[0061] Most of the current detection systems commonly use transfer technology, which uses a self-supervised method for training. The specific processing method is to add the target domain data to the baseline model trained from the source domain for self-supervised learning, that is, to align the source domain data with the target domain data through high-dimensional feature space mapping to realize the transfer from the source domain to the target domain. The advantage is that the target domain data does not need to be labeled, but a sufficient number of target domain sample data is required. In the case where the target domain lacks sufficient data or it is difficult to obtain data, the false detection rate is high, the detection accuracy is low, the transfer effect is poor, and more training time is also required, which cannot achieve fast transfer.

[0062] In view of these problems, the present application proposes an adaptive model transfer method to realize fast transfer from the source domain to the target domain and solve the problem of poor model transfer performance in the case of insufficient target domain data. Compared with the current similar method, the adaptive model transfer method of the present application can complete training with a small amount of labeled target domain images to realize fast transfer of the model and solve the problem of poor model transfer performance in the case of insufficient target domain images. The performance of the transferred model is excellent. The model transfer method of the present application applied to actual industrial production can greatly reduce the production cost of enterprises; for the labeling situation of target domain images (all labeled, partially labeled and all unlabeled), the present application proposes an adaptive design to improve the flexibility of use, and users can use it according to actual needs to cope with different detection scenarios, thereby greatly improving the production efficiency of the production plant while reducing the cost.

[0063] The concept of the present application is introduced as follows. The present application adopts a deep learning technique to construct the entire training process. Referring to FIG. 1, the overall processing flow of the technical solution of the present application includes three parts: a baseline branch G, a target domain labeled data branch R, and a target domain unlabeled data branch S. The input of branch G is a source domain image, the input of branch R is a target domain image with labels, and the input of branch S is a target domain image without labels. Model M is a model to be migrated for performing a target task, which is a baseline model with advanced detection performance after training in the source domain, and then is migrated to the target domain to complete adaptive migration training of the model. M of different branches refers to the same model, which accepts corresponding inputs and outputs corresponding prediction results when processing flows of different branches are performed.

[0064] For the case that there is a certain difference between the data distribution of the source domain sample and the target domain sample, i.e., the problem of "domain shift", unlike the commonly used method, the present application models the source domain image as a noisy sample of the target domain image, and regards the domain adaptation process from the source domain to the target domain as a "denoising" problem to be solved.

[0065] The inputs of the three parts are different, and the functions and emphases are also different. The specific design idea is that, for the baseline branch G, a corresponding label denoiser is designed from the perspective of the label of the source domain image to remove the label "noise" of the source domain image, so that the source domain image is matched with the target domain image. The target domain labeled data branch R and the target domain unlabeled data branch S are adaptive dynamic selection branches, which support a supervised mode with all images labeled, a semi-supervised mode with part of the images labeled, and an unsupervised mode with all images unlabeled. Specifically, there are three cases: first, if all the target domain images input have corresponding labels, then all the target domain images perform the processing flow of the target domain labeled data branch R, and the processing of the target domain unlabeled data branch S is not performed; second, if part of the target domain images input have labels, then the target domain images with labels perform the processing flow of the target domain labeled data branch R, and the target domain images without labels perform the processing flow of the target domain unlabeled data branch S; third, if all the target domain images input do not have corresponding labels, then all the target domain images perform the processing flow of the target domain unlabeled data branch S, and the processing of the target domain labeled data branch R is not performed.

[0066] In use, the user can flexibly select the training mode according to the task requirement. For example, if the target domain image collection quantity is relatively sufficient, but the labeling workload is large or the task target is fast migration of a small sample, the unsupervised mode of the present application can be used for pre-training to obtain an initial model that can be used as auxiliary labeling. When the model training is completed and verified by manual checking, most of the inference results output by the initial model can be directly used as labeling to reduce the labeling workload. For the target domain images with poor inference effect, corresponding labeling can be manually marked, and the corresponding sample data is provided to the model training as important sample data or difficult sample data to realize fast migration. Meanwhile, the obtained initial model can also be used as a pre-training model for subsequent training to greatly reduce the training time. If the target domain image is less, a part or all of the data can be manually marked with corresponding labeling and input into the model to complete the migration training of the model through the supervised mode or the semi-supervised mode. The model converges fast, and the performance of the migrated model on the target task is excellent. In order to ensure the performance of the model on the target task after migration, preferably, a small amount of labeled target domain images are used for training, that is, the semi-supervised mode is used.

[0067] The present application designs corresponding loss functions for the three parts of the baseline branch G, the target domain labeled data branch R and the target domain unlabeled data branch S, respectively, for training. The key is the design of the "denoising" of the labeling of the source domain image, the adaptive combination design of the R branch and the S branch and the design of the corresponding loss function.

[0068] The model migration method of the present application will be described below in combination with FIG. 1. The model migration method is used to migrate the baseline model obtained from the source domain to the target domain to obtain a migration model. Please refer to FIG. 2. The model migration method in some embodiments includes steps 110-150, which will be described below.

[0069] Step 110: Obtain the source domain image x and the corresponding labeling y, and obtain the baseline model M trained by the source domain image x and the corresponding labeling y.

[0070] Step 120: Obtain the target domain image.

[0071] Part or all of the target domain images have corresponding labeling, or none of them has corresponding labeling. If part or all of the target domain images have corresponding labeling, the labeling of the target domain image with corresponding labeling is obtained. Whether the target domain image is labeled and the amount of labeling is determined by the user according to the actual task requirement. In FIG. 1, x1 represents the labeled target domain image, y1 represents the labeling thereof, and x2 represents the unlabeled target domain image.

[0072] Step 130: input the labeling y of the source domain image into the learnable denoiser TC to obtain the denoised labeling y tc .

[0073] This step is to "denoise" the label y, model the source domain image as a noisy sample of the target domain image, and thus convert the domain adaptation process from the source domain to the target domain into a "denoising" problem to solve.

[0074] As shown in FIG. 1, the source domain image x is input into the baseline model M to obtain the source domain prediction result p. In an embodiment, the denoiser TC is constructed according to the label y and the source domain prediction result p, and the specific expression is: y tc =(1-β)×onehot(y)+β×p,

[0075] Where onehot(y) represents performing one-hot encoding operation on y, and β is a learnable parameter.

[0076] The label y is processed by the denoiser TC to output the "clean" label y tc In essence, it is to prompt the denoiser TC to learn the adaptive smoothing (i.e., "denoising") of the original label.

[0077] If there is a target domain image without a corresponding label, step 140 is performed, otherwise step 150 is directly performed.

[0078] Step 140: input the target domain image x2 without a corresponding label into the initial baseline model M eval to obtain the second target domain prediction result p2, and perform a data augmentation operation on the target domain image x2 without a corresponding label to obtain the second target domain augmented image x 2A , and the second target domain prediction result p2 is used as the second target domain augmented image x 2A corresponding pseudo-label.

[0079] AUG in FIG. 1 refers to the data augmentation operation, which can be specifically flipping, contrast enhancement, and adding noise, etc. The initial baseline model M eval is a baseline model whose parameters are not updated before training, or can be understood as a baseline model that performs eval operation (i.e., the parameters remain fixed and are not updated) during the training process.

[0080] Step 150: use the source domain image x and the corresponding denoised label y tc , the target domain image x1 with the corresponding label and the corresponding label y1, and / or the second target domain augmented image x 2A and the corresponding pseudo-label p2 to train the baseline model M and the denoiser TC. The baseline model obtained at the end of the training is the migration model that completes the migration.

[0081] It can be understood that at this time, the source domain image x, the target domain image x1 with the corresponding label, and the second target domain augmented image x 2AWith the corresponding label information, they can be used for deep learning training. In the present application, the denoiser TC is learnable, and parameter updating is performed together in training.

[0082] For the target domain image x1 with corresponding label and the second target domain augmented image x 2A , if all target domain images have labels, only the target domain image x1 is used for training; if all target domain images do not have labels, all target domain images need to be processed in step 140, and finally only the obtained second target domain augmented image x 2A is used for training; if some target domain images have labels and some do not, the second target domain augmented image x 2A obtained by the target domain image x1 with labels and the target domain image x2 without labels is used for training together.

[0083] Please refer to FIG. 1 and FIG. 3, in some embodiments, step 150 includes the following steps:

[0084] Step 151: input the source domain image x into the baseline model M to obtain the source domain prediction result p;

[0085] If there is a target domain image x1 with corresponding label, step 152 is performed; if there is a target domain image x2 without corresponding label, step 153 is performed;

[0086] Step 152: input the target domain image x1 into the baseline model M to obtain the first target domain prediction result p1;

[0087] Step 153: input the second target domain augmented image x 2A obtained by performing data augmentation operation on the target domain image x2 into the baseline model M to obtain the second target domain augmented prediction result p 2A ;

[0088] Step 154: train the baseline model M and the denoiser TC according to the total loss function Loss.

[0089] It should be noted that steps 151-153 have no fixed execution order and can be executed in any order or simultaneously.

[0090] The total loss function Loss is determined by the first loss function L1, and the second loss function L2 and / or the third loss function L3. Please refer to FIG. 1, the first loss function L1 is determined according to the difference between the source domain prediction result p and the corresponding denoising label y tc , the second loss function L2 is determined at least according to the difference between the first target domain prediction result p1 and the corresponding label y1, and the third loss function L3 is determined according to the difference between the second target domain augmented prediction result p 2AThe difference between the corresponding pseudo-label p2 and the target label p2 is determined.

[0091] The first loss function L1, the second loss function L2, and the third loss function L3 correspond to the training of the G branch, R branch, and S branch, respectively, and they can be designed according to the training objectives of their respective branches. The training objectives or functions of each branch are explained below.

[0092] In the domain-adaptive model transfer training, the input to the G branch is entirely source domain images with corresponding annotations. This design serves two purposes: first, it fully utilizes the source domain images, transferring the baseline model M's recognition capabilities for source domain images to the target domain images; second, addressing the "domain offset" problem—a data distribution discrepancy between source and target domain images—the source domain images are modeled as noise samples of the target domain images. A denoising TC is designed to "de-denoise" the annotations of the source domain images, ensuring a match between the source and target domain images. This significantly reduces the model's false detection rate for the target domain images and improves detection performance.

[0093] The input to the R branch is a target domain image x1 with corresponding annotations, which is used to control the training and fitting of the baseline model M to the target domain image, learning the ability to perform corresponding tasks on the target domain image.

[0094] The input to the S branch is two unlabeled target domain images. The data-augmented target domain images are used as training samples, and the initial baseline model M... eval The prediction of the target domain image x2 without corresponding annotations is used as a pseudo-annotation to guide the baseline model M to fit the enhanced image of the target domain image x2. In essence, this is a self-supervised training method, which allows the baseline model M to complete efficient training and improve detection accuracy without labeled data.

[0095] In some embodiments, as shown in FIG1, if a target domain image x1 with corresponding annotations exists, the same data augmentation operation is also performed on the target domain image x1 with corresponding annotations and the corresponding annotations y1 to obtain a first target domain augmented image x. 1A and the corresponding first extended label y 1A Expand the first target domain to include the image x. 1A Inputting the baseline model M yields the first target domain augmentation prediction result p. 1A The second loss function L2 further expands the prediction result p based on the first target domain. 1A and the corresponding first extended label y 1A The differences between them, the prediction results of the first target domain expansion p 1A The difference between the prediction result p1 and the corresponding first target domain prediction result is determined.

[0096] In the adaptive training process of model migration, the source domain images are sufficient, and the target domain images are rare. Therefore, the embodiment adds a data expansion branch to the R branch: performing the same data expansion operation on the target domain image x1 with the corresponding label y1 to obtain a first target domain expansion image x 1A and the corresponding first expansion label y 1A ; inputting the first target domain expansion image x 1A into the baseline model M to obtain a first target domain expansion prediction result p 1A .

[0097] In the embodiment, the first target domain expansion image x 1A and the corresponding first expansion label y 1A are introduced through data expansion for learning, aiming to train and fit the baseline model M on the target domain expansion image; at the same time, in order to enhance the attention of the baseline model M on the target domain image, the first target domain expansion prediction result p 1A and the corresponding first target domain prediction result p1 are compared to construct a second loss function L2, aiming to train the baseline model M consistently and guide the baseline model M to output a more consistent prediction result on the target domain image x1 and the first target domain expansion image x 1A .

[0098] The purpose of expanding the target domain image is twofold: first, the target domain image is difficult to collect and the number is small, and the number of target domain images can be increased by using data enhancement means; second, through consistent training of the original target domain image and the expanded target domain image, the recognition ability of the baseline model M on the target domain image is enhanced, and essentially, the pixel consistency between the prediction of the baseline model M on the target domain image and the prediction of the baseline model M on the corresponding perturbed image (target domain expansion image) of the target domain image is enhanced, and the detection performance and generalization performance of the model are improved.

[0099] For the total loss function Loss, in some embodiments, according to the labeling condition of the target domain image, there are three cases:

[0100] If all the target domain images have corresponding labels, the expression of the total loss function Loss is: Loss = δ1L1 + δ2L2,

[0101] If all the target domain images do not have corresponding labels, the expression of the total loss function Loss is: Loss = δ1L1 + δ3L3,

[0102] If some of the target domain images have corresponding labels and some do not, the expression of the total loss function Loss is: Loss = δ1L1 + δ2L2 + δ3L3,

[0103] wherein d1, d2 and d3 are preset weight coefficients. In one example, d1 = 1.0, d2 = 1.0, and d3 = 0.8.

[0104] The above model migration method can be applied to a defect detection task. By migrating the baseline model trained from the defect samples in the source domain to the target domain, a defect detection model capable of performing a defect detection task in the target domain is obtained. The difference is that in the defect detection task, the source domain image and the target domain image are specifically defect sample images in the source domain and the target domain, and the label is specifically a defect label.

[0105] wherein the source domain and the target domain are different imaging scenarios, and the different imaging scenarios refer to different measured objects (e.g., different products) and / or different imaging conditions, and the imaging conditions include imaging equipment, shooting parameters (e.g., exposure time, aperture size, etc.), illumination, and / or shooting environment (different backgrounds, different production lines, indoor and outdoor environments, etc.).

[0106] The defect label is used to mark the defects in the image, including the position, size and / or category of the defects, etc., and is usually represented by a defect label image with the same resolution as the image. Each pixel point of the defect label image represents whether the corresponding pixel point in the marked image belongs to a defect. If there are multiple different types of defects, it can also represent which type of defect the corresponding pixel point in the marked image belongs to.

[0107] Referring to FIG. 4, the model migration method applied to defect detection includes steps 210-250, which are described below.

[0108] Step 210: Obtain the source domain image x and the corresponding defect label y, and obtain the baseline model M trained from the source domain image x and the corresponding defect label y.

[0109] The baseline model M trained from the source domain image x and the corresponding defect label y can perform a defect detection task in the source domain and has good defect detection performance on the source domain image.

[0110] Step 220: Obtain the target domain image.

[0111] wherein part or all of the target domain images have corresponding defect labels, or none of the target domain images have corresponding defect labels. If part or all of the target domain images have corresponding defect labels, the defect labels of the target domain images with corresponding defect labels are obtained. Whether the target domain images are labeled and the amount of defect labels are determined by the user according to the actual task needs.

[0112] Step 230: input the defect label y of the source domain image x into the learnable denoiser TC to obtain the denoised label y tc .

[0113] In an embodiment, the denoiser TC is constructed according to the defect annotation y and the source domain prediction p, and the specific expression is: y tc = (1 - β) x onehot (y) + β x p,

[0114] Wherein, onehot (y) represents performing one-hot encoding operation on y, and β is a learnable parameter.

[0115] The defect annotation y is processed by the denoiser TC, and the "clean" annotation y tc is output, which essentially is to prompt the denoiser TC to learn adaptive smoothing (i.e., "denoising") of the original defect annotation.

[0116] If there is a target domain image without a corresponding defect annotation, step 240 is performed, otherwise step 250 is directly performed.

[0117] Step 240: input the target domain image x2 without a corresponding defect annotation into the initial baseline model M eval to obtain the second target domain prediction p2, and perform data augmentation operation on the target domain image x2 without a corresponding defect annotation to obtain the second target domain augmented image x 2A , and the second target domain augmented image x 2A is input into the denoiser TC to obtain the corresponding pseudo annotation p2.

[0118] Step 250: use the source domain image x and the corresponding denoising annotation y tc , and the target domain image x1 with a corresponding defect annotation and the corresponding defect annotation y1 and / or the second target domain augmented image x 2A and the corresponding pseudo annotation p2 to train the baseline model M and the denoiser TC. The baseline model obtained at the end of the training is the defect detection model of the target domain.

[0119] Please refer to FIG. 1 and FIG. 5, in some embodiments, step 250 includes the following steps:

[0120] Step 251: input the source domain image x into the baseline model M to obtain the source domain prediction p;

[0121] If there is a target domain image x1 with a corresponding defect annotation, step 152 is performed; if there is a target domain image x2 without a corresponding defect annotation, step 153 is performed;

[0122] Step 252: input the target domain image x1 into the baseline model M to obtain the first target domain prediction p1;

[0123] Step 253: input the second target domain augmented image x 2AThe baseline model M is input to obtain a second target domain augmented prediction result p 2A ;

[0124] Step 254: The baseline model M and the denoiser TC are trained according to the total loss function Loss.

[0125] It should be noted that steps 251-253 do not have a fixed execution order and can be executed in any order or simultaneously.

[0126] The total loss function Loss is determined by the first loss function L1, and the second loss function L2 and / or the third loss function L3. Here, the first loss function L1 is determined according to the difference between the source domain prediction result p and the corresponding denoising label y tc , the second loss function L2 is determined at least according to the difference between the first target domain prediction result p1 and the corresponding defect label y1, and the third loss function L3 is determined according to the difference between the second target domain augmented prediction result p 2A and the corresponding pseudo label p2. Each loss function can be designed according to the training target of each branch described above.

[0127] In some embodiments, the source domain prediction result p includes the probability that the source domain image x belongs to each classification category. There are multiple classification categories, and at least one of the multiple classification categories is used to represent a defect. For example, there are two classification categories, one representing no defect and one representing a defect; there can also be more classification categories, one representing no defect and the others representing various defect types.

[0128] Correspondingly, the defect label y of the source domain image x includes the actual probability that the source domain image x belongs to each classification category, and the denoised representation is also the probability of each classification category, i.e., the denoising label y tc includes the probability corresponding to each classification category obtained by processing the defect label y of the source domain image x by the denoiser TC. The expression of the first loss function L1 is:

[0129] Where n is the total number of classification categories, j is the serial number of the classification category, p j represents the probability that the source domain prediction result p includes the source domain image x belonging to the jth classification category, (y tc ) j represents the probability corresponding to the jth classification category in the denoising label y tc .

[0130] In some embodiments, if there is a target domain image x1 with a corresponding defect label, the same data augmentation operation is also performed on the target domain image x1 with the corresponding defect label and the corresponding defect label y1 to obtain the first target domain augmented image x 1Aand the corresponding first augmented label y 1A ; the first target domain augmented image x 1A The baseline model M is input to obtain the first target domain augmented prediction result p 1A The second loss function L2 is further determined according to the difference between the first target domain augmented prediction result p 1A and the corresponding first augmented label y 1A and the difference between the first target domain augmented prediction result p 1A and the corresponding first target domain prediction result p1.

[0131] Referring to FIG. 1, the second loss function L2 is composed of three parts: L R1 determined according to the difference between the first target domain prediction result p1 and the corresponding defect label y1, L 1A determined according to the difference between the first target domain augmented prediction result p 1A and the corresponding first augmented label y R1A , and a consistency loss function L 1A determined according to the difference between the first target domain augmented prediction result p R11A and the corresponding first target domain prediction result p1.

[0132] In some embodiments, the baseline model M is used for pixel-level defect detection on the target domain image x1, and the first target domain prediction result includes a first predicted multi-class score map p1, and the elements in the first predicted multi-class score map p1 are used to represent the probability that the corresponding pixel point in the target domain image x1 with the corresponding defect label belongs to each classification category. Similarly, at least one of the classification categories is used to represent a defect, for example, there are two classification categories, one represents the background and the other represents the defect; the classification categories can also have more, one represents the background and the others represent various defect types. The defect label of the target domain image x1 includes a target domain category label map y1, and the elements in the target domain category label map y1 are used to represent the actual classification category of the corresponding pixel point in the target domain image x1.

[0133] The first target domain augmented prediction result includes a first augmented predicted multi-class score map p 1A , and the elements in the first augmented predicted multi-class score map p 1A are used to represent the probability that the corresponding pixel point in the first target domain augmented image belongs to each classification category. The first augmented label includes a target domain augmented category label map y 1A obtained by a data augmentation operation on the target domain category label map y1.

[0134] The first predicted multi-class score map p1, the target domain category label map y1, the first augmented predicted multi-class score map p 1A and the target domain augmented category label map y1A The resolutions of the first prediction multi-class score map p1 and the first augmented prediction multi-class score map p2 are the same as the resolution of the target domain image x1.

[0135] L R1 The intersection over union loss function is adopted. Since the resolution of the industrial image is large and the defect area is relatively small, the intersection over union loss can better balance the loss contribution between different categories and avoid the loss of certain categories being too large to cause the model to fail to learn effectively. R1 The expression of the loss function is as follows:

[0136] Wherein, argmax(p1) represents the first class prediction map obtained by taking the classification category with the maximum probability in each element of the first prediction multi-class score map p1 as the final class prediction result. The number of intersection elements between the first class prediction map and the corresponding target domain class label map y1 is represented by argmax(p1)∩y1, and the number of union elements between the first class prediction map and the corresponding target domain class label map y1 is represented by argmax(p1)∪y1.

[0137] L R1A The intersection over union loss function is also adopted, and the expression is as follows:

[0138] Wherein, argmax(p 1A ) represents the first augmented class prediction map obtained by taking the classification category with the maximum probability in each element of the first augmented prediction multi-class score map p 1A , argmax(p 1A )∩y 1A represents the number of intersection elements between the first augmented class prediction map and the corresponding target domain augmented class label map y 1A , and argmax(p 1A )∪y 1A represents the number of union elements between the first augmented class prediction map and the corresponding target domain augmented class label map y 1A .

[0139] L R11A The expression of the loss function is as follows:

[0140] Wherein, m is the total number of elements, i is the serial number of the element, (p1) represents the one-dimensional flattened vector of p1, (p1) i is the i-th element of (p1), (p 1A ) represents the one-dimensional flattened vector of p 1A , (p 1A ) i is the one-dimensional flattened vector of p 1Athe i-th element of the one-dimensional flattened vector. The one-dimensional flattened vector refers to a vector formed by arranging the values of each channel of each element in the image in order. For example, for a 3x3x3 image, there are 9 elements and 3 channels, and the one-dimensional flattened vector is a 27-element vector formed by arranging the values of each channel of each element in order.

[0141] For the second loss function L2, in one embodiment, the expression is: L2 = a1L R1 + a2L R1A + a3L R11A ,

[0142] wherein a1, a2 and a3 are preset weight coefficients. In one example, a1 = 1.0, a2 = 1.0, and a3 = 0.6.

[0143] In some embodiments, the second target domain prediction result includes a second predicted multi-class score map p2, and the elements in the second predicted multi-class score map p2 are used to represent the probability that the corresponding pixel point in the target domain image x2 without a corresponding defect label belongs to each classification category. Similarly, at least one of the classification categories is used to represent a defect. Correspondingly, the second target domain augmented prediction result includes a second augmented prediction multi-class score map p 2A , and the elements in the second augmented prediction multi-class score map p 2A are used to represent the probability that the corresponding pixel point in the second target domain augmented image belongs to each classification category.

[0144] The resolution of the second predicted multi-class score map p2 and the resolution of the second augmented prediction multi-class score map p 2A are the same as the resolution of the target domain image x2.

[0145] The expression of the third loss function L3 is:

[0146] wherein w is the total number of elements (here, the total number of elements refers to the total number of elements in p2 or p 2A ), k is the serial number of the element, (p2) represents the one-dimensional flattened vector of p2, (p2) k is the k-th element of (p2), (p 2A ) represents the one-dimensional flattened vector of p 2A , (p 2A ) k is the k-th element of p 2A .

[0147] The expression of the total loss function Loss can be referred to in the foregoing, and will not be described here again.

[0148] After the training process is completed, a defect detection model M is obtained, and based on the defect detection model M, the present application provides a defect detection method, please refer to FIG. 6, some embodiments of the defect detection method include:

[0149] Step 310: obtaining a to-be-detected image of a target domain, that is, the to-be-detected image and the target domain image are images under the same imaging scene, for example, can be images of products on a production line in the target domain;

[0150] Step 320: inputting the to-be-detected image into the defect detection model M for defect detection to obtain a defect detection result of the to-be-detected image.

[0151] The defect detection result can include a defect prediction map of the to-be-detected image. In some embodiments, as shown in FIG. 7 (the left image is the to-be-detected image, and the right image is the defect prediction map), the defect prediction map is a binary image (that is, the classification category has two), the black pixel point represents the background, and the white pixel point represents the defect.

[0152] On the basis of the above model migration method, the present application further provides a model migration device for migrating a baseline model obtained from a source domain to a target domain to obtain a migration model, please refer to FIG. 8, one embodiment of the device includes a source domain data acquisition unit 11, a target domain data acquisition unit 12, a denoising unit 13, a pseudo-label generation unit 14 and a training unit 15, which are described below.

[0153] The source domain data acquisition unit 11 is used to acquire source domain images and corresponding labels, and acquire a baseline model trained by the source domain images and the corresponding labels.

[0154] The target domain data acquisition unit 12 is used to acquire target domain images, some or all of which have corresponding labels, or none of which has a corresponding label, if some or all of the target domain images have corresponding labels, the labels of the target domain images with corresponding labels are acquired.

[0155] The denoising unit 13 is connected with the source domain data acquisition unit 11, and is used to input the labels of the source domain images into a learnable denoiser to obtain denoised labels. The expression of the denoiser is described above.

[0156] The pseudo-label generation unit 14 is connected with the source domain data acquisition unit 12, and is used to input the target domain images without corresponding labels into the initial baseline model to obtain a second target domain prediction result, and perform a data augmentation operation on the target domain images without corresponding labels to obtain a second target domain augmented image, and take the second target domain prediction result as the corresponding pseudo-label of the second target domain augmented image.

[0157] The initial baseline model is a baseline model whose parameters are not updated before training.

[0158] The training unit 15 is connected with the source domain data acquisition unit 11, the source domain data acquisition unit 12, the denoising unit 13 and the pseudo-label generation unit 14, and is configured to train the baseline model and the denoiser by using the source domain images and the corresponding denoising labels, and the target domain images with the corresponding labels and the corresponding pseudo-labels and / or the second target domain augmented images and the corresponding pseudo-labels. The baseline model obtained at the end of the training is the migration model after the migration is completed.

[0159] In some embodiments, the training of the baseline model and the denoiser by the training unit 15 includes the following steps: inputting the source domain images into the baseline model to obtain source domain prediction results; if there are target domain images with corresponding labels, inputting the target domain images into the baseline model to obtain first target domain prediction results; if there are target domain images without corresponding labels, inputting the second target domain augmented images obtained by performing data augmentation operation on the target domain images without corresponding labels into the baseline model to obtain second target domain augmented prediction results; and training the baseline model and the denoiser according to the total loss function Loss.

[0160] The total loss function Loss is determined by the first loss function L1, the second loss function L2 and / or the third loss function L3. The first loss function L1 is determined according to the difference between the source domain prediction results and the corresponding denoising labels. The second loss function L2 is determined at least according to the difference between the first target domain prediction results and the corresponding labels. The third loss function L3 is determined according to the difference between the second target domain augmented prediction results and the corresponding pseudo-labels.

[0161] In some embodiments, if there are target domain images with corresponding labels, the training unit 15 further performs the same data augmentation operation on the target domain images with corresponding labels and the corresponding labels to obtain first target domain augmented images and corresponding first augmented labels; and inputs the first target domain augmented images into the baseline model to obtain first target domain augmented prediction results. The second loss function L2 is further determined according to the difference between the first target domain augmented prediction results and the corresponding first augmented labels, and the difference between the first target domain augmented prediction results and the corresponding first target domain prediction results.

[0162] For how to set the loss functions, please refer to the relevant description of the model migration method above, which will not be repeated here.

[0163] On the basis of the above defect detection method and model migration device, the present application further provides a defect detection device. Please refer to FIG. 9. In one embodiment, the device includes an image to be detected acquisition module 20, a detection module 30 and a model training module 40, which will be described below.

[0164] The image to be detected acquisition module 20 is configured to acquire target domain images to be detected.

[0165] The detection module 30 is connected with the image acquisition module 20 to be detected, and is configured to input the image to be detected into the pre-trained defect detection model to perform defect detection, so as to obtain a defect detection result of the image to be detected.

[0166] The model training module 40 is connected with the detection module 30, and is configured to train the defect detection model. The model training module 40 comprises a source domain data acquisition unit 41, a target domain data acquisition unit 42, a denoising unit 43, a pseudo-label generation unit 44 and a training unit 45.

[0167] The source domain data acquisition unit 41 is configured to acquire source domain images and corresponding defect labels, and acquire a baseline model trained by the source domain images and the corresponding defect labels.

[0168] The target domain data acquisition unit 42 is configured to acquire target domain images, some or all of which have corresponding defect labels, or none of which has corresponding defect labels. If some or all of the target domain images have corresponding defect labels, the defect labels of the target domain images having corresponding defect labels are acquired.

[0169] The denoising unit 43 is connected with the source domain data acquisition unit 41, and is configured to input the defect labels of the source domain images into the learnable denoiser to obtain denoised labels. The expression of the denoiser is described above.

[0170] The pseudo-label generation unit 44 is connected with the target domain data acquisition unit 42, and is configured to, if there are target domain images without corresponding defect labels, input the target domain images into the initial baseline model to obtain second target domain prediction results, perform data augmentation on the target domain images without corresponding defect labels to obtain second target domain augmented images, and take the second target domain prediction results as the corresponding pseudo labels of the second target domain augmented images.

[0171] The training unit 45 is connected with the source domain data acquisition unit 41, the target domain data acquisition unit 42, the denoising unit 43 and the pseudo-label generation unit 44, and is configured to train the baseline model and the denoiser by using the source domain images and the corresponding denoised labels, the target domain images with corresponding defect labels and the corresponding defect labels, and / or the second target domain augmented images and the corresponding pseudo labels. The baseline model obtained at the end of the training is the defect detection model of the target domain.

[0172] In some embodiments, the training unit 45 trains the baseline model and the denoiser includes the following steps: inputting the source domain image into the baseline model to obtain a source domain prediction result; if there is a target domain image with corresponding defect label, inputting the target domain image into the baseline model to obtain a first target domain prediction result; if there is a target domain image without corresponding defect label, inputting a second target domain augmented image obtained by performing a data augmentation operation on the target domain image without corresponding defect label into the baseline model to obtain a second target domain augmented prediction result; and training the baseline model and the denoiser according to a total loss function Loss.

[0173] The total loss function Loss is determined by a first loss function L1, a second loss function L2, and / or a third loss function L3. Here, the first loss function L1 is determined according to the difference between the source domain prediction result and the corresponding denoising label, the second loss function L2 is determined at least according to the difference between the first target domain prediction result and the corresponding defect label, and the third loss function L3 is determined according to the difference between the second target domain augmented prediction result and the corresponding pseudo label.

[0174] In some embodiments, if there is a target domain image with corresponding defect label, the training unit 45 further performs the same data augmentation operation on the target domain image with corresponding defect label and the corresponding defect label to obtain a first target domain augmented image and a corresponding first augmented label; and inputs the first target domain augmented image into the baseline model to obtain a first target domain augmented prediction result. The second loss function L2 is further determined according to the difference between the first target domain augmented prediction result and the corresponding first augmented label, and the difference between the first target domain augmented prediction result and the corresponding first target domain prediction result.

[0175] In some embodiments, the second loss function L2 is composed of three parts: L R1 determined according to the difference between the first target domain prediction result p1 and the corresponding defect label y1, L 1A determined according to the difference between the first target domain augmented prediction result p 1A and the corresponding first augmented label y R1A , and a consistency loss function L 1A determined according to the difference between the first target domain augmented prediction result p R11A and the corresponding first target domain prediction result p1.

[0176] For details of how to set each loss function, please refer to the relevant introduction of the model migration method above, which will not be repeated here.

[0177] The technical solution of the present application has the following three advantages compared with the prior art.

[0178] (1) The required target domain sample quantity is small, the data cost of enterprises is reduced, and the migration speed is fast. The migration model proposed in the application is used for migration training, only a small amount of target domain data is required for training, the obtained migration model has excellent detection performance, and the production cost of enterprises is greatly reduced.

[0179] (2) The domain adaptation process from the source domain to the target domain is regarded as a "denoising" problem for solving, the source domain image is modeled as a noise sample of the target domain image, the labeling of the source domain image is corrected, the matching of the source domain image and the target domain image is realized, the obtained migration model has low false detection rate, high detection accuracy, and greatly improved detection performance.

[0180] (3) The labeling of the input target domain image is adaptively designed, the flexibility of use is improved, and the use scene of the technology is widened. The model migration method of the application has a supervised mode, a semi-supervised mode and an unsupervised mode, and can automatically determine the training mode according to the labeling of the target domain image. The user can arrange the input data according to the actual needs of the task, without manually selecting the training mode, so that the use is flexible and convenient, and the work efficiency is improved. In the use process, the user can flexibly label the target domain image according to the actual needs to cope with different task needs and detection scenes, and the model migration method of the application can adaptively perform migration training.

[0181] Those skilled in the art can understand that all or part of the functions of the various methods in the above embodiments can be realized by hardware or by a computer program. When all or part of the functions in the above embodiments are realized by a computer program, the program can be stored in a computer readable storage medium, which can include read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are realized by executing the program by a computer. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, the above functions are realized. In addition, when all or part of the functions in the above embodiments are realized by a computer program, the program can also be stored in a server, another computer, a storage medium such as a disk, an optical disk, a flash disk or a mobile hard disk, and is downloaded or copied into the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, the above functions are realized.

[0182] The above application of specific examples is used to illustrate the application, which is only used to help understand the application, and does not limit the application. According to the idea of the application, those skilled in the art can make several simple deductions, deformations or substitutions.

Claims

1. A defect detection method, characterized in that, include: Obtain the image to be detected in the target domain; The image to be detected is input into a pre-trained defect detection model for defect detection to obtain the defect detection result of the image to be detected; The defect detection model is trained in the following way: Obtain the source domain image and the corresponding defect annotations, and obtain the baseline model trained by the source domain image and the corresponding defect annotations; Acquire a target domain image, wherein the source domain and the target domain are different imaging scenarios, and the different imaging scenarios refer to different objects under test and / or different imaging conditions; some or all of the target domain image has corresponding defect annotations, or none of them has corresponding defect annotations; if some or all of the target domain image has corresponding defect annotations, then acquire the defect annotations of the target domain image with corresponding defect annotations. The defect annotations of the source domain image are input into a learnable denoiser to obtain denoised annotations; If there is a target domain image without corresponding defect annotations, it is input into the initial baseline model to obtain the second target domain prediction result, and the target domain image without corresponding defect annotations is augmented to obtain the second target domain augmented image. The second target domain prediction result is used as the corresponding pseudo-annotation of the second target domain augmented image; the initial baseline model is the baseline model whose parameters were not updated before training. The baseline model and the denoiser are trained using the source domain image and the corresponding denoised annotation, the target domain image with the corresponding defect annotation and the corresponding defect annotation, and / or the second target domain augmented image and the corresponding pseudo-annotation; the defect detection model is the baseline model obtained at the end of training.

2. The defect detection method as described in claim 1, characterized in that, The step of training the baseline model and the denoiser using the source domain image and corresponding denoised annotations, and the target domain image with corresponding defect annotations and corresponding defect annotations and / or the second target domain augmented image and corresponding pseudo-annotations, includes: The source domain image is input into the baseline model to obtain the source domain prediction result; If a target domain image with corresponding defect annotations exists, it is input into the baseline model to obtain the first target domain prediction result; If there is a target domain image without corresponding defect annotations, the second target domain augmented image obtained by performing data augmentation on the target domain image without corresponding defect annotations will be input into the baseline model to obtain the second target domain augmentation prediction result. The baseline model and the denoiser are trained based on the total loss function Loss; The total loss function Loss is determined by a first loss function L1, a second loss function L2, and / or a third loss function L3. The first loss function L1 is determined based on the difference between the source domain prediction result and the corresponding denoised label. The second loss function L2 is determined based at least on the difference between the first target domain prediction result and the corresponding defect label. The third loss function L3 is determined based on the difference between the second target domain expansion prediction result and the corresponding pseudo label.

3. The defect detection method as described in claim 2, characterized in that, The expression for the denoiser is: y tc = (1-β)×onehot(y)+β×p, Among them, y tc y is the denoising annotation, y is the defect annotation of the source domain image, onehot(y) represents the one-hot encoding operation performed on y, p is the source domain prediction result, and β is a learnable parameter.

4. The defect detection method as described in claim 2, characterized in that, Also includes: If a target domain image with corresponding defect annotations exists, the same data augmentation operation is performed on the target domain image with corresponding defect annotations and the corresponding defect annotations to obtain a first target domain augmented image and a corresponding first augmented annotation; the first target domain augmented image is input into the baseline model to obtain the first target domain augmentation prediction result; The second loss function L2 is also determined based on the difference between the first target domain expansion prediction result and the corresponding first expansion label, and the difference between the first target domain expansion prediction result and the corresponding first target domain prediction result.

5. The defect detection method as described in claim 4, characterized in that, The first target domain prediction result includes a first predicted multi-class score map p1, where elements in the first predicted multi-class score map p1 are used to represent the probability that a corresponding pixel in the target domain image with corresponding defect annotations belongs to each classification category; at least one of the classification categories is used to represent a defect. The defect annotation of the target domain image with corresponding defect annotations includes a target domain category annotation map y1, and the elements in the target domain category annotation map y1 are used to represent the actual classification category of the corresponding pixel in the target domain image with corresponding defect annotations. The first target domain augmentation prediction result includes the first augmentation prediction multi-class score map p. 1A The first expanded prediction multi-class score map p 1A The elements in the first target domain expanded image are used to represent the probability that the corresponding pixel belongs to each classification category; The first augmented annotation includes the target domain category annotation map y1 obtained after data augmentation operation. 1A ; The first predicted multi-class score map p1, the target domain category label map y1, and the first augmented predicted multi-class score map p 1A and the target domain extended category annotation map y 1A The resolution of each image is the same as that of the target domain image with corresponding defect annotations. The expression for the second loss function L2 is: L2=α1L R1 +α2L R1A +α3L R11A , Where α1, α2, and α3 are preset weighting coefficients, Wherein, argmax(p1) represents the first category prediction map obtained by taking the category with the highest probability from the elements in the first predicted multi-class score map p1, argmax(p1)∩y1 represents the number of intersection elements between the first category prediction map and the corresponding target domain category label map y1, and argmax(p1)∪y1 represents the number of union elements between the first category prediction map and the corresponding target domain category label map y1; Wherein, argmax(p 1A ) represents the first augmented prediction multi-class score map p 1A The first expanded class prediction map is obtained by taking the class with the highest probability from the elements in the array, argmax(p 1A )∩y 1A This represents the first expanded category prediction map and the corresponding target domain expanded category annotation map y. 1A The number of elements in the intersection between them, argmax(p 1A )∪y 1A This represents the first expanded category prediction map and the corresponding target domain expanded category annotation map y. 1A The number of elements in the union of the two sets; Where m is the total number of elements, i is the element index, and (p1) represents the one-dimensional flattened vector of p1. i For the i-th element of (p1), (p 1A ) represents p 1A A one-dimensional flattened vector, (p 1A ) i For p 1A The i-th element, where the one-dimensional flattening vector refers to the vector formed by arranging the values ​​of each channel of each element in the graph in order.

6. The defect detection method as described in claim 2, characterized in that, The source domain prediction result includes the probability that the source domain image belongs to each classification category; at least one of the classification categories is used to represent a defect; The defect annotation of the source image includes the actual probability that the source image belongs to each classification category, and the denoising annotation includes the probability corresponding to each classification category obtained after the defect annotation of the source image is processed by the denoiser. The expression for the first loss function L1 is: Where n is the total number of categories, j is the category number, and p j The probability that the source domain image belongs to the j-th classification category is included in the source domain prediction result. tc ) j This represents the probability corresponding to the j-th classification category in the denoised annotation.

7. The defect detection method as described in claim 2, characterized in that, The second target domain prediction result includes a second predicted multi-class score map p2, where elements in the second predicted multi-class score map p2 are used to represent the probability that a corresponding pixel in the target domain image without corresponding defect annotation belongs to each classification category; at least one of the classification categories is used to represent a defect. The second target domain augmentation prediction result includes the second augmentation prediction multi-class score map p. 2A The second expanded prediction multi-class score map p 2A The elements in the array are used to represent the probability that the corresponding pixel in the second target domain augmented image belongs to each classification category; The second predicted multi-class score map p2 and the second augmented predicted multi-class score map p 2A The resolution of all of them is the same as that of the target domain image that does not have corresponding defect annotations; The expression for the third loss function L3 is: Where w is the total number of elements, k is the element index, and (p2) represents the one-dimensional flattened vector of p2. k For the k-th element of (p2), (p 2A ) represents p 2A A one-dimensional flattened vector, (p 2A ) k For p 2A The k-th element, where the one-dimensional flattening vector refers to the vector formed by arranging the values ​​of each channel of each element in the graph in order.

8. The defect detection method according to any one of claims 2 to 7, characterized in that, If all the target domain images have corresponding defect annotations, then the expression for the total loss function Loss is: Loss = δ1L1 + δ2L2, If none of the target domain images have corresponding defect annotations, then the expression for the total loss function Loss is: Loss = δ1L1 + δ3L3 If the target domain image portion has corresponding defect annotations, then the expression for the total loss function Loss is: Loss = δ1L1 + δ2L2 + δ3L3 δ1, δ2 and δ3 are preset weight coefficients.

9. The defect detection method as described in claim 1, characterized in that, The imaging conditions include imaging equipment, shooting parameters, lighting and / or shooting environment.

10. A model transfer method, characterized in that, The model transfer method is used to transfer a baseline model obtained from a source domain to a target domain to obtain a transferred model. Obtain the source domain image and its corresponding annotations, and obtain the baseline model trained using the source domain image and its corresponding annotations; Obtain a target domain image, wherein some or all of the target domain image has corresponding labels, or none of them have corresponding labels. If some or all of the target domain image has corresponding labels, then obtain the labels of the target domain image with corresponding labels. The annotations of the source domain image are input into a learnable denoiser to obtain denoised annotations; If there is a target domain image without corresponding annotations, it is input into the initial baseline model to obtain the second target domain prediction result, and the target domain image without corresponding annotations is augmented to obtain the second target domain augmented image. The second target domain prediction result is used as the corresponding pseudo-annotation of the second target domain augmented image; the initial baseline model is the baseline model whose parameters were not updated before training. The baseline model and the denoiser are trained using the source domain image and the corresponding denoised annotations, the target domain image with the corresponding annotations and the corresponding annotations, and / or the second target domain augmented image and the corresponding pseudo-annotations; the transfer model is the baseline model obtained at the end of training.

11. The model transfer method as described in claim 10, characterized in that, The step of training the baseline model and the denoiser using the source domain image and corresponding denoised annotations, and the target domain image with corresponding annotations and corresponding annotations and / or the second target domain augmented image and corresponding pseudo-annotations, includes: The source domain image is input into the baseline model to obtain the source domain prediction result; If a target domain image with corresponding annotations exists, it is input into the baseline model to obtain the first target domain prediction result; If there is a target domain image without corresponding annotation, the second target domain augmented image obtained by performing data augmentation on the target domain image without corresponding annotation is input into the baseline model to obtain the second target domain augmentation prediction result. The baseline model and the denoiser are trained based on the total loss function Loss; The total loss function Loss is determined by a first loss function L1, a second loss function L2, and / or a third loss function L3. The first loss function L1 is determined based on the difference between the source domain prediction result and the corresponding denoised label. The second loss function L2 is determined based at least on the difference between the first target domain prediction result and the corresponding label. The third loss function L3 is determined based on the difference between the second target domain augmentation prediction result and the corresponding pseudo label.

12. The model transfer method as described in claim 11, characterized in that, Also includes: If a target domain image with corresponding annotations exists, the same data augmentation operation is performed on the target domain image with corresponding annotations and the corresponding annotations to obtain a first target domain augmented image and a corresponding first augmented annotation; the first target domain augmented image is input into the baseline model to obtain the first target domain augmentation prediction result; The second loss function L2 is also determined based on the difference between the first target domain expansion prediction result and the corresponding first expansion label, and the difference between the first target domain expansion prediction result and the corresponding first target domain prediction result.

13. A model transfer method, characterized in that, This is used to transfer a baseline model obtained from the source domain to a target domain to obtain a transfer model. The target domain and the source domain have different imaging scenarios. The different imaging scenarios refer to the same object being measured from different sources, similar objects being measured, and / or different imaging conditions. The model transfer method includes: Obtain the baseline model of the source domain, as well as the images used to train that baseline model and their corresponding annotations; The annotations of the source domain image are input into a learnable denoiser to obtain denoised annotations; Acquire the target domain image; If the target domain image has corresponding annotations, then obtain the annotations of the target domain image with corresponding annotations; If the target domain image does not have a corresponding label, it is input into the initial baseline model to obtain the second target domain prediction result, and the target domain image without a corresponding label is augmented to obtain the second target domain augmented image. The second target domain prediction result is used as the corresponding pseudo label of the second target domain augmented image. The initial baseline model is the baseline model whose parameters have not been updated before training. The baseline model and the denoiser are trained using the source domain image and the corresponding denoised annotations, the target domain image with the corresponding annotations and the corresponding annotations, and / or the second target domain augmented image and the corresponding pseudo-annotations; the transfer model is the baseline model obtained at the end of training.

14. A defect detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected in the target domain; The detection module is connected to the image acquisition module and is used to input the image to be detected into a pre-trained defect detection model for defect detection in order to obtain the defect detection result of the image to be detected. A model training module, connected to the detection module, is used to train the defect detection model, including: The source domain data acquisition unit is used to acquire source domain images and corresponding defect annotations, and to acquire a baseline model trained by the source domain images and corresponding defect annotations. The target domain data acquisition unit is used to acquire a target domain image, wherein the source domain and the target domain are different imaging scenarios, and the different imaging scenarios refer to different objects under test and / or different imaging conditions; some or all of the target domain image has corresponding defect annotations, or none of them has corresponding defect annotations; if some or all of the target domain image has corresponding defect annotations, then the defect annotations of the target domain image with corresponding defect annotations are acquired. The denoising unit, connected to the source domain data acquisition unit, is used to input the defect annotations of the source domain image into a learnable denoiser to obtain denoised annotations. The pseudo-label generation unit is connected to the target domain data acquisition unit. If there is a target domain image without corresponding defect labels, it is used to input the image into the initial baseline model to obtain the second target domain prediction result, and to perform data augmentation operation on the target domain image without corresponding defect labels to obtain the second target domain augmented image. The second target domain prediction result is used as the corresponding pseudo-label of the second target domain augmented image. The training unit, connected to the source domain data acquisition unit, the target domain data acquisition unit, the denoising unit, and the pseudo-label generation unit, is used to train the baseline model and the denoiser using the source domain image and the corresponding denoised label, the target domain image with the corresponding defect label and the corresponding defect label, and / or the second target domain augmented image and the corresponding pseudo label. The defect detection model is the baseline model obtained at the end of training; the initial baseline model is the baseline model whose parameters were not updated before training.

15. A model transfer device, characterized in that, The model transfer apparatus is used to transfer a baseline model obtained from a source domain to a target domain to obtain a transferred model, the model transfer apparatus comprising: The source domain data acquisition unit is used to acquire source domain images and corresponding annotations, and to acquire the baseline model trained by the source domain images and corresponding annotations; The target domain data acquisition unit is used to acquire a target domain image, wherein some or all of the target domain image has corresponding labels, or none of them have corresponding labels. If some or all of the target domain image has corresponding labels, the label of the target domain image with corresponding labels is acquired. The denoising unit is connected to the source domain data acquisition unit and is used to input the annotations of the source domain image into a learnable denoiser to obtain denoised annotations. The pseudo-label generation unit is connected to the target domain data acquisition unit. If there is a target domain image without corresponding labels, it is used to input the image into the initial baseline model to obtain the second target domain prediction result, and to perform data augmentation operation on the target domain image without corresponding labels to obtain the second target domain augmented image. The second target domain prediction result is used as the corresponding pseudo-label of the second target domain augmented image. The training unit, connected to the source domain data acquisition unit, the target domain data acquisition unit, the denoising unit, and the pseudo-label generation unit, is used to train the baseline model and the denoiser using the source domain image and the corresponding denoised label, as well as the target domain image with the corresponding label and the corresponding label and / or the second target domain augmented image and the corresponding pseudo-label. The transfer model is the baseline model obtained at the end of training; the initial baseline model is the baseline model whose parameters were not updated before training.

16. A computer-readable storage medium, characterized in that, The medium stores a program that can be executed by a processor to implement the defect detection method as described in any one of claims 1 to 9 or the model transfer method as described in any one of claims 10 to 13.

17. A computer program product comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the defect detection method as described in any one of claims 1 to 9 or the model transfer method as described in any one of claims 10 to 13.

Images