Defect detection model training method and device and sample data generation method and device

Abstract

The invention provides a defect detection model training method and device, equipment and a storage medium, and relates to the technical field of machine learning, and the method comprises the steps: obtaining first sample data and labels, the first sample data being picture data obtained in an actual production process, and including pictures of non-defective products and pictures of defective products; second sample data and labels are obtained, the second sample data are product pictures with artificially set defects, the first sample data and the second sample data are input into a defect detection model, and the defect detection model is trained. In this way, the problem that the detection effect of the trained defect detection model is poor due to the fact that the number and types of training samples are small in the training process of an existing defect detection model is solved.

Description

Technical Field

[0001] The present application relates to the technical field of machine learning, and in particular, to a method for training a defect detection model, a method for generating sample data, and an apparatus therefor. Background Art

[0002] After the production of a product is completed, automatically performing defect detection through a trained model is of great significance for constructing an automated production line, improving the product qualification rate, and thus liberating productivity. An industrial-level defect automatic detection model has extremely high requirements for the accuracy of detection. Therefore, a large number of samples are required to improve the sensitivity and robustness of the automatic detection model.

[0003] However, in the actual production process, especially in production lines where the industry is relatively mature and the error rate of products is low, the probability of producing certain specific defect samples is extremely low, resulting in an unbalanced distribution of training data when using defect samples to train a defect detection model. The number of many types of defect samples is seriously insufficient, and the trained model is extremely prone to overfitting for certain defects, thereby resulting in a poor effect of the defect detection model. Summary of the Invention

[0004] In view of this, the present application provides a method for training a defect detection model, a method for generating sample data, and an apparatus therefor, aiming to solve the problem that the detection effect of the existing defect detection model is poor due to the small number and types of training samples during the training process.

[0005] In a first aspect, the present application provides a method for training a defect detection model, including:

[0006] Obtaining first sample data and annotations, where the first sample data is image data obtained in the actual production process, including images of defect-free products and images of defective products;

[0007] Obtaining second sample data and annotations, where the second sample data is images of products with artificially set defects;

[0008] Inputting the first sample data and the second sample data into a defect detection model to train the defect detection model.

[0009] Optionally, before inputting the first sample data and the second sample data into the defect detection model, the method further includes:

[0010] Performing data augmentation processing on the first sample data and the second sample data respectively;

[0011] The inputting the first sample data and the second sample data into the defect detection model includes:

[0012] Input the first sample data and the second sample data after the data augmentation process into the defect detection model.

[0013] Optionally, the number of various defect samples input into the defect detection model respectively meets the threshold.

[0014] In a second aspect, an embodiment of the present application provides a method for generating sample data, which is used to generate product pictures with artificially set defects in training a defect detection model. The method includes:

[0015] Input the customized feature and the customized feature prompt into a pre-trained second sample data generation model to generate the second sample data;

[0016] The customized feature is an artificially set product defect; the customized feature prompt is a corresponding text description of the artificially set product defect.

[0017] Optionally, the process of training the second sample data generation model includes:

[0018] Obtain the third sample data, where the third sample data is picture data of defective products in the actual production process;

[0019] Input the third sample data into a feature extractor, and the feature extractor extracts features from the third sample;

[0020] Input the features extracted from the third sample and the text description of the third sample into the second sample data generation model to generate a reconstructed sample;

[0021] Adjust the second sample data generation model, and judge whether the second sample data generation model is trained by judging the difference between the reconstructed sample and the third sample;

[0022] When the difference between the reconstructed sample and the third sample reaches the minimum value, the training of the second sample data generation model is completed.

[0023] Optionally, the step of inputting the customized feature and the customized feature prompt into a pre-trained second sample data generation model to generate the second sample data includes:

[0024] Obtain a customized contour, where the customized contour is an artificially set product defect contour;

[0025] Input the customized contour into a feature extractor, and the feature extractor extracts features from the customized contour to obtain a customized feature;

[0026] Input the customized feature and the customized feature prompt into the second sample data generation model, and obtain the second sample data through the second sample data generation model.

[0027] Optionally, the obtaining of the customized contour includes:

[0028] Obtaining a picture of a defect-free product, and modifying the picture of the defect-free product, including performing one or more of adding, reducing, superimposing, and deforming the contour line of the product in the picture, to obtain the customized contour.

[0029] Optionally, the obtaining of the customized contour includes:

[0030] Obtaining a picture of a defect-free product, and modifying the picture of the defect-free product, including randomly adding bubbles and / or scratches to the picture of the defect-free product, to obtain the customized contour.

[0031] In a third aspect, an embodiment of the present application provides a defect detection model training device, which is characterized in that the device includes:

[0032] An obtaining unit, configured to obtain first sample data and annotations, where the first sample data is picture data obtained in the actual production process, including pictures of defect-free products and pictures of defective products;

[0033] The obtaining unit is further configured to obtain second sample data and annotations, where the second sample data is pictures of products with artificially set defects;

[0034] A training unit, configured to input the first sample data and the second sample data into a defect detection model, and train the defect detection model.

[0035] Optionally, the device further includes: a data processing unit, configured to perform data augmentation processing on the first sample data and the second sample data respectively;

[0036] The training unit is specifically configured to: input the first sample data and the second sample data after the data augmentation processing into a defect detection model.

[0037] Optionally, the number of various defect samples input into the defect detection model respectively meets a threshold.

[0038] In a fourth aspect, an embodiment of the present application provides a sample data generation device, configured to generate sample data for training a defect detection model, and the device includes:

[0039] An input unit, configured to input a customized feature and a customized feature prompt into a pre-trained second sample data generation model;

[0040] A data generation unit, configured to generate the second sample data;

[0041] The customized feature is a product defect set artificially; the customized feature prompt is the corresponding text description of the product defect set artificially.

[0042] This application provides a method for training a defect detection model. When executing the method, first sample data and annotations are obtained. The first sample data is image data obtained during the actual production process, including images of defect-free products and images of defective products. Second sample data and annotations are also obtained. The second sample data is images of products with artificially set defects. Then, the first sample data and the second sample data are input into the defect detection model to train the defect detection model. In this way, by using both real images of products and product defect images generated by artificially setting defects as training samples to train the defect detection model, the types of samples on which the defect detection model is trained are relatively rich, solving the problem that the detection effect of the existing defect detection model is poor due to the small number and types of training samples during the training process. Brief Description of the Drawings

[0043] To more clearly illustrate the technical solutions in this embodiment or the prior art, the following will briefly introduce the drawings required for the description of the embodiment or the prior art. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of a method for training a defect detection model provided by an embodiment of this application;

[0045] Figure 2 It is a flowchart of a method for generating sample data provided by an embodiment of this application;

[0046] Figure 3 It is a schematic diagram of the training process of a second sample data generation model provided by an embodiment of this application;

[0047] Figure 4 It is a schematic diagram of generating second sample data by using a second sample data generation model provided by an embodiment of this application;

[0048] Figure 5 It is a schematic diagram of generating second sample data provided by an embodiment of this application;

[0049] Figure 6 It is a specific architecture schematic diagram of a second sample data generation model provided by an embodiment of this application;

[0050] Figure 7Schematic diagram of a defect detection model training device provided by an embodiment of the present application;

[0051] Figure 8 Schematic diagram of a sample data generation device provided by an embodiment of the present application. Detailed implementation manners

[0052] As described in the background art of the present application, in the actual production process, especially in a production line with mature industry and low product error rate, the probability of producing certain specific defect samples is extremely low, resulting in unbalanced distribution of training data when training a defect detection model with defect samples, and the trained model is extremely prone to overfitting for certain defects, thus resulting in poor defect detection model performance.

[0053] To solve the above technical problems, an embodiment of the present application provides a defect detection model training method, which includes:

[0054] Obtain first sample data and annotations, where the first sample data is picture data obtained in the actual production process, including pictures of defect-free products and pictures of defective products; and obtain second sample data and annotations, where the second sample data is pictures of products with artificially set defects, and then input the first sample data and the second sample data into the defect detection model to train the defect detection model. In this way, by using both the real pictures of products and the product defect pictures generated by artificially setting defects as training samples to train the defect detection model, the types of samples on which the defect detection model is based during training are relatively rich, solving the problem that the detection performance of the trained defect detection model is poor due to the small number and types of training samples in the existing defect detection model training process.

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0056] Figure 1 Flowchart of a defect detection model training method provided by an embodiment of the present application. As shown in Figure 1 The defect detection model training method provided by the embodiment of the present application may include:

[0057] S101. Obtain first sample data and annotations, where the first sample data is picture data obtained in the actual production process, including pictures of defect-free products and pictures of defective products.

[0058] The first sample data includes pictures of normal and defective products. When the product is a contact lens, the first sample data is pictures of non-defective contact lenses and pictures of defective contact lenses, where the types of defects include surface defects such as impurities, scratches, splits, etc., edge defects such as multi-sided, curled edges, missing edges, etc., and also problems such as air bubbles, pores inside the contact lens, multiple contact lenses overlapping, and contact lenses being upside down.

[0059] The annotation of the first sample includes the annotation of the specific defect positions of the defective contact lenses, the annotation of the type of each defect, and the generation of an annotation mask equal in size to the real picture, where different defect types can be corresponding to different pixel values.

[0060] S102. Obtain the second sample data and its annotation, where the second sample data is pictures of products with artificially set defects.

[0061] Since in the actual production process, the types of generated defect samples are limited. Especially for production lines with better production processes, the types of defect samples are very few. To ensure that the defect detection model can recognize various types of defects, it is necessary to artificially set the defect types and generate pictures of defective products through artificial means, that is, the second sample data. When obtaining the second sample data, the corresponding annotation of the second sample data is obtained at the same time. It should be noted that the annotation process can be automatically generated according to the artificially set defects and then manually confirmed, or directly manually annotated for the second sample data. In addition, the defect detection model in this application includes various types such as an image classification model, an object detection model, and a semantic segmentation model based on machine learning during training.

[0062] S103. Input the first sample data and the second sample data into the defect detection model to train the defect detection model.

[0063] The first sample data is real sample data, and the second sample data is pictures of defective products with artificially set defects, that is, artificial sample data. Using the real sample data and the artificial sample data for model training at the same time ensures the richness of the samples. Inputting these sample data into the defect detection model, the defect detection model obtains a large number of images of defective products of different types, making the types of samples on which the defect detection model is trained relatively rich, and solving the problem that the detection effect of the trained defect detection model is poor due to the small number and types of training samples in the training process of the existing defect detection model.

[0064] In an implementation manner of the embodiment of the present application, before inputting the first sample data and the second sample data into the defect detection model, the method further includes: respectively performing data augmentation processing on the first sample data and the second sample data; the step of inputting the first sample data and the second sample data into the defect detection model includes: inputting the first sample data and the second sample data after the data augmentation processing into the defect detection model.

[0065] The data augmentation processing includes traditional data augmentation processing. Its essence is to generate the value equivalent to a larger amount of data on the basis of the existing limited data without actually collecting more data, that is, the process of generating incremental data according to the existing data samples according to rules. It is to process the existing sample data to obtain more sample data. Specifically, it can be processed by magnifying and reducing, rotating, projective transformation, changing hue, saturation, brightness, adding noise, blurring, and random splicing of the existing sample data. In this embodiment, after performing data augmentation processing on the first sample data and the second sample data, more sample data is obtained for training the defect detection model.

[0066] In an implementation manner of the embodiment of the present application, the number of defective samples in the first sample data and the second sample data respectively meets the threshold. The same number threshold can be set for various types of defective samples, or the number ratio of various types of defective samples can be set. Preferably, the ratio is close to 1, or other thresholds can be set according to the experience and requirements of those skilled in the art. Preferably, the number distribution of various types of defective samples is uniform and can meet the training needs of the detection model.

[0067] The above embodiment of the present application provides a method for training a defect detection model, in which the defect detection model is trained by inputting the first sample data and the second sample data into the defect detection model. The following introduces the generation process of the second sample data through specific embodiments.

[0068] Figure 2 It is a flowchart of a method for generating sample data provided by the embodiment of the present application. As shown in combination with Figure 2 The method for generating sample data provided by the embodiment of the present application may include:

[0069] S201. Input the customized feature and the customized feature prompt into a pre-trained second sample data generation model.

[0070] S202. Generate the second sample data. The above S201 and S202 will be introduced in combination below. The customized feature is a product defect set artificially, and the customized feature prompt is the corresponding text description of the product defect set artificially. Since the above second sample data is a product defect picture customized according to actual requirements, in order to reduce the workload of manual work, in the embodiment of the present application, the second sample data is generated by a pre-trained second sample data generation model. The following combines Figure 3 to introduce the training process of the second sample data generation model. Figure 3 FIG. is a schematic diagram of the training process of a second sample data generation model provided by an embodiment of the present application. As Figure 3 shown, the specific training process of the pre-trained second sample data generation model includes:

[0071] Obtain a real sample, where the real sample is the third sample data, and the third sample data is the picture data of defective products in the actual production process. Input the third sample data into a feature extractor, and the feature extractor extracts features from the third sample. Use the features extracted from the third sample as real features and the text description of the third sample, that is, the real feature prompt, and input them into the second sample data generation model. The second sample data generation model is a directional generation model to generate a reconstructed sample. In order to make the second sample data generated by the second sample data generation model similar to the actual product defect picture, it is necessary to debug the second sample data generation model. Specifically, after generating the reconstructed sample, adjust the second sample data generation model, and judge whether the second sample data generation model is trained by judging the difference between the reconstructed sample and the third sample. When the difference value between the reconstructed sample and the third sample is the smallest, the training of the second sample data generation model is completed.

[0072] After completing the training of the second sample data generation model through the above process, a model that can generate the second sample data is obtained. The following combines Figure 4 to introduce the process of generating the second sample data using the second sample data generation model. Figure 4 FIG. is a schematic diagram of generating the second sample data using the second sample data generation model provided by an embodiment of the present application. As Figure 4 shown, this process includes:

[0073] Obtain a customized contour, which is a product defect contour set artificially, that is, a contour image generated according to the required defect type and objective defect law. After passing through a feature extractor, the features extracted from the contour image are similar to the real features and meet the requirement of accurately describing defect features (such as the feature of a bubble being circular or elliptical). After obtaining the customized contour, input the customized contour into the feature extractor, and the feature extractor extracts features from the customized contour to obtain customized features. For example, the canny edge detection algorithm can be used for edge feature extraction to obtain the customized features in the customized contour. The picture corresponding to the customized features is a defect feature randomly generated according to objective laws according to requirements. In this example, contact lenses are taken as an example. Therefore, the customized features mentioned can include features of product pictures such as scratches, cracks, air bubbles, polygons, curled edges, missing edges, and overlaps. When applied to the detection of other products, it can include defect features known to those skilled in the art in other products. Each feature is different. The customized features are artificially set product defects, and the customized feature prompts are corresponding text descriptions of the artificially set product defects. Through the customized feature prompts, further guidance can be provided during the generation of directional samples to ensure the consistency between the generated samples and the expectations. For example, when the defect in the customized features is specifically a scratch on the surface of a contact lens, the corresponding customized feature prompt is "scratch". Then input the customized features and the customized feature prompts into the second sample data generation model, that is, the directional generation model in the figure, and obtain a directional generated sample through the second sample data generation model, that is, the second sample data, which is a picture generated for a contact lens containing specific defects.

[0074] Obtaining the customized contour includes: obtaining a picture of a defect-free product, and modifying the picture of the defect-free product, including performing one or more of the operations of increasing, decreasing, superimposing, and deforming the contour line of the product in the picture, and also obtaining the customized contour by randomly adding air bubbles and / or scratches to the picture of the defect-free product. Specifically, refer to Figure 5 , Figure 5 FIG. [FIGURE NUMBER] is a schematic diagram for generating a second sample data provided in an embodiment of the present application, which includes a schematic diagram of a customized contour, a schematic diagram of customized features, and the generated second sample data, that is, the directional generated sample in the figure. When generating the customized contour in this embodiment, the following process is specifically included:

[0075] Obtain a picture of a defect-free product, that is, a normal sample. A normal sample refers to a contour that is a perfect circle or an ellipse with a low eccentricity. Modify the picture of the defect-free product, including performing one or more of the operations of increasing, decreasing, superimposing, and deforming the contour line of the product in the picture to obtain the customized contour. Specifically, obtain contours with the following defect types:

[0076] Please note that the [FIGURE NUMBER] in needs to be filled with the actual figure number. Also, the ,

[0074] , etc. tags are likely related to specific figure or reference numbers in the original document and should be adjusted according to the actual situation.Crimping, based on a normal circle / ellipse, randomly select an arc, and add a sine function with a random amplitude and period or a sawtooth wave function with a random amplitude and period to the arc radius corresponding to the arc, to obtain a customized profile with a crimping defect;

[0077] Overlap, based on a normal circle / ellipse, randomly perform an offset, or superimpose the radius and angle with another circle / ellipse, to obtain a customized profile with an overlap defect;

[0078] Edge defect: Based on a normal circle / ellipse, randomly select an arc, and replace the arc edge corresponding to the arc with a straight line, to obtain a customized profile with an edge defect;

[0079] Multi-sided, based on a normal circle / ellipse, randomly select an arc, and replace the arc edge corresponding to the arc with multiple arcs with a random number of arcs and a radius randomly within a certain range, to obtain a customized profile with a multi-variable defect;

[0080] Bubble, randomly add an ellipse with a random radius, a random eccentricity, and a random direction, to obtain a customized profile with a bubble defect;

[0081] Scratch, randomly add a straight line with a random length, to obtain a customized profile with a scratch defect.

[0082] According to the fact that multiple defects may coexist in the samples during actual production, the customization of different types of defects can be directly superimposed. For example, a certain defect profile includes two types of defects: crimping and multi-sided. After obtaining the customized profile, extract the edge features corresponding to the profile, which simulate the edge features of the real samples, so as to ensure the consistency between the prediction result and the real samples. Input the customized edge features and the text prompts of the customized features into the trained second sample data generation model to generate defect pictures that match the expected defect type and location, ensuring the diversity and authenticity of the pictures, and improving the accuracy and sensitivity of the corresponding defect detection model.

[0083] The following combines Figure 6 to introduce the specific architecture of the above-trained second sample data generation model. Figure 6Schematic diagram of the specific architecture of a second sample data generation model provided by an embodiment of the present application. The specific architecture of the second sample data generation model includes: trainable parameter 1 and fixed parameter 2; the model input includes input feature 3, input feature prompt 4, time series 5, and input signal 6; the model output is output signal 7. Trainable parameter 1 includes zero convolution layer 11, stable diffusion encoder 12, and zero convolution layer 13. Fixed parameter 2 includes stable diffusion encoder 22 and stable diffusion decoder 23. At the beginning of model training, all parameters of zero convolution layers 11 and 13 are set to 0, and the parameters of stable diffusion encoder 12 are a copy of the parameters of stable diffusion encoder 22. During the training process, only the parameters in trainable parameter 1 (including zero convolution layers 11 and 13, stable diffusion encoder 12) are changed, while the parameters in fixed parameter 2 remain unchanged. In one implementation manner of the embodiment of the present application, the second sample data generation model is a ControlNet model, that is, a ControlNet based on Stable Diffusion model, which is a neural network structure model that controls the diffusion model by adding additional conditions.

[0084] The second sample data generation model includes multiple sampling steps, and each step is encoded as 0, 1, 2, etc. in natural numbers in sequence. In this example, 20 sampling steps are adopted. The step encoding is the time series 5 of this step. In each sampling step, trainable parameter 1 obtains input feature 3 (i.e., real or customized feature) as the input of zero convolution 11; obtains input signal 6 (i.e., the output signal 7 of the previous sampling step; if it is the first sampling step, it is a random signal), adds the input signal 6 and the input feature 3 processed by the zero convolution layer as the input of stable diffusion encoder 12; and obtains the corresponding customized feature prompt 4 and the time series 5 of this sampling step as the control information of stable diffusion encoder 12.

[0085] In each sampling step, fixed parameter 2 obtains input signal 6 as the input of stable diffusion encoder 22, and obtains the corresponding customized feature prompt 4 and the time series 5 of this sampling step as the control information of stable diffusion encoder 22. The sum of the output of stable diffusion decoder 22 and the output of zero convolution layer 13 is used as the input of stable diffusion decoder 23, and the output signal 7 of this step is calculated as the input signal 6 for the next sampling step; in the final sampling step, the output signal 7 is the picture corresponding to the second sample data.

[0086] The above are some specific implementation manners of the defect detection model training method provided by the embodiment of the present application. Based on this, the present application also provides a corresponding device. Next, the device provided by the embodiment of the present application will be introduced from the perspective of functional modularization.

[0087] Figure 7Schematic structural diagram of a defect detection model training device provided by an embodiment of the present application. In combination with Figure 7 As shown, the defect detection model training device 700 provided by an embodiment of the present application includes:

[0088] An acquisition unit 710, configured to acquire first sample data and annotations, where the first sample data is picture data acquired during the actual production process, including pictures of defect-free products and pictures of defective products;

[0089] The acquisition unit is further configured to acquire second sample data and annotations, where the second sample data is pictures of products with artificially set defects;

[0090] A training unit 720, configured to input the first sample data and the second sample data into a defect detection model to train the defect detection model.

[0091] In an implementation manner of an embodiment of the present application, the device further includes: a data processing unit, configured to perform data augmentation processing on the first sample data and the second sample data respectively;

[0092] The training unit is specifically configured to: input the first sample data and the second sample data after the data augmentation processing into a defect detection model.

[0093] In an implementation manner of an embodiment of the present application, the number of various defect samples input into the defect detection model respectively meets a threshold.

[0094] Figure 8 Schematic structural diagram of a sample data generation device provided by an embodiment of the present application. In combination with Figure 8 As shown, the sample data generation device provided by an embodiment of the present application includes:

[0095] An input unit 810, configured to input a customized feature and a customized feature prompt into a pre-trained second sample data generation model;

[0096] A data generation unit 820, configured to generate the second sample data;

[0097] The customized feature is an artificially set product defect; the customized feature prompt is a corresponding text description of the artificially set product defect.

[0098] In an implementation manner of an embodiment of the present application, the process of training the second sample data generation model includes:

[0099] Acquire third sample data, where the third sample data is picture data of defective products during the actual production process;

[0100] Input the third sample data into a feature extractor, and the feature extractor extracts features from the third sample;

[0101] Input the features extracted from the third sample and the text description of the third sample into a second sample data generation model to generate a reconstructed sample;

[0102] Adjust the second sample data generation model, and determine whether the second sample data generation model is trained completely by judging the difference between the reconstructed sample and the third sample;

[0103] When the difference between the reconstructed sample and the third sample reaches the minimum value, the training of the second sample data generation model is completed.

[0104] In an implementation manner of the embodiment of the present application, the input model is specifically used for:

[0105] Obtain a customized profile, where the customized profile is a product defect profile set artificially;

[0106] Input the customized profile into a feature extractor, and the feature extractor extracts features from the customized profile to obtain customized features;

[0107] Input the customized features and the customized feature prompt into the second sample data generation model, and obtain the second sample data through the second sample data generation model.

[0108] In an implementation manner of the embodiment of the present application, the obtaining of the customized profile includes:

[0109] Obtain a picture of a defect-free product, and transform the picture of the defect-free product, including performing one or more of the operations of adding, reducing, superimposing, and deforming the contour line of the product in the picture, to obtain the customized profile.

[0110] In an implementation manner of the embodiment of the present application, the obtaining of the customized profile includes:

[0111] Obtain a picture of a defect-free product, and transform the picture of the defect-free product, including randomly adding bubbles and / or scratches in the picture of the defect-free product, to obtain the customized profile.

[0112] The embodiment of the present application also provides a corresponding device and a computer storage medium for implementing the solution provided by the embodiment of the present application.

[0113] Wherein, the device includes a memory and a processor, the memory is used for storing instructions or codes, and the processor is used for executing the instructions or codes so that the device executes the method described in any embodiment of the present application.

[0114] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of the present application.

[0115] From the description of the above embodiments, those skilled in the art can clearly understand that all or part of the steps in the above embodiment methods can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment or some parts of the embodiments of the present application.

[0116] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0117] It should also be noted that the embodiments in this specification are all described in a progressive manner. The same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, they are described relatively simply. For the relevant parts, reference can be made to the description of the method embodiments. The device and apparatus embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components described as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0118] As described above, it is only a specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for training a defect detection model, characterized in that, the method includes: Obtain the first sample data and annotations, where the first sample data is image data obtained during the actual production process, including images of defect-free products and images of defective products; Obtain the second sample data and annotations, where the second sample data is images of products with artificially set defects; Input the first sample data and the second sample data into the defect detection model to train the defect detection model.

2. The method according to claim 1, characterized in that, before inputting the first sample data and the second sample data into the defect detection model, the method further includes: Perform data augmentation processing on the first sample data and the second sample data respectively; The inputting the first sample data and the second sample data into the defect detection model includes: Input the first sample data and the second sample data after the data augmentation processing into the defect detection model.

3. The method according to claim 1, characterized in that, The number of various defect samples input into the defect detection model respectively meets the threshold.

4. A method for generating sample data, characterized in that, used to generate images of products with artificially set defects in training the defect detection model, the method includes: Input the customized feature and the customized feature prompt into the pre-trained second sample data generation model; Generate the second sample data; the customized feature is the artificially set product defect; the customized feature prompt is the corresponding text description of the artificially set product defect.

5. The method according to claim 4, characterized in that, The process of training the second sample data generation model includes: Obtain the third sample data, where the third sample data is image data of defective products during the actual production process; Input the third sample data into the feature extractor, and the feature extractor extracts features from the third sample; Input the features extracted from the third sample and the text description of the third sample into the second sample data generation model to generate a reconstructed sample; Adjust the second sample data generation model, and judge whether the second sample data generation model is trained by judging the difference between the reconstructed sample and the third sample; When the difference between the reconstructed sample and the third sample reaches the minimum value, the training of the second sample data generation model is completed.

6. The method according to claim 4, characterized in that, The inputting the customized feature and the customized feature prompt into the pre-trained second sample data generation model to generate the second sample data includes: Obtain the customized contour, where the customized contour is the artificially set product defect contour; Input the customized contour into the feature extractor, and the feature extractor extracts features from the customized contour to obtain the customized feature; Input the customized feature and the customized feature prompt into the second sample data generation model, and obtain the second sample data through the second sample data generation model.

7. The method according to claim 6, characterized in that, The obtaining the customized contour includes: Obtain a picture of a defect-free product, and transform the picture of the defect-free product, including performing one or more of the operations of adding, reducing, superimposing, and deforming the contour line of the product in the picture, to obtain the customized contour.

8. The method according to any one of claims 6 or 7, characterized in that the obtaining of the customized contour includes: Obtain a picture of a defect-free product, and transform the picture of the defect-free product, including randomly adding bubbles and / or scratches in the picture of the defect-free product, to obtain the customized contour.

9. A defect detection model training device, characterized in that the device includes: An acquisition unit, configured to acquire first sample data and annotations, where the first sample data is picture data acquired during actual production, including pictures of defect-free products and pictures of defective products; The acquisition unit is further configured to acquire second sample data and annotations, where the second sample data is pictures of products with artificially set defects; A training unit, configured to input the first sample data and the second sample data into a defect detection model, and train the defect detection model.

10. A sample data generation device, characterized in that it is used to generate pictures of products with artificially set defects in training a defect detection model, and the device includes: An input unit, configured to input customized features and customized feature prompts into a pre-trained second sample data generation model; A data generation unit, configured to generate the second sample data; The customized features are artificially set product defects; the customized feature prompts are corresponding text descriptions of the artificially set product defects.

Images