A defect sample generation method, device, equipment and storage medium

Abstract

The application discloses a kind of defect sample generation method, device, equipment and storage medium, it is related to visual detection technical field, comprising: construct graphics generation module, and generate random graphics by the graphics generation module;Obtain initial image sample, and based on the random graphics and the initial image sample carry out image sample generation operation, to obtain corresponding target defect sample;The pixel-level label corresponding to the target defect sample is generated, to obtain the training data containing the target defect sample and the pixel-level label, so as to utilize the training data to train image defect detection model.The application can solve the problem that sufficient defect samples meeting the requirements are difficult to obtain in actual generation by synthesizing target defect samples based on random graphics generated by graphics generation module and obtained initial image samples, realize the rapid mass generation of defect samples, and reduce the time cost and labor cost consumed by artificial collection and sample labeling.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a method, apparatus, device and storage medium for generating defect samples. Background Technology

[0002] With the rapid development and application of technologies such as artificial intelligence and computer vision, the technology of appearance defect detection has gone through two main stages: manual quality inspection and traditional image processing. At present, it is constantly developing towards deep learning. As a result, my country's machine vision industry has entered a historical juncture of rapid development and transformation.

[0003] Currently, in the application of deep learning detection technology, visual inspection system manufacturers need to manually collect and label a large number of defect samples for model training. The time-consuming and labor-intensive process of manually building a defect database leads to the training process being quite time-consuming and labor-intensive. In other words, most existing deep learning-based appearance defect detection methods require manual collection and labeling of a large number of defect samples for training, which is extremely time-consuming and labor-intensive. Furthermore, current deep learning-based defect sample generation methods require a certain number of labeled samples for supervised training. However, in actual production processes, it is usually difficult to obtain a sufficient number of defect samples that meet the requirements. Moreover, current defect sample generation can only be based on defect images of known categories, while in reality, target category defects are constantly being generated. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for generating defect samples, which can quickly generate a large number of defect samples and reduce the time and labor costs associated with manual collection and sample labeling. The specific solution is as follows:

[0005] In a first aspect, the present invention discloses a method for generating defect samples, comprising:

[0006] Construct a graphics generation module, and generate random graphics through the graphics generation module;

[0007] An initial image sample is obtained, and an image sample generation operation is performed based on the random graphic and the initial image sample to obtain the corresponding target defect sample;

[0008] Pixel-level labels corresponding to the target defect samples are generated to obtain training data containing the target defect samples and the pixel-level labels, so as to train the image defect detection model using the training data.

[0009] Optionally, the step of obtaining an initial image sample and performing an image sample generation operation based on the random graphic and the initial image sample to obtain a corresponding target defect sample includes:

[0010] Obtain normal samples or defective samples of any category label;

[0011] An image sample generation operation is performed based on the normal sample or the defect sample with any category label and the random graphic to obtain the corresponding target defect sample.

[0012] Optionally, the step of generating image samples based on the normal samples or defect samples with any category label and the random graphics to obtain corresponding target defect samples includes:

[0013] A defect synthesis module is constructed, and the defect synthesis module is used to perform image sample generation operations using the normal sample and the random graphic to obtain the corresponding target defect sample;

[0014] Alternatively, the defective sample of the normal sample or any category label, the random image, and the predetermined target category label are input into a pre-trained defective sample generator to obtain a target defective sample output by the defective sample generator with the specified category as the target category label.

[0015] Optionally, the step of generating image samples using the normal samples and the random graphics to obtain corresponding target defect samples includes:

[0016] A region is randomly cropped from the normal sample to obtain the corresponding region sample;

[0017] The region samples are subjected to various random perturbation processes to obtain perturbed region samples;

[0018] Perform a logical AND operation on the region sample and the perturbed region sample to obtain the corresponding synthesis result;

[0019] The synthesized result is randomly pasted onto the normal sample to obtain the corresponding target defect sample;

[0020] Accordingly, generating the pixel-level label corresponding to the target defect sample includes:

[0021] Based on the synthesis result, the random graphic is randomly pasted into the position of the normal sample and expanded into a pixel-level label corresponding to the target defect sample.

[0022] Optionally, the defective sample (either a normal sample or any category label), the random image, and a predetermined target category label are input into a pre-trained defective sample generator to obtain a target defective sample output by the defective sample generator, with the specified category being the target category label, including:

[0023] The defect sample of the normal sample or any category label, the random graphic, and the predetermined target category label are input into the first defect sample generator, which is pre-trained using the first preset training method, to obtain the target defect sample output by the first defect sample generator after training, whose specified category is the target category label.

[0024] The first defect sample generator after training is a defect sample generator obtained by training a first initial defect sample generator based on a first neural network using normal samples and defect samples with image-level classification labels and image-level category control parameters.

[0025] Optionally, the defective sample (either a normal sample or any category label), the random image, and a predetermined target category label are input into a pre-trained defective sample generator to obtain a target defective sample output by the defective sample generator, with the specified category being the target category label, including:

[0026] The defect sample of the normal sample or any category label, the random graphic and the predetermined target category label are input into the second defect sample generator which is pre-trained using the second preset training method, so as to obtain the target defect sample output by the trained second defect sample generator with the specified category as the target category label.

[0027] The trained second defect sample generator is a defect sample generator obtained by training a second initial defect sample generator based on a second neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters.

[0028] Optionally, the defect sample generation method includes:

[0029] Randomly select any pixel-level label from the pixel-level labels of the defect sample training set, and determine the random pixel-level label as the current target pixel-level label;

[0030] Accordingly, the step of inputting the defective sample (either a normal sample or any category label), the random image, and a predetermined target category label into a pre-trained defective sample generator to obtain a target defective sample output by the defective sample generator with the specified category as the target category label includes:

[0031] The defect sample of the normal sample or any of the category labels and the predetermined target category label are input into the third defect sample generator which is trained using the third preset training method. The target pixel-level label is used to replace the random image input into the third defect sample generator to obtain the target defect sample output by the trained third defect sample generator with the specified category as the target category label.

[0032] The trained third defect sample generator is a defect sample generator obtained by training a third initial defect sample generator based on a third neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters.

[0033] Secondly, the present invention discloses a defect sample generation apparatus, comprising:

[0034] A graphics generation module is used to construct a graphics generation module and generate random graphics through the graphics generation module;

[0035] The sample acquisition module is used to acquire initial image samples;

[0036] The defect sample generation module is used to perform image sample generation operations based on the random graphic and the initial image sample to obtain the corresponding target defect sample;

[0037] The label generation module is used to generate pixel-level labels corresponding to the target defect sample, and obtain training data containing the target defect sample and the pixel-level labels, so as to use the training data to train the image defect detection model.

[0038] Thirdly, the present invention discloses an electronic device, comprising:

[0039] Memory, used to store computer programs;

[0040] A processor is configured to execute the computer program to implement the steps of the aforementioned disclosed defect sample generation method.

[0041] Fourthly, the present invention discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed defect sample generation method.

[0042] As can be seen, the present invention provides a defect sample generation method, comprising: constructing a graphics generation module and generating random graphics through the graphics generation module; acquiring initial image samples and performing image sample generation operations based on the random graphics and the initial image samples to obtain corresponding target defect samples; generating pixel-level labels corresponding to the target defect samples to obtain training data containing the target defect samples and the pixel-level labels, so as to use the training data to train an image defect detection model. Therefore, this application, by synthesizing target defect samples based on random graphics generated by the graphics generation module and the acquired initial image samples, can solve the problem of difficulty in obtaining a sufficient number of defect samples that meet the requirements in actual generation, achieving rapid and large-scale generation of defect samples, and reducing the time and labor costs consumed by manual collection and sample labeling. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 This is a flowchart of a defect sample generation method disclosed in this invention;

[0045] Figure 2 This is a flowchart of a specific defect sample generation method disclosed in this invention;

[0046] Figure 3 This is a schematic diagram of an unsupervised defect sample generation method disclosed in this invention;

[0047] Figure 4 This is a schematic diagram of a specific unsupervised defect sample generation method disclosed in this invention;

[0048] Figure 5 This is a flowchart of another specific defect sample generation method disclosed in this invention;

[0049] Figure 6 This is a schematic diagram of a weakly supervised defect sample generation method disclosed in this invention;

[0050] Figure 7 This is a schematic diagram of a weakly supervised defect sample generation method for normal samples disclosed in this invention.

[0051] Figure 8 This is a schematic diagram of a weakly supervised defect sample generation method for defect samples disclosed in this invention;

[0052] Figure 9 This is a schematic diagram of a semi-supervised defect sample generation method disclosed in this invention;

[0053] Figure 10 This is a schematic diagram of a semi-supervised defect sample generation method for normal samples disclosed in this invention;

[0054] Figure 11 This is a schematic diagram of a semi-supervised defect sample generation method for defect samples disclosed in this invention;

[0055] Figure 12 This is a schematic diagram of another semi-supervised defect sample generation method for normal samples disclosed in this invention.

[0056] Figure 13 This is a schematic diagram of another semi-supervised defect sample generation method for defect samples disclosed in this invention.

[0057] Figure 14 This is a schematic diagram of a defect sample generation device disclosed in this invention;

[0058] Figure 15 This is a structural diagram of an electronic device disclosed in this invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Currently, most deep learning-based methods for detecting appearance defects require manual collection and labeling of a large number of defect samples for training, which is extremely time-consuming and labor-intensive. In actual production processes, it is often difficult to obtain a sufficient number of defect samples that meet the requirements, and current defect sample generation can only be based on defect images of known categories. To address this, this invention provides a defect sample generation scheme that can quickly generate a large number of defect samples and reduce the time and labor costs associated with manual collection and labeling.

[0061] This invention discloses a method for generating defect samples, see [link to relevant documentation]. Figure 1 As shown, the method includes:

[0062] Step S11: Construct a graphics generation module and generate random graphics through the graphics generation module.

[0063] In this embodiment, a graphic generation module is constructed, and random graphics are generated through the graphic generation module. For example, a graphic generation module F is constructed, which can generate various random graphics x. f For example, random fractal graphics, random Bezier graphics, Perlin noise graphics, and combinations thereof, etc., and the construction of the graphics generation module can be implemented in various ways without specific limitations. For example, random fractal graphics can be generated through the graphics generation module, and the Mandelbrot set and Julia set in the fractal graphics can be realized by constructing an iterative function system: Here, z and c both belong to the set of complex numbers. According to chaos theory, even a small change in the initial value of this iterative function system will cause a huge difference in the iterative process, thus allowing the generation of infinitely varied fractal patterns. Similarly, random Bezier patterns and Perlin noise patterns can be generated using their respective formulas. These random patterns can also be binarized to obtain a binarized random pattern x. f .

[0064] Step S12: Obtain an initial image sample, and perform an image sample generation operation based on the random graphic and the initial image sample to obtain the corresponding target defect sample.

[0065] In this embodiment, after generating random images through the graphics generation module, initial image samples are obtained, and image sample generation operations are performed based on the random graphics and the initial image samples to obtain corresponding target defect samples. Specifically, normal samples or defect samples with any category label are obtained; image sample generation operations are performed based on the normal samples or defect samples with any category label and the random graphics to obtain corresponding target defect samples. It can be understood that in this embodiment, the generation of target defect samples can use only normal samples and random graphics, without the need for manual collection of defect samples and labeling, thereby achieving unsupervised defect sample generation; or it can use normal samples or defect samples with any category label and random graphics, without the need for manual annotation of pixel-level labels on defect samples, only image-level classification labels are needed, thereby achieving weakly supervised defect sample generation; or it can use manually annotated pixel-level labels on defect samples to achieve semi-supervised defect sample generation. In other words, target defect samples can be generated in an unsupervised manner, using only normal samples to generate new defect samples without the need for manual collection of defect samples and labeling. Alternatively, target defect samples can be generated in a weakly supervised manner, using normal samples or defect samples with image-level labels to generate new defect samples of a specified category. Or, target defect samples can be generated in a semi-supervised manner, using normal samples or defect samples with pixel-level labels to generate new defect samples of a specified category at a specified location.

[0066] Step S13: Generate pixel-level labels corresponding to the target defect sample to obtain training data containing the target defect sample and the pixel-level labels, so as to use the training data to train the image defect detection model.

[0067] In this embodiment, when performing image sample generation operations based on the random graphics and the initial image samples to obtain the corresponding target defect samples, pixel-level labels corresponding to the target defect samples are generated simultaneously to obtain training data containing the target defect samples and the pixel-level labels. This training data is then used to train the image defect detection model. Generating the corresponding pixel-level labels simultaneously with the target defect samples avoids the need for manual labeling of defects at the pixel level.

[0068] Therefore, in this embodiment of the application, by synthesizing target defect samples based on random graphics generated by the graphics generation module and the obtained initial image samples, the problem of difficulty in obtaining a sufficient number of defect samples that meet the requirements in actual generation can be solved, thereby realizing the rapid and large-scale generation of defect samples and reducing the time and labor costs of manual collection and sample labeling.

[0069] See Figure 2 As shown, this embodiment of the invention discloses a specific method for generating defect samples. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.

[0070] Step S21: Construct a graphics generation module and generate random graphics through the graphics generation module.

[0071] Step S22: Obtain normal samples.

[0072] It should be noted that the normal samples mentioned are unlabeled normal samples.

[0073] Step S23: Construct a defect synthesis module, and use the normal sample and the random graphic to perform image sample generation operation to obtain the corresponding target defect sample.

[0074] Step S24: Generate pixel-level labels corresponding to the target defect sample to obtain training data containing the target defect sample and the pixel-level labels, so as to use the training data to train the image defect detection model.

[0075] It is understandable that the constructed defect synthesis module synthesizes target defect samples using normal samples and random graphics generated by the graphics generation module. For example, see... Figure 3 As shown, a graph generation module F is constructed to generate random graphs x. fThen, through the constructed defect synthesis module S, normal sample x is synthesized. nor and random graph x f The input is fed into the defect synthesis module S to synthesize the target defect sample, that is, to synthesize a new defect sample y. def And generate its corresponding pixel-level label y m .

[0076] In this embodiment, the target defect sample can be generated using only unlabeled normal samples, without the need for manual collection of defect samples and creation of labels, thereby enabling unsupervised defect sample generation.

[0077] In the process of generating image samples using the normal sample and the random graphic through the defect synthesis module to obtain the corresponding target defect sample, the process may specifically include: randomly cropping a region from the normal sample to obtain a corresponding region sample; performing various random perturbations on the region sample to obtain a perturbed region sample; performing a logical AND operation on the region sample and the perturbed region sample to obtain a corresponding synthesis result; randomly pasting the synthesis result onto the normal sample to obtain the corresponding target defect sample; and expanding the random graphic into a pixel-level label corresponding to the target defect sample based on the position of the synthesis result randomly pasted onto the normal sample. For example, see... Figure 4 As shown, a defect synthesis module S is constructed, which takes a normal sample x as input. nor and random graph x f Perform the following operation: randomly select from the normal sample x nor By cropping a region, we obtain the region sample x. p And by constructing a random perturbation module for the region sample x p Various random perturbations are applied with a certain probability, including random color perturbation, random rotation, random scaling, and adding random noise. Then, the perturbated x is processed. p and random graph x f Perform a logical AND operation to synthesize the corresponding result y. p Then, the result y obtained after the logical AND operation is... p Randomly paste back to normal sample x nor The target defect sample y is obtained from the above. def And based on the randomly pasted location, the random graphic x f Expanded to target defect sample y def The corresponding pixel-level label y m .

[0078] For details regarding step S21, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0079] Therefore, in this embodiment of the application, the target defect sample is synthesized by combining the random graphic generated by the graphic generation module and the obtained normal sample. This process only uses normal samples to generate the target defect sample, without the need for manual collection of defect samples and labeling. This achieves the generation of defect samples in an unsupervised manner and solves the problem of obtaining a sufficient number of defect samples that meet the requirements in actual generation. It enables the rapid and large-scale generation of defect samples and reduces the time and labor costs of manual collection and sample labeling.

[0080] See Figure 5 As shown, this embodiment of the invention discloses a specific method for generating defect samples. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.

[0081] Step S31: Construct a graphics generation module and generate random graphics through the graphics generation module.

[0082] Step S32: Obtain normal samples or defective samples of any category label.

[0083] Step S33: Input the defect sample of the normal sample or any category label, the random image, and the predetermined target category label into the pre-trained defect sample generator to obtain the target defect sample output by the defect sample generator with the specified category as the target category label.

[0084] Step S34: Generate pixel-level labels corresponding to the target defect sample to obtain training data containing the target defect sample and the pixel-level labels, so as to use the training data to train the image defect detection model.

[0085] In this embodiment, a normal sample or a defect sample with any category label, a random graphic, and a pre-determined target category label are input to a trained defect sample generator, which can then generate and output a target defect sample with the specified category as the target category label.

[0086] In a first specific implementation, the defect sample (either a normal sample or a defect sample with any category label), the random image, and a predetermined target category label are input into a first defect sample generator trained using a first preset training method. This results in a target defect sample output by the trained first defect sample generator, with the specified category being the target category label. The trained first defect sample generator is a defect sample generator trained using normal samples and defect samples with image-level classification labels and image-level category control parameters on a first initial defect sample generator constructed based on a first neural network. It is understood that the training samples for training the first defect sample generator are normal samples and defect samples with image-level classification labels. Through the trained first defect sample generator, a target defect sample of a specified category can be generated using normal samples or defect samples with any category label, a random image, and a predetermined target category label. That is, using normal samples or defect samples with any category label and a random image, pixel-level labeling of the defect samples is not required manually; only image-level classification labels are needed, thus achieving weakly supervised defect sample generation. For example, see [link to relevant documentation]. Figure 6 As shown, a defect sample generator G is trained, and the training samples are normal samples and defect samples with image-level classification labels. The normal sample x... nor or defective sample x def The random graphic x generated by the graphic generation module F f The image-level category control parameter c (i.e., the pre-determined target category label) is input into the trained defect sample generator G to obtain a target defect sample of category c. and the corresponding pixel-level tags

[0087] For example, for a normal sample x nor See Figure 7 As shown, the normal sample x nor The random graphic x generated by the graphic generation module m And the image-level category control parameter c is input into the defect sample generator G, which can generate pixel-level labels x. m Generate a target defect sample of category c at the location. and the corresponding pixel-level tags For defect samples of category n (Regardless of whether or not it has image-level classification labels), see Figure 8 As shown, this refers to defect samples of category n. The random graphic x generated by the graphic generation module F m The image-level category control parameter c is input into the defect sample generator G, which can generate target defect samples of category c. and the corresponding pixel-level tags In other words, the generated target defect sample Normal areas and defect samples The normal region is the same, and the defect sample generator G is in the target defect sample. China will fix defective samples The defective area, in the pixel-level label x m The location generates a defect of category c. Therefore, this process does not require manual pixel-level labeling of defects, only image-level classification labels, making it a weakly supervised method.

[0088] It should be noted that there are various possible models for constructing the first defect sample generator G, and no specific restrictions are imposed here. For example, the model can employ variational autoencoders, generative adversarial networks (GANs), streaming models, diffusion models, etc. Taking a GAN as an example, the training of the defect sample generator G can specifically include: constructing a defect sample generator G and a discriminator D using a neural network, where the discriminator D contains two classification heads Di. gan and D cls During the discriminator training phase, the discriminator D is trained. gan For normal sample x nor Defect samples and new defect samples Perform a true / false distinction and train the discriminator D. cls right The classification is performed, and the discriminator loss function is as follows:

[0089]

[0090] Where E represents the mean.

[0091] During the training phase of the defect sample generator G, the defect sample generator G is trained to reconstruct the pixel-level labels of the input. Generate sufficiently realistic target defect samples of category c, where the defect sample generator loss function is:

[0092]

[0093] Where ||||2 represents the root mean square.

[0094] Using any neural network training algorithm, such as stochastic gradient descent, the defect sample generator G and a discriminator D are trained until convergence reaches the preset performance. Then, the discriminator is discarded, and the defect sample generator G alone can be used to complete the weakly supervised defect sample generation.

[0095] In a second specific implementation, the defect sample (either a normal sample or a defect sample with any category label), the random image, and the predetermined target category label are input into a second defect sample generator trained using a second preset training method. This results in a target defect sample output by the trained second defect sample generator, with the specified category being the target category label. The trained second defect sample generator is a defect sample generator obtained by training a second initial defect sample generator constructed based on a second neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters. It is understood that the training samples for training the second defect sample generator are normal samples and defect samples with pixel-level classification labels. Through the trained second defect sample generator, a target defect sample of the specified category can be generated using normal samples or defect samples with any category label, random images, and the predetermined target category label. That is, using normal samples or defect samples with any category label and random images requires manual annotation of the pixel-level labels on the defect samples, thus achieving semi-supervised defect sample generation. For example, see [link to documentation]. Figure 9 As shown, a defect sample generator G is trained, and the training samples are normal samples and defect samples with pixel-level classification labels. The normal sample x... nor or defective sample x def The random graphic x generated by the graphic generation module F f and pixel-level category control parameter c m Input defect sample generator G m The target defect sample of category c is obtained. and the corresponding pixel-level tags

[0096] For example, for a normal sample x nor See Figure 10 As shown, the normal sample x nor The random graphic x generated by the graphic generation module m And category label c input defect sample generator G m It can be used in pixel-level labels x m Generate a new defect sample of category c at the location. and the corresponding pixel-level tags For defect samples of category n (with pixel-level tags) See also Figure 11 As shown, the random graphic x generated by the graphic generation module m And category label c input defect sample generator G m It can generate new defect samples of category c. and the corresponding pixel-level tags In other words, the newly generated defect samples Normal areas and defect samples The normal region is the same, and the defect sample generator G generates new defect samples. China will fix defective samples The defective area, in x m The location generates a defect of category c. This indicates that the process uses manually labeled, pixel-level tags for defects, representing a semi-supervised approach.

[0097] It should be noted that, similar to the weakly supervised approach, the model constituting the second defect sample generator G can have multiple schemes, and no specific restrictions are imposed here. If a generative adversarial network is used, its training instances are also similar to those described in the weakly supervised approach.

[0098] In the third specific implementation, any pixel-level label is randomly selected from the pixel-level labels of the defect sample training set, and the random pixel-level label is determined as the current target pixel-level label; the defect sample of the normal sample or the defect sample of any category label and the predetermined target category label are input into the third defect sample generator obtained by training using the third preset training method, and the target pixel-level label is used to replace the random image input into the third defect sample generator to obtain the target defect sample output by the trained third defect sample generator with the specified category as the target category label; wherein, the trained third defect sample generator is a defect sample generator obtained by training the third initial defect sample generator based on the third neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters. Understandably, the training samples used to train the third defect sample generator are also normal samples and defect samples with pixel-level classification labels. After training, the third defect sample generator can generate target defect samples of a specified category using normal samples or defect samples with any category label, selected target pixel-level labels, and pre-determined target category labels. That is, using normal samples or defect samples with any category label and target pixel-level labels, the pixel-level labels of the defect samples need to be manually annotated, thus achieving semi-supervised defect sample generation. For example, for a normal sample x... nor See Figure 12 As shown, a pixel-level label is randomly selected from the pixel-level labels of the defect sample training set. The pixel-level label and normal sample x nor And category label k input defect sample generator G m It can be used for pixel-level tags. Generate a new defect sample of category k at the location. and the corresponding pixel-level tags

[0099] For defect samples of category n (with pixel-level tags) See also Figure 13 As shown, a pixel-level label is randomly selected from the pixel-level labels of the defect sample training set. The pixel-level label and defect samples And category label k input defect sample generator G m It can generate new defect samples of category k. and the corresponding pixel-level tags In other words, the newly generated defect samples Normal areas and defect samples The normal region is the same, and the defect sample generator G generates new defect samples. China will fix defective samples The defective area, in the pixel-level label A defect of category k is generated at the given location. This indicates that the process uses manually labeled, pixel-level tags for defects, representing a semi-supervised approach.

[0100] It should be noted that, similar to the weakly supervised approach described above, the model constituting the third defect sample generator G can have multiple implementations, and no specific restrictions are imposed here. If a generative adversarial network is used, its training instances are also similar to those in the weakly supervised approach described above.

[0101] For details regarding steps S31 to S32 and S34, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0102] Therefore, in this embodiment of the application, target defect samples are synthesized by using random graphics generated by the graphics generation module and the obtained initial image samples through weak supervision or semi-supervision. This enables the generation of a large number of defect samples under different data annotation conditions, which has high flexibility and adaptability. This solves the problem of not being able to obtain a sufficient number of defect samples that meet the requirements in actual generation, and enables the rapid generation of a large number of defect samples, while reducing the time and labor costs of manual collection and sample annotation.

[0103] Accordingly, embodiments of the present invention also disclose a defect sample generation device, see [link to relevant documentation]. Figure 14 As shown, the device includes:

[0104] The graphic generation module 11 is used to construct the graphic generation module and generate random graphics through the graphic generation module;

[0105] Sample acquisition module 12 is used to acquire initial image samples;

[0106] The defect sample generation module 13 is used to perform image sample generation operations based on the random graphic and the initial image sample to obtain the corresponding target defect sample;

[0107] The label generation module 14 is used to generate pixel-level labels corresponding to the target defect sample, and obtain training data containing the target defect sample and the pixel-level labels, so as to use the training data to train the image defect detection model.

[0108] As can be seen from the above, in this embodiment of the invention, by synthesizing target defect samples based on random graphics generated by the graphics generation module and the obtained initial image samples, the problem of difficulty in obtaining a sufficient number of defect samples that meet the requirements in actual generation can be solved, thereby achieving rapid and large-scale generation of defect samples and reducing the time and labor costs of manual collection and sample labeling.

[0109] In some specific embodiments, the sample acquisition module 12 may specifically include:

[0110] The sample acquisition submodule is used to acquire normal samples or defective samples of any category label;

[0111] In some specific embodiments, the defect sample generation module 13 may specifically include:

[0112] The defect sample generation submodule is used to perform image sample generation operations based on the normal sample or the defect sample with any category label and the random graphic to obtain the corresponding target defect sample.

[0113] In some specific embodiments, the defect sample generation submodule may specifically include:

[0114] A synthesis module building unit, used to build a defect synthesis module;

[0115] The first defect sample generation unit is used to perform image sample generation operations using the normal sample and the random graphic through the defect synthesis module to obtain the corresponding target defect sample.

[0116] Alternatively, a second defect sample generation unit is used to input the normal sample or the defect sample with any category label, the random graphic, and the predetermined target category label into a pre-trained defect sample generator to obtain a target defect sample output by the defect sample generator with the specified category as the target category label.

[0117] In some specific embodiments, the first defect sample generation unit may specifically include:

[0118] The sample cropping subunit is used to randomly crop a region from the normal sample to obtain the corresponding region sample.

[0119] The perturbation processing subunit is used to perform various random perturbation processes on the region sample to obtain the perturbated region sample.

[0120] The operation processing subunit is used to perform a logical AND operation on the region sample and the perturbed region sample to obtain the corresponding synthesis result;

[0121] The first defect sample generation subunit is used to randomly paste the synthesis result onto the normal sample to obtain the corresponding target defect sample;

[0122] In some specific embodiments, the label generation module 14 may specifically include:

[0123] The label determination submodule is used to expand the random graphic into a pixel-level label corresponding to the target defect sample by randomly pasting it to the position of the normal sample according to the synthesis result.

[0124] In some specific embodiments, the second defect sample generation unit may specifically include:

[0125] The second defect sample generation subunit is used to input the defect sample of the normal sample or any category label, the random graphic, and the predetermined target category label into the first defect sample generator obtained by training using the first preset training method, so as to obtain the target defect sample output by the trained first defect sample generator with the specified category as the target category label.

[0126] The first defect sample generator after training is a defect sample generator obtained by training a first initial defect sample generator based on a first neural network using normal samples and defect samples with image-level classification labels and image-level category control parameters.

[0127] In some specific embodiments, the second defect sample generation unit may specifically include:

[0128] The third defect sample generation subunit is used to input the defect sample of the normal sample or any category label, the random graphic and the predetermined target category label into the second defect sample generator which is pre-trained using the second preset training method, so as to obtain the target defect sample output by the trained second defect sample generator with the specified category as the target category label.

[0129] The trained second defect sample generator is a defect sample generator obtained by training a second initial defect sample generator based on a second neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters.

[0130] In some specific embodiments, the defect sample generation device may further include:

[0131] The label selection unit is used to randomly select any pixel-level label from the pixel-level labels of the defect sample training set, and determine the any pixel-level label as the current target pixel-level label;

[0132] In some specific embodiments, the second defect sample generation unit may specifically include:

[0133] The third defect sample generation subunit is used to input the defect sample of the normal sample or any of the category labels and the predetermined target category label into the third defect sample generator obtained by training using the third preset training method, and to use the target pixel-level label to replace the random image input into the third defect sample generator, so as to obtain the target defect sample output by the trained third defect sample generator with the specified category as the target category label.

[0134] The trained third defect sample generator is a defect sample generator obtained by training a third initial defect sample generator based on a third neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters.

[0135] Furthermore, embodiments of the present invention also provide an electronic device. Figure 15 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of the invention.

[0136] Figure 15 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present invention. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the defect sample generation method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0137] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0138] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0139] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the defect sample generation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0140] Furthermore, this embodiment of the invention also discloses a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the defect sample generation method steps disclosed in any of the foregoing embodiments.

[0141] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0142] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0143] The above provides a detailed description of a defect sample generation method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for generating defect samples, characterized in that, include: Construct a graphics generation module, and generate random graphics through the graphics generation module; An initial image sample is obtained, and an image sample generation operation is performed based on the random graphic and the initial image sample to obtain the corresponding target defect sample; Pixel-level labels corresponding to the target defect samples are generated to obtain training data containing the target defect samples and the pixel-level labels, so as to train the image defect detection model using the training data; The step of obtaining an initial image sample and performing an image sample generation operation based on the random graphic and the initial image sample to obtain a corresponding target defect sample includes: obtaining a normal sample or a defect sample with any category label; and performing an image sample generation operation based on the normal sample or the defect sample with any category label and the random graphic to obtain a corresponding target defect sample. The step of generating an image sample based on the defect sample (either the normal sample or any category label) and the random graphic to obtain a corresponding target defect sample includes: constructing a defect synthesis module and using the defect synthesis module to generate an image sample using the normal sample and the random graphic to obtain a corresponding target defect sample; or, inputting the defect sample (either the normal sample or any category label) and the random graphic, along with a predetermined target category label, into a pre-trained defect sample generator to obtain a target defect sample output by the defect sample generator with the specified category being the target category label.

2. The defect sample generation method according to claim 1, characterized in that, The step of generating image samples using the normal samples and the random graphics to obtain corresponding target defect samples includes: A region is randomly cropped from the normal sample to obtain the corresponding region sample; The region samples are subjected to various random perturbation processes to obtain perturbed region samples; Perform a logical AND operation on the region sample and the perturbed region sample to obtain the corresponding synthesis result; The synthesized result is randomly pasted onto the normal sample to obtain the corresponding target defect sample; Accordingly, generating the pixel-level label corresponding to the target defect sample includes: Based on the synthesis result, the random graphic is randomly pasted into the position of the normal sample and expanded into a pixel-level label corresponding to the target defect sample.

3. The defect sample generation method according to claim 1, characterized in that, The step of inputting the defect sample (either a normal sample or any category label), the random image, and a predetermined target category label into a pre-trained defect sample generator to obtain a target defect sample output by the defect sample generator with the specified category as the target category label includes: The defect sample of the normal sample or any category label, the random graphic, and the predetermined target category label are input into the first defect sample generator, which is pre-trained using the first preset training method, to obtain the target defect sample output by the first defect sample generator after training, whose specified category is the target category label. The first defect sample generator after training is a defect sample generator obtained by training a first initial defect sample generator based on a first neural network using normal samples and defect samples with image-level classification labels and image-level category control parameters.

4. The defect sample generation method according to claim 1, characterized in that, The defect sample, which is either a normal sample or a defect sample with any category label, along with the random image and a predetermined target category label, are input into a pre-trained defect sample generator to obtain a target defect sample output by the defect sample generator, whose specified category is the target category label. This includes: The defect sample of the normal sample or any category label, the random graphic and the predetermined target category label are input into the second defect sample generator which is pre-trained using the second preset training method, so as to obtain the target defect sample output by the trained second defect sample generator with the specified category as the target category label. The trained second defect sample generator is a defect sample generator obtained by training a second initial defect sample generator based on a second neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters.

5. The defect sample generation method according to any one of claims 1 to 4, characterized in that, Also includes: Randomly select any pixel-level label from the pixel-level labels of the defect sample training set, and determine the random pixel-level label as the current target pixel-level label; Accordingly, the step of inputting the defective sample (either a normal sample or any category label), the random image, and a predetermined target category label into a pre-trained defective sample generator to obtain a target defective sample output by the defective sample generator with the specified category as the target category label includes: The defect sample of the normal sample or any of the category labels and the predetermined target category label are input into the third defect sample generator which is trained using the third preset training method. The target pixel-level label is used to replace the random image input into the third defect sample generator to obtain the target defect sample output by the trained third defect sample generator with the specified category as the target category label. The trained third defect sample generator is a defect sample generator obtained by training a third initial defect sample generator based on a third neural network using normal samples and defect samples with pixel-level classification labels and pixel-level category control parameters.

6. A defect sample generation device, characterized in that, include: A graphics generation module is used to construct a graphics generation module and generate random graphics through the graphics generation module; The sample acquisition module is used to acquire initial image samples; The defect sample generation module is used to perform image sample generation operations based on the random graphic and the initial image sample to obtain the corresponding target defect sample; The label generation module is used to generate pixel-level labels corresponding to the target defect sample, and obtain training data containing the target defect sample and the pixel-level labels, so as to use the training data to train the image defect detection model; Specifically, the sample acquisition module is used to acquire normal samples or defective samples of any category label; Specifically, the defect sample generation module is used to perform image sample generation operations based on the normal sample or the defect sample with any category label and the random graphic to obtain the corresponding target defect sample; Specifically, the defect sample generation module is used to construct a defect synthesis module, and to perform image sample generation operations using the normal sample and the random image to obtain the corresponding target defect sample; or, the defect sample with the normal sample or any category label, the random image, and the predetermined target category label are input into a pre-trained defect sample generator to obtain a target defect sample output by the defect sample generator with the specified category as the target category label.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the defect sample generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the defect sample generation method as described in any one of claims 1 to 5.

Images