New type product defect image generation method, device and equipment and storage medium

Abstract

The invention discloses a novel-model product defect image generation method and device, equipment and a storage medium. The method comprises the steps that an old-model product defect image sample and a novel-model product defect-free image sample are acquired; repairing each old model product defect image in the old model product defect image sample to obtain an old model product defect-free image sample, and determining an old model product defect difference image sample between the old model product defect-free image sample and the old model product defect image sample; according to a preset defect difference value image generation model, a simulation defect difference value image sample of the mask in any shape is generated, and the preset defect difference value image generation model is obtained through training according to an old model product defect difference value image sample; and superposing the simulation defect difference image sample and the new-type product defect-free image sample to obtain a new-type product defect image sample. The defect detection precision and the detection efficiency are improved.

Description

Technical Field

[0001] The present application relates to the technical field of defective image generation, and particularly to a method, device, equipment and storage medium for generating defective images of new model products. Background Art

[0002] With the continuous development of deep learning, tasks such as image classification, object detection, and object segmentation have made leapfrog progress. The application of deep learning technology in the field of machine vision has enabled defect detection to achieve higher-precision detection results compared with traditional detection algorithms. The training of neural network models based on deep learning technology usually requires obtaining a large number of training set samples. In many actual industrial production scenarios, the scenario of changing the model of products on the production line is very common. Although there are a large number of defective samples collected from old model products, the number of defective samples on new model products is often limited, or even difficult to obtain quickly, which seriously restricts the defect detection accuracy based on deep learning.

[0003] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] Embodiments of the present application provide a method, device, equipment and storage medium for generating defective images of new model products, aiming to solve the technical problem of low defect detection accuracy caused by too few defective samples of new model products after product model replacement in existing industrial scenarios.

[0005] Embodiments of the present application provide a method for generating defective images of new model products. The method for generating defective images of new model products includes:

[0006] Obtain defective image samples of old model products and defect-free image samples of new model products;

[0007] Repair each defective image of the old model product in the defective image sample of the old model product to obtain a defect-free image sample of the old model product, and determine a defective difference image sample between the defect-free image sample of the old model product and the defective image sample of the old model product;

[0008] Generate a simulated defective difference image sample with an arbitrarily shaped mask according to a preset defective difference image generation model, and the preset defective difference image generation model is trained according to the defective difference image sample of the old model product;

[0009] Overlay the simulated defective difference image sample and the defect-free image sample of the new model product to obtain a defective image sample of the new model product.

[0010] Optionally, the steps of repairing each defective image of the old model product in the defective image sample of the old model product to obtain a defect-free image sample of the old model product and determining a defective difference image sample of the old model product between the defect-free image sample of the old model product and the defective image sample of the old model product include:

[0011] Extract the defect positions and defect shapes corresponding to each defective image of the old model product in the defective image sample of the old model product;

[0012] Obtain a defect mask sample according to the defect positions and the defect shapes corresponding to each defective image of the old model product;

[0013] Input the defective image sample of the old model product and the defect mask sample into a preset image completion model to obtain a defect-free image sample of the old model product;

[0014] Subtract each defect-free image of the old model product in the defect-free image sample of the old model product from the corresponding defective image of the old model product in the defective image sample of the old model product to obtain each defective difference image of the old model product;

[0015] Combine each defective difference image of the old model product to obtain the defective difference image sample of the old model product.

[0016] Optionally, the steps of inputting the defective image sample of the old model product and the defect mask sample into a preset image completion model to obtain a defect-free image sample of the old model product include:

[0017] Replace the defect regions corresponding to each defect mask in the defect mask sample with the image background respectively to obtain a defect-free image of the old model product corresponding to each defect mask;

[0018] Combine each defect-free image of the old model product to obtain the defect-free image sample of the old model product.

[0019] Optionally, before the step of generating a simulated defective difference image sample with an arbitrary shape mask according to a preset defective difference image generation model, it further includes:

[0020] Perform model training according to the defect mask sample and the defective difference image sample of the old model product to obtain a preset defective difference image generation model.

[0021] Optionally, the steps of performing model training according to the defect mask sample and the defective difference image sample of the old model product to obtain a preset defective difference image generation model include:

[0022] Input the defect mask sample into a defective difference generator to obtain an initial defective difference image sample;

[0023] Input the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain an objective function;

[0024] Train the objective function through an optimizer to obtain the preset defect difference image generation model.

[0025] Optionally, the step of inputting the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain an objective function includes:

[0026] Input the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain the defect difference authenticity probability;

[0027] Determine the adversarial loss, L1 loss, and feature matching loss according to the defect difference authenticity probability;

[0028] Determine the objective function according to the adversarial loss, the L1 loss, and the feature matching loss.

[0029] Optionally, the step of training the objective function through an optimizer to obtain the preset defect difference image generation model includes:

[0030] Use an optimizer to perform optimization training on the objective function;

[0031] When the number of training times reaches the preset number of times, obtain the preset defect difference sample generation model.

[0032] In addition, to achieve the above object, the present application also provides a generation device for defect images of a new model product, including:

[0033] An acquisition module, configured to acquire a defect image sample of an old model product and a defect-free image sample of a new model product;

[0034] A repair and difference module, configured to repair each defect image of the old model product defect image sample to obtain a defect-free image sample of the old model product, and determine an old model product defect difference image sample between the defect-free image sample of the old model product and the defect image sample of the old model product;

[0035] A generation module, configured to generate a simulation defect difference image sample with an arbitrarily shaped mask according to a preset defect difference image generation model, where the preset defect difference image generation model is trained according to the old model product defect difference image sample;

[0036] An overlay module, configured to overlay the simulation defect difference image sample and the defect-free image sample of the new model product to obtain a defect image sample of the new model product.

[0037] In addition, to achieve the above object, the present application further provides a generating device for defect images of a new model product, including: a memory, a processor, and a generating program for defect images of a new model product stored on the memory and executable on the processor. When the generating program for defect images of a new model product is executed by the processor, the steps of the above-mentioned generating method for defect images of a new model product are implemented.

[0038] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which a generating program for defect images of a new model product is stored. When the generating program for defect images of a new model product is executed by a processor, the steps of the above-mentioned generating method for defect images of a new model product are implemented.

[0039] The technical solution of a generating method, device, equipment, and storage medium for defect images of a new model product provided in the embodiments of the present application is as follows: by obtaining defect image samples of an old model product and defect-free image samples of a new model product; by repairing each defect image in the defect image samples of the old model product into a defect-free image of the old model product, obtaining a defect difference image sample between the defect-free image samples of the old model product and the defect image samples of the old model product; performing arbitrary-shaped defect generation according to a preset defect difference image generation model to obtain a simulated defect difference image sample; and superimposing the obtained simulated defect difference image sample on the defect-free image samples of the new model product to obtain defect image samples of the new model product. By augmenting the defect sample image set of the new product model with a large number of simulated defect sample images, the number of defect image samples of the new model product is increased, and the increased number of defect image samples of the new model product can be used to detect and identify target detection images, thereby improving the defect detection accuracy of the new product model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic flowchart of the first embodiment of the generating method for defect images of a new model product of the present application;

[0041] Figure 2 It is a schematic flowchart of the second embodiment of the generating method for defect images of a new model product of the present application;

[0042] Figure 3 It is a schematic flowchart of the third embodiment of the generating method for defect images of a new model product of the present application;

[0043] Figure 4 It is a schematic flowchart of the fifth embodiment of the generating method for defect images of a new model product of the present application;

[0044] Figure 5 It is a schematic flowchart of the sixth embodiment of the generating method for defect images of a new model product of the present application;

[0045] Figure 6 It is a functional block diagram of the device for generating defect images of the new model product of this application;

[0046] Figure 7 It is a schematic structural diagram of the hardware operating environment involved in the solution of the embodiment of this application.

[0047] The realization of the purpose of this application, functional features and advantages will be further described in conjunction with the embodiments with reference to the accompanying drawings. The above-mentioned accompanying drawings are only diagrams of one embodiment and not all of the invention. Specific embodiments

[0048] To better understand the above technical solution, the exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0049] To solve the above problems, this application proposes a method for generating defect images of new model products, including: obtaining defect image samples of old model products and defect-free image samples of new model products; repairing each defect image of the old model product in the defect image sample of the old model product to obtain a defect-free image sample of the old model product, and determining the defect difference image sample between the defect-free image sample of the old model product and the defect image sample of the old model product; generating a simulated defect difference image sample with an arbitrarily shaped mask according to a preset defect difference image generation model, and the preset defect difference image generation model is trained according to the defect difference image sample of the old model product; superimposing the simulated defect difference image sample and the defect-free image sample of the new model product to obtain a defect image sample of the new model product.

[0050] In the above manner, according to the preset defect difference sample generation model and the defect-free image of the new product model, defects are generated to obtain the simulated defect sample of the new product model. By augmenting the defect sample image set with a large number of simulated defect sample images of the new model product, the defect map of the new model product can be quickly obtained. Using the increased number of defect samples of the new model product to detect and identify the target detection image of the new model product can improve the accuracy during defect detection and enhance the detection efficiency.

[0051] As Figure 1 shown, in the first embodiment of this application, the method for generating defect images of the new model product of this application includes the following steps:

[0052] Step S110: Obtain defective image samples of old model products and defect-free image samples of new model products.

[0053] In this embodiment, the defect-free image samples of new model products include multiple defect-free images of new model products. A defect-free image of a new model refers to a defect-free image of a new model product that needs to undergo defect simulation.

[0054] Step S120: Repair each defective image of the old model product in the defective image samples of the old model product to obtain defect-free image samples of the old model product, and determine the defective difference image samples of the old model product between the defect-free image samples of the old model product and the defective image samples of the old model product.

[0055] In this embodiment, each defective image of the old model product in the defective image samples of the old model product is complemented according to a preset image completion model to obtain defect-free image samples of the old model product and defective difference image samples of the old model product between the defective image samples of the old model product.

[0056] In this embodiment, first, a pre-trained image restoration model based on the ImageNet dataset obtained according to the DDNM algorithm (Zero-shot image restoration using denoising diffusion null-space model) is obtained. Given a source image and its corresponding mask image, this algorithm can automatically complete the source image area corresponding to the mask according to the image background. In this application, a mask image corresponding to the shape and position of the defective image of the old model product is obtained based on the defective image of the old model product, and on this basis, the DDNM preset image completion model is used to infer the defective image of the old model product and its corresponding defect mask to obtain a defect-free image of the old model product.

[0057] Step S130: Generate simulated defective difference image samples with masks of any shape according to a preset defective difference image generation model, and the preset defective difference image generation model is trained according to the defective difference image samples of the old model product.

[0058] In this embodiment, the preset defective difference image generation model refers to a simulated defective difference image obtained by learning and training the defective images of the old model product through a convolutional neural network.

[0059] In this embodiment, the sizes of the defect-free images of the new model product and the simulated defective difference images are both 256x256, and they can also be other sizes, but this embodiment takes 256x256 as an example for illustration.

[0060] In this embodiment, the possible defect positions and defect shapes of each simulated defect difference image in the simulated defect difference image sample can be extracted to obtain a defect mask; the defect mask is input into a preset defect difference image generation model to generate a simulated defect difference image sample with a mask of any shape.

[0061] Step S140, superimpose the simulated defect difference image sample and the defect-free image sample of the new model product to obtain a defect image sample of the new model product.

[0062] According to the above technical solution, in this embodiment, a simulated defect difference image sample is generated according to a preset defect difference image generation model, and then superimposed with the defect-free image sample of the new model product to obtain a defect image sample of the new model product. The defect sample image set of the new model product is augmented by a large number of defect images of the new model product, and subsequent detection and recognition of the target detection image using the increased number of defect samples of the new model product can improve the defect detection efficiency and accuracy of the new model product.

[0063] Further, based on the first embodiment, with reference to Figure 2 , in the second embodiment of the present application, step S120 includes:

[0064] Step S121, extract the defect positions and defect shapes corresponding to each defect image of the old model product defect image sample.

[0065] Step S122, obtain a defect mask sample according to the defect positions and the defect shapes corresponding to each defect image of the old model product.

[0066] In this embodiment, each defect image in the old model product defect image sample is respectively subjected to image preprocessing to obtain a defect mask corresponding to each defect image of the old model product.

[0067] It should be noted that performing image preprocessing on the old model product defect image refers to performing defect annotation on the defect image to obtain a binary defect mask.

[0068] In this embodiment, the old model product defect image includes a defect area and a non-defect area, the pixel value corresponding to the defect area is 255, and the pixel value corresponding to the non-defect area is 0.

[0069] Step S123, input the old model product defect image sample and the defect mask sample into a preset image completion model to obtain a defect-free image sample of the old model product.

[0070] In this embodiment, a defect-free image sample of the old model product after repair is obtained according to the defect image sample of the old model product, the defect mask sample, and the pre-trained image completion model. That is, the defect image sample of the old model product and the defect mask sample are input into the preset DDNM image completion model for inference. At this time, the defect area corresponding to the defect mask corresponding to the defect image of the old model product is repaired, the original defect area disappears and is replaced by the image background. Here, it is called the defect-free image of the old model product after repair.

[0071] Step S124: Subtract each defect-free image of the old model product in the defect-free image sample of the old model product from the corresponding defect image of the old model product in the defect image sample of the old model product to obtain each defect difference image of the old model product.

[0072] In this embodiment, the corresponding defect difference image of the old model product is obtained by subtracting the defect-free image of the old model product from the corresponding repaired defect image of the old model product.

[0073] Step S125: Combine each defect difference image of the old model product to obtain the defect difference image sample of the old model product.

[0074] According to the above technical solution, in this embodiment, by inputting the defect image sample of the old model product and the defect mask sample into the preset image completion model, a defect-free image sample of the old model product is obtained. The defect image of the old model product is repaired by the preset image completion model, which improves the repair efficiency and the defect repair effect. In addition, the corresponding defect difference image of the old model product is obtained by subtracting the defect-free image of the old model product from the corresponding repaired defect image of the old model product, which improves the accuracy of the defect difference image of the old model product.

[0075] Further, based on the second embodiment, referring to Figure 3 , in the third embodiment of the present application, step S123 includes:

[0076] Step S1231: Use the image background to replace the defect area corresponding to each defect mask in the defect mask sample respectively to obtain the defect-free image of the old model product corresponding to each defect mask.

[0077] In this embodiment, the defect image of the old model product includes a defect area and a non-defect area. The pixel value corresponding to the defect area is 255, and the pixel value corresponding to the non-defect area is 0. Each defect area has a corresponding defect mask.

[0078] In this embodiment, the defect image samples and defect mask samples of the old model products are input into the DDNM preset image completion model for inference. The defect areas corresponding to the defect masks of the defect images of the old model products are repaired, the defect areas disappear and are replaced by the image background, and defect-free images of the old model products after repair are obtained.

[0079] Step S1232: Combine the defect-free images of each old model product to obtain the defect-free image sample of the old model product.

[0080] In this embodiment, after repairing the defect images of each old model product, defect-free images of each old model product corresponding to the defect images of each old model product are obtained. Finally, the defect-free images of each old model product are combined to obtain a defect-free image sample of the old model product. The repair accuracy of the defect images of the old model product is improved.

[0081] Further, based on the second embodiment, in the fourth embodiment of the present application, before step S130, it further includes:

[0082] Step S210: Train a model according to the defect mask sample and the defect difference image sample of the old model product to obtain a preset defect difference image generation model.

[0083] It should be noted that after obtaining the defect difference image sample of the old model product and the corresponding defect mask sample, a model is trained based on the defect mask sample and the defect difference image sample of the old model product to obtain a preset defect difference image generation model.

[0084] In this embodiment, the arbitrary-shaped mask is input into the preset defect difference image generation model to obtain a simulated defect difference image sample of the arbitrary-shaped mask.

[0085] According to the above technical solution, in this embodiment, a preset defect difference image generation model is constructed through the defect mask sample and the defect difference image sample of the old model product, and a simulated defect difference image sample of the arbitrary-shaped mask is obtained based on the preset defect difference image generation model, improving the generation efficiency and accuracy of the simulated defect difference image.

[0086] Further, based on the fourth embodiment, referring to Figure 4 , in the fifth embodiment of the present application, step S210 includes:

[0087] Step S211: Input the defect mask sample into the defect difference generator to obtain an initial defect difference image sample.

[0088] In a specific implementation, the defect mask sample is input into a defect difference generator to obtain an initial defect difference image sample. The role of the defect difference generator is to map the corresponding defect regions in the defect mask into defect differences of corresponding shapes.

[0089] In this embodiment, the defect difference generator includes an encoder and a decoder. Among them, the encoder is composed of multiple convolutional modules connected in series, and each convolutional module is composed of a convolutional layer, a batch normalization layer, and an activation layer. The decoder is composed of multiple deconvolutional modules connected in series, and each deconvolutional module is composed of a deconvolutional layer, a batch normalization layer, and an activation layer. The convolutional kernel size of the convolutional layer in each convolutional module and deconvolutional module of the defect difference generator is 4x4. There is a skip connection layer between the convolutional layer and the deconvolutional layer to connect the features obtained from each convolutional layer to the features obtained from the deconvolutional layer. The activation module of the output layer of the defect difference generator is Tanh. The encoder network structure of the defect difference generator is:

[0090] Conv64 - LeakyReLU - Conv128 - BN - LeakyReLU - Conv256 - BN - LeakyReLU - Conv512 - BN - LeakyReLU - Conv512.

[0091] The decoder network structure of the defect difference generator is:

[0092] ReLU - Deconv512 - BN - ReLU - Deconv512 - BN - ReLU - Deconv256 - BN - ReLU - Deconv128 - BN - ReLU - Deconv64 - Tanh.

[0093] Among them, Conv and Deconv represent the convolutional layer and the deconvolutional layer respectively, and the numbers behind them represent the number of feature maps in the module. LeakyReLU, ReLU, and Tanh are different activation functions. BN represents batch normalization.

[0094] Step S212, input the initial defect difference image sample and the defect difference image sample of the old model product into a defect difference discriminator to obtain an objective function.

[0095] It should be noted that the initial defect difference image sample and the defect difference image sample of the old model product need to be input into the defect difference discriminator respectively. The defect difference discriminator is used to discriminate whether the input comes from the real distribution or the generated distribution. The defect difference discriminator and the defect difference generator promote each other in an adversarial training manner, enabling the defect difference generator to learn the real distribution of the defect differences.

[0096] The defect difference discriminator consists of multiple convolutional modules. Each convolutional module is composed of a convolutional layer, a batch normalization layer, and an activation layer LeakyReLU. The discriminator contains 4 convolutional modules, and the convolutional kernel size of each module is 4x4. The network structure of the defect difference discriminator can be expressed as:

[0097] Conv64 - BN - LeakyReLU - Conv128 - BN - LeakyReLU - Conv256 - BN - LeakyReLU - Conv512.

[0098] Among them, Conv represents the convolutional layer, and the number after it represents the number of feature maps. BN represents batch normalization, and LeakyReLU is the activation function.

[0099] It should be noted that the initial defect difference image sample and the defect difference image sample of the old model product are respectively input into the defect difference discriminator to obtain the objective function.

[0100] Step S213, train the objective function through an optimizer to obtain the preset defect difference image generation model.

[0101] In this embodiment, the optimizer can be an ADAM optimizer.

[0102] According to the above technical solution in this embodiment, the present application trains a preset defect difference image generation model through a defect mask sample and a defect difference image sample of an old model product, which is convenient for subsequently determining a simulated defect difference image sample based on the preset defect difference image generation model.

[0103] Furthermore, based on the fifth embodiment, referring to Figure 5 , in the sixth embodiment of the present application, step S212 includes:

[0104] Step S2121, input the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain the defect difference authenticity probability.

[0105] It should be noted that the initial defect difference image sample and the defect difference image sample of the old model product are respectively input into the defect difference discriminator, and the defect difference discriminator will obtain the authenticity probability of the initial defect difference image.

[0106] Step S2122, determine the adversarial loss, L1 loss, and feature matching loss according to the defect difference authenticity probability.

[0107] In this embodiment, after obtaining the defect difference authenticity probability, the adversarial loss, L1 loss, and feature matching loss can be determined. Among them, the adversarial loss is used to ensure the authenticity of the graph, and the adversarial loss is:

[0108] L GAN (G, D) = E x，y [log D(x, y)] + E x [log(1 - D(x, G(x)))];

[0109] The L1 loss (i.e., the L1 norm loss) is used to ensure the content similarity between the initial generated image and the training defect image, and the L1 loss is:

[0110] L L1 = E x，y [||y - G(x)||1];

[0111] The feature matching loss is used to ensure the feature similarity between the generated defect difference and the training set defect difference, and the feature matching formula is:

[0112] L fm = E x，y [||f(x) - f(y)||1].

[0113] In the above formulas, G is the defect difference generator, D is the defect difference discriminator, E is the expectation of the overall training defect image, x is the input defect mask, y is the defect difference image of the old model product, and f is the feature extraction network using the feature layer of the VGG16 model pre-trained on the ImageNet dataset.

[0114] Step S2123, determine the objective function according to the adversarial loss, the L1 loss, and the feature matching loss.

[0115] In this embodiment, the objective function is obtained by adding the adversarial loss, the L1 norm, and the feature matching loss.

[0116] Further, based on the fifth embodiment and the sixth embodiment, in the seventh embodiment of the present application, step S213 includes:

[0117] Step S2131, use an optimizer to optimize and train the objective function.

[0118] Step S2132, when the number of training times reaches the preset number of times, obtain a preset defect difference sample generation model.

[0119] In this embodiment, the optimizer refers to the ADAM optimizer, and the preset number of times refers to the target number of times for the ADAM optimizer to optimize and train the objective function. The ADAM optimizer is used to train and optimize the objective function. When the number of training and optimization times reaches the preset number of times, it indicates that the initially generated image output by the defect generator is already similar to the real defect image, and both the defect generator and the defect generator have reached stability. At this time, according to the defect generator, the trained preset defect difference sample generation model is obtained.

[0120] In a specific implementation, the defect image of the old model product is preprocessed to obtain a defect mask, the defect mask is input into the defect difference generator, and an initial defect difference image is output. The initial defect difference image and the defect difference image of the old model product are respectively input into the defect difference discriminator, and the authenticity probability of the defect difference can be obtained and the gradient is backpropagated. Training and optimization are performed through the optimizer. If the initial defect difference image output by the defect difference generator meets the expectation, the training is stopped. If it does not meet the expectation, the defect mask is continuously input into the defect difference generator for training until the initial defect difference image output by the defect difference generator meets the expectation. Thereby improving the training accuracy of the preset initial defect difference image.

[0121] An embodiment of the method for generating a defect image of a new model product is provided in the embodiments of the present application. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0122] As Figure 6 shown, a device for generating a defect image of a new model product provided by the present application, the device for generating a defect image of the new model product includes:

[0123] An acquisition module 10, configured to acquire a defect image sample of an old model product and a defect-free image sample of a new model product.

[0124] A repair and difference module 20, configured to repair each defect image in the defect image sample of the old model product to obtain a defect-free image sample of the old model product, and determine a defect difference image sample between the defect-free image sample of the old model product and the defect image sample of the old model product.

[0125] A generation module 30, configured to generate a simulated defect difference image sample with an arbitrary shape mask according to a preset defect difference image generation model, where the preset defect difference image generation model is trained according to the defect difference image sample of the old model product.

[0126] An overlay module 40, configured to overlay the simulated defect difference image sample and the defect-free image sample of the new model product to obtain a defect image sample of the new model product.

[0127] The specific implementation of the device for generating defect images of the new model product in this application is basically the same as each embodiment of the method for generating defect images of the new model product described above, and will not be elaborated here.

[0128] As Figure 7 shown, Figure 7 FIG. is a schematic structural diagram of the hardware operating environment of the device for generating defect images of the new model product according to the embodiment of the present application. The device for generating defect images of the new model product may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory, such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0129] Those skilled in the art can understand that Figure 7 the structure of the device for generating defect images of the new model product shown in

[0130] As Figure 7 shown, in the memory 1005 as a storage medium, there may be included an operating system, a network communication module, a user interface module, and a program for generating defect images of the new model product. Among them, the operating system is a program for managing and controlling the hardware and software resources of the device for generating defect images of the new model product, and for running the program for generating defect images of the new model product and other software or programs.

[0131] In Figure 7 the device for generating defect images of the new model product shown, the user interface 1003 is mainly used to connect to the terminal and perform data communication with the terminal; the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the processor 1001 may be used to call the program for generating defect images of the new model product stored in the memory 1005.

[0132] In this embodiment, the device for generating defect images of the new model product includes: a memory 1005, a processor 1001, and a program for generating defect images of the new model product stored on the memory and executable on the processor, where:

[0133] When the processor 1001 calls the generation program of the new model product defect image stored in the memory 1005, the following operations are performed:

[0134] Obtain the defect image samples of the old model product and the defect-free image samples of the new model product;

[0135] Repair each defect image of the old model product defect image sample to obtain a defect-free image sample of the old model product, and determine the defect difference image sample between the defect-free image sample of the old model product and the defect image sample of the old model product;

[0136] Generate a simulation defect difference image sample of an arbitrary-shaped mask according to a preset defect difference image generation model, and the preset defect difference image generation model is trained according to the defect difference image sample of the old model product;

[0137] Overlay the simulation defect difference image sample and the defect-free image sample of the new model product to obtain a defect image sample of the new model product.

[0138] When the processor 1001 calls the generation program of the new model product defect image stored in the memory 1005, the following operations are performed:

[0139] Extract the defect positions and defect shapes corresponding to each defect image of the old model product defect image sample;

[0140] Obtain a defect mask sample according to the defect position and the defect shape corresponding to each defect image of the old model product;

[0141] Input the defect image sample of the old model product and the defect mask sample into a preset image inpainting model to obtain a defect-free image sample of the old model product;

[0142] Subtract each defect-free image of the old model product in the defect-free image sample of the old model product from the corresponding defect image of the old model product in the defect image sample of the old model product to obtain each defect difference image of the old model product;

[0143] Combine each defect difference image of the old model product to obtain the defect difference image sample of the old model product.

[0144] When the processor 1001 calls the generation program of the new model product defect image stored in the memory 1005, the following operations are performed:

[0145] Replace the defect area corresponding to each defect mask in the defect mask sample with the image background respectively to obtain the defect-free image of the old model product corresponding to each defect mask;

[0146] Combine the defect-free images of each old model product to obtain the defect-free image sample of the old model product.

[0147] When the processor 1001 calls the generation program of the defect image of the new model product stored in the memory 1005, the following operations are performed:

[0148] Perform model training based on the defect mask sample and the defect difference image sample of the old model product to obtain a preset defect difference image generation model.

[0149] When the processor 1001 calls the generation program of the defect image of the new model product stored in the memory 1005, the following operations are performed:

[0150] Input the defect mask sample into the defect difference generator to obtain an initial defect difference image sample;

[0151] Input the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain an objective function;

[0152] Train the objective function through an optimizer to obtain the preset defect difference image generation model.

[0153] When the processor 1001 calls the generation program of the defect image of the new model product stored in the memory 1005, the following operations are performed:

[0154] Input the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain the authenticity probability of the defect difference;

[0155] Determine the adversarial loss, L1 loss, and feature matching loss according to the authenticity probability of the defect difference;

[0156] Determine the objective function according to the adversarial loss, the L1 loss, and the feature matching loss.

[0157] When the processor 1001 calls the generation program of the defect image of the new model product stored in the memory 1005, the following operations are performed:

[0158] Use an optimizer to perform optimization training on the objective function;

[0159] When the number of training times reaches the preset number of times, obtain a preset defect difference sample generation model.

[0160] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium storing a program for generating defect images of new model products. When the program for generating defect images of new model products is executed by a processor, it implements each step of the method for generating defect images of new model products as described above and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0161] Since the storage medium provided in the embodiment of the present application is the storage medium adopted for implementing the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, those skilled in the art can understand the specific structure and variations of the storage medium, so it will not be elaborated here. Any storage medium adopted for the method of the embodiment of the present application belongs to the scope to be protected by the present application.

[0162] It should be noted that in this text, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or system including that element.

[0163] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0164] Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that makes a contribution to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, television, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0165] The above are only the preferred embodiments of the present application and do not limit the patent scope of the present application accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A method for generating defect images of a new model product, characterized in that, The method for generating defect images of the new model product includes: Obtaining defect image samples of the old model product and defect-free image samples of the new model product; Repairing each defect image of the old model product in the defect image samples of the old model product to obtain defect-free image samples of the old model product, and determining the defect difference image samples of the old model product between the defect-free image samples of the old model product and the defect image samples of the old model product; Generating simulation defect difference image samples with masks of arbitrary shapes according to a preset defect difference image generation model, where the preset defect difference image generation model is trained based on the defect difference image samples of the old model product; Overlaying the simulation defect difference image samples and the defect-free image samples of the new model product to obtain defect image samples of the new model product.

2. The method for generating defect images of the new model product according to claim 1, characterized in that, The steps of repairing each defect image of the old model product in the defect image samples of the old model product to obtain defect-free image samples of the old model product, and determining the defect difference image samples of the old model product between the defect-free image samples of the old model product and the defect image samples of the old model product include: Extracting the defect positions and defect shapes corresponding to each defect image of the old model product in the defect image samples of the old model product; Obtaining defect mask samples according to the defect positions and the defect shapes corresponding to each defect image of the old model product; Inputting the defect image samples of the old model product and the defect mask samples into a preset image completion model to obtain defect-free image samples of the old model product; Subtracting each defect-free image of the old model product in the defect-free image samples of the old model product from the corresponding defect image of the old model product in the defect image samples of the old model product to obtain defect difference images of each old model product; Combining the defect difference images of each old model product to obtain the defect difference image samples of the old model product.

3. The method for generating the defective image of the new model product according to claim 2, wherein The steps of inputting the defect image samples of the old model product and the defect mask samples into a preset image completion model to obtain defect-free image samples of the old model product include: Replacing the defect regions corresponding to each defect mask in the defect mask samples with the image background respectively to obtain defect-free images of the old model product corresponding to each defect mask; Combining the defect-free images of the old model product to obtain the defect-free image samples of the old model product.

4. The method for generating defect images of the new model product according to claim 2, characterized in that, Before the step of generating simulation defect difference image samples with masks of arbitrary shapes according to a preset defect difference image generation model, it further includes: Performing model training based on the defect mask samples and the defect difference image samples of the old model product to obtain a preset defect difference image generation model.

5. The method for generating defect images of a new model product according to claim 4, characterized in that, The steps of performing model training based on the defect mask samples and the defect difference image samples of the old model product to obtain a preset defect difference image generation model include: Inputting the defect mask samples into a defect difference generator to obtain initial defect difference image samples; Inputting the initial defect difference image samples and the defect difference image samples of the old model product into a defect difference discriminator to obtain an objective function; Training the objective function through an optimizer to obtain the preset defect difference image generation model.

6. The method for generating defect images of the new model product according to claim 5, characterized in that, The step of inputting the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain the objective function includes: Input the initial defect difference image sample and the defect difference image sample of the old model product into the defect difference discriminator to obtain the authenticity probability of the defect difference; Determine the adversarial loss, L1 loss, and feature matching loss according to the authenticity probability of the defect difference; Determine the objective function according to the adversarial loss, the L1 loss, and the feature matching loss.

7. The method for generating defect images of new model products according to claim 5 or 6, characterized in that, The step of training the objective function through an optimizer to obtain the preset defect difference image generation model includes: Use an optimizer to perform optimization training on the objective function; When the number of training times reaches the preset number, obtain the preset defect difference sample generation model.

8. An apparatus for generating defective images of a new model product, characterized in that, The generation device of the defect image of the new model product includes: An acquisition module for acquiring the defect image sample of the old model product and the defect-free image sample of the new model product; A repair and difference module for repairing each defect image of the old model product in the defect image sample of the old model product to obtain a defect-free image sample of the old model product, and determining the defect difference image sample of the old model product between the defect-free image sample of the old model product and the defect image sample of the old model product; A generation module for generating a simulated defect difference image sample with an arbitrary-shaped mask according to a preset defect difference image generation model, where the preset defect difference image generation model is trained according to the defect difference image sample of the old model product; An overlay module for overlaying the simulated defect difference image sample and the defect-free image sample of the new model product to obtain a defect image sample of the new model product.

9. A device for generating defect images of a new model product, characterized in that, The generation device of the defect image of the new model product includes: a memory, a processor, and a generation program of the defect image of the new model product stored on the memory and running on the processor. When the generation program of the defect image of the new model product is executed by the processor, the steps of the generation method of the defect image of the new model product according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a generation program of the defect image of the new model product. When the generation program of the defect image of the new model product is executed by a processor, the steps of the generation method of the defect image of the new model product according to any one of claims 1-7 are implemented.

Images