Defect image generation method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of machine vision, in particular to a defect image generation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring an original target image, performing image interception on the original target image based on a preset interception region to obtain an original local image; performing mask processing on the original local image based on a preset mask region to obtain a mask local image; inputting a preset target defect and the mask local image into a preset defect generation model, generating the preset target defect in the preset mask region in the mask local image to obtain a defect local image; and fusing the defect local image and the original target image based on the preset interception region to obtain a target defect image. The application can adapt to the training data expansion requirement of a high-resolution image scene.

Description

Technical Field

[0001] This application relates to the technical field of machine vision, and in particular to a method, apparatus, electronic device and storage medium for generating defect images. Background Technology

[0002] In the field of industrial automation, machine vision-based surface defect detection technology relies on a large number of labeled defect samples. When defect images are naturally acquired during actual production line operation, the scarcity and uneven distribution of some defect samples result in insufficient training datasets to meet model training requirements. Related technologies propose using defect image generation techniques to expand the training dataset; however, due to hardware cost limitations, defect image generation techniques are typically suitable for low-resolution images below 512×512. In high-precision industrial inspection scenarios involving ultra-high-resolution images (e.g., 8192×8192) acquired by line scan cameras, combined with minute defects, these techniques are ill-suited to the data expansion needs of industrial inspection. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for generating defective images. These can adapt to the training data expansion needs of high-resolution image scenes.

[0004] In a first aspect, this application provides a method for generating a defect image, the method comprising: acquiring an original target image; cropping the original target image based on a preset cropping region to obtain an original local image; performing masking processing on the original local image based on a preset masking region to obtain a masked local image; inputting a preset target defect and the masked local image into a preset defect generation model, generating the preset target defect in the preset masking region of the masked local image to obtain a defect local image; and fusing the defect local image and the original target image based on the preset cropping region to obtain a target defect image.

[0005] The embodiments described above crop the original target image into a local image by pre-selecting a cropping region, avoiding direct defect synthesis on the full-size ultra-high-resolution image. This compresses the processing scope from global to local, reducing memory usage and hardware costs. The original local image is masked to obtain a masked local image, allowing the defect generation model to generate the pre-selected target defect only within the pixel space defined by the masked region, thus obtaining the defect local image. Based on the corresponding position of the pre-selected cropping region, the generated defect local image is pasted onto the original target image, seamlessly replacing the local region to form the target defect image. While preserving the high-resolution background information of the original target image, this achieves the same resolution as the original target image, enabling the target defect image to adapt to the training data expansion needs of high-resolution image scenes.

[0006] In some embodiments of this application, before inputting the preset target defect and the masked local image into a preset defect generation model, the method further includes: acquiring an original image sample, and extracting a defect image sample from the original image sample; wherein the defect image sample is an image sample including the preset target defect; determining the defect region of the defect image sample, performing masking processing on the defect image sample based on the defect region to generate a masked image sample; iteratively training an initial adversarial model based on the preset target defect, the masked image sample, and the defect image sample, and obtaining the defect generation model when the initial adversarial model satisfies a preset iteration condition.

[0007] The embodiments described above obtain defect image samples by cropping the original image samples, and then perform masking processing on the defect sample images. This reduces the input size of the training data (masked image samples) input to the initial adversarial model from the entire original image sample to a local region. Furthermore, the masking process removes redundant pixel information unrelated to the defect generation region. This allows the initial adversarial model to focus on feature extraction and parameter updates within the masked region corresponding to the masked image samples during training, avoiding invalid calculations of pixels outside the masked region. This reduces the memory usage for model training and improves training efficiency.

[0008] In some embodiments of this application, the initial adversarial model includes a generator and a discriminator; the step of iteratively training the initial adversarial model based on the preset target defect, the mask image sample, and the defect image sample, and obtaining the defect generation model when the initial adversarial model meets the preset iteration conditions, includes: if the preset iteration conditions are not met, performing a first model update, including: inputting the preset target defect and the mask image sample into the generator, generating the preset target defect in the mask region of the mask image sample, and obtaining a defect image to be examined; inputting the defect image to be examined and the defect image sample into the discriminator for image discrimination, generating a first discrimination result, and calculating the model loss of the generator based on the first discrimination result; updating the parameters of the generator based on the model loss, repeating the first model update steps until the preset iteration conditions are met, and determining the iteratively updated generator as the defect generation model.

[0009] The embodiments described above in this application input a preset target defect and a mask image sample into a generator. The generator generates the preset target defect in the masked region of the mask image sample, thereby synthesizing a defect image to be examined. The generated defect image to be examined and the defect image sample are input into a discriminator. The discriminator performs image discrimination and outputs a first discrimination result, thereby distinguishing between real samples and samples generated by the generator. Then, the model loss is calculated based on the first discrimination result, forcing the generator and discriminator to compete against each other through parameter updates. This iteratively optimizes the defect generation quality of the generator and the image discrimination ability of the discriminator until a preset iteration condition is met. The iteratively updated generator is then used as the defect generation model. This adversarial training mechanism makes the images generated by the final defect generation model visually indistinguishable from real defects, thereby reducing the splicing marks or unnaturalness of the defect generation images.

[0010] In some embodiments of this application, calculating the model loss of the generator based on the first discrimination result includes: calculating an adversarial loss based on the first discrimination result, and calculating a background reconstruction loss between the defect image to be tested and the defect image sample; and determining the model loss using the adversarial loss, the background reconstruction loss, and a preset weight coefficient.

[0011] The embodiments of this application calculate the adversarial loss based on the first discrimination result, and drive the generator to generate realistic defect textures through the adversarial loss. The background reconstruction loss between the defect image to be tested and the defect image sample is calculated. Based on the background reconstruction loss, the generator is driven to ensure that background pixels outside the mask area are not distorted or blurred when generating defects. The background reconstruction loss is weighted using preset weight coefficients and then added to the adversarial loss to determine the model loss. The model loss is used to update the generator's parameters, enabling the generator to improve the realism of the generated defect image while maintaining the original image background, thereby avoiding background blurring or edge artifacts and other background variation problems.

[0012] In some embodiments of this application, the method further includes: updating the parameters of the discriminator according to the adversarial loss, and using the updated discriminator to perform the first model update in the next round.

[0013] The above embodiments of this application optimize the image discrimination ability of the discriminator by updating the discriminator while updating the generator parameters. This forces the generator to face a stronger discriminator with more accurate discrimination ability in the new training rounds, thereby forcing it to generate higher quality images and thus improving the image generation ability of the trained defect generation model.

[0014] In some embodiments of this application, the method further includes: performing quality discrimination on the target defect image to obtain a second discrimination result; if the target defect image is determined to meet a preset quality standard based on the second discrimination result, adding the target defect image to a machine learning sample set; if the target defect image is determined not to meet the preset quality standard based on the second discrimination result, identifying the target defect image as a low-quality defect image and outputting a low-quality prompt message.

[0015] The embodiments of this application perform quality judgment on the target defect image. If the second judgment result shows that the target defect image meets the preset quality standard, the target defect image is determined to have high fidelity and effectiveness, and is added to the machine learning sample set as training data. If the second judgment result shows that the target defect image does not meet the preset quality standard, the target defect image is intercepted and marked, thereby ensuring the data reliability of the machine learning sample set.

[0016] In some embodiments of this application, the method further includes: statistically analyzing the proportion of the low-quality defect images in the target defect image to obtain an output low-quality rate; if the output low-quality rate exceeds a preset low-quality rate threshold, triggering a second model update for the defect generation model.

[0017] The embodiments of this application calculate the low-quality output rate of the current defect generation model by statistically analyzing the proportion of low-quality defect images in the target defect image. If the low-quality output rate does not exceed the threshold, it indicates that the current generation capability of the defect generation model is stable and can continue to run; if the low-quality output rate exceeds the preset low-quality rate threshold, it indicates that the current generation capability of the defect generation model has declined or is insufficient to adapt to the current operating scenario. By introducing new training data, adjusting the learning rate, or recalibrating the network parameters, a second model update of the defect generation model is triggered to correct the current generation deviation of the defect generation model.

[0018] Secondly, this application provides a defect image generation apparatus, comprising: an image cropping module, configured to acquire an original target image and crop the original target image based on a preset cropping region to obtain an original partial image; a mask generation module, configured to perform masking processing on the original partial image based on a preset masking region to obtain a masked partial image; a defect generation module, configured to input a preset target defect and the masked partial image into a preset defect generation model, and generate the preset target defect in a preset masking region in the masked partial image to obtain a defected partial image; and a defect fusion module, configured to fuse the defected partial image and the original target image based on the preset cropping region to obtain a target defect image.

[0019] Thirdly, this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described defect image generation method.

[0020] Fourthly, this application provides a computer storage medium storing program instructions that, when executed on a processor, cause the processor to perform the aforementioned defect image generation method. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a defect image generation method in related technologies. Figure 2 A flowchart illustrating a defect image generation method provided in some embodiments of this application; Figure 3 Schematic diagrams illustrating examples of defect image generation methods provided in some embodiments of this application; Figure 4 This application provides some embodiments of a defect image generation method, which serve as an image comparison diagram. Figure 5 This is a schematic diagram illustrating the effect of the defect image generation method provided in some embodiments of this application; Figure 6 A flowchart illustrating the defect generation model training method provided in some embodiments of this application; Figure 7 A schematic diagram illustrating examples of defect generation model training methods provided in some embodiments of this application; Figure 8 A schematic flowchart illustrating the model output quality inspection method provided in some embodiments of this application; Figure 9 A virtual structural diagram of a defect image generation apparatus provided in some embodiments of this application; Figure 10 The diagram shows the structure of an electronic device provided in some embodiments of this application. Detailed Implementation

[0022] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0023] In the embodiments of this application, it should be noted that, unless otherwise expressly specified and limited, the word "for example" is used to indicate an example, illustration, or description. Any embodiment or design scheme described as "for example" in the embodiments of this application should not be construed as being better or more advantageous than other embodiments or design schemes. Specifically, the use of the word "for example" is intended to present the relevant concepts in a specific manner.

[0024] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0025] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more features. Furthermore, in the description of this application, "multiple" means two or more, unless otherwise expressly and specifically limited.

[0026] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0027] In the field of industrial automation, machine vision-based surface defect detection technology relies on a large number of labeled defect samples. When defect images are naturally collected during actual production line operation, the scarcity and uneven distribution of some defect samples generated in actual production lead to an insufficient training dataset, making it difficult to meet the needs of model training.

[0028] Related technologies propose using defect image generation techniques to augment the training dataset. Please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart illustrating a defect image generation method in related technologies. Figure 1The defect image generation method shown typically includes: step S100, acquiring a defect-free image; step S200, using the defect-free image as input, generating a defect image from the defect-free image; and step S300, adding the defect image to the machine learning training set. However, due to hardware cost limitations, defect image generation technology is generally suitable for low-resolution images below 512×512. In high-precision industrial inspection scenarios involving ultra-high-resolution images (e.g., 8192×8192) acquired by line scan cameras combined with minute defects, it is difficult to adapt to the data expansion needs of industrial inspection.

[0029] In view of this, this application provides a defect image generation method, apparatus, electronic device and storage medium, which can overcome hardware cost limitations and adapt to the training data expansion needs of high-resolution image scenes.

[0030] Please see Figure 2 , Figure 2 This is a flowchart illustrating a defect image generation method provided in some embodiments of this application. Depending on different needs, Figure 2 The order of some steps in the flowchart shown can be changed, and some steps can be omitted. The following section will explain... Figure 2 Each step is explained in detail.

[0031] Step S200: Obtain the original target image, and perform image cropping on the original target image based on the preset cropping area to obtain the original local image.

[0032] In some embodiments of this application, the original target image represents a complete image acquired from an industrial field acquisition device or image dataset, such as an image of an anodized aluminum metal plate. To adapt to application scenarios such as metal surface defect detection and the development of AI optical inspection instruments, where the defect size differs significantly from the image size and the defect is easily obscured by the entire image, the original target image can be acquired using a high-resolution image acquisition device, such as a line scan camera supporting a resolution of 8192×8192.

[0033] In some embodiments of this application, the preset cropping region refers to a specific spatial coordinate or pixel range in the original target image that is pre-defined to limit the data cropping range. For example, it is a rectangular coordinate frame with a length and width of 128 or 256 pixels, based on the image center. Based on the coordinate range defined by the preset cropping region, an image cropping operation is performed on the original target image, using the image data within the preset cropping region as the original local image. This reduces the data input size of the subsequent defect generation model from the entire image to a local region, thus lowering computational power consumption and hardware costs.

[0034] Step S201: Perform masking processing on the original local image based on the preset masking region to obtain a masked local image.

[0035] In some embodiments of this application, the preset mask region represents a predefined two-dimensional coordinate range in the original local image that requires specific pixel operations. For example, in a 256×256 pixel original local image, a circular or irregular range with a center of 64×64 pixels is set as the preset mask region. Masking processing is performed on the original local image based on the preset mask region. For example, the grayscale values ​​of pixels involved in the preset mask region in the original local image are set to 0, while the grayscale values ​​of pixels outside the preset mask region remain unchanged, thereby generating a masked local image. While preserving the background texture of the original local image, a blank area with missing information is generated to limit the area where the subsequent defect generation model needs to generate defects.

[0036] Step S202: Input the preset target defect and the masked local image into the preset defect generation model, generate the preset target defect in the preset masked area in the masked local image, and obtain the defect local image.

[0037] In some embodiments of this application, the preset target defect is a pre-defined specific type or morphological feature of imperfection that needs to be simulated in an image, such as color unevenness defects, scratches, bumps, or color difference defects on a metal surface. The defect generation model is a pre-trained neural network model capable of generating the preset target defect in a preset mask region within a masked local image. The defect generation model can extract and analyze the background features of non-masked areas in the masked local image, and combine the semantic information of the preset target defect to synthesize and generate corresponding defect pixels within the preset mask region. The defect local image generated by the defect generation model has already generated the preset target defect in the preset mask region, for example, an image in which a preset mask region in a 256×256 pixel masked local image is filled with scratch texture.

[0038] Step S203: Based on the preset cropping region, fuse the local image of the defect and the original target image to obtain the target defect image.

[0039] In some embodiments of this application, the spatial coordinate information defined by the preset cropping area is read, and the defect local image is aligned and fitted with the original target image in space based on the read spatial coordinate information. The original pixel data in the preset cropping area of ​​the original target image is removed, and the pixel data of the defect local image is used for in-situ overlay and replacement. In this way, the local image containing realistic defect features is seamlessly embedded into the full image of the high-resolution original target image, generating a target defect image that retains its original appearance but naturally embeds the preset target defect (such as uneven staining defect or scratch defect) in the specified position.

[0040] In some embodiments of this application, by performing local cropping and masking on the original target image to generate a defect local image, and then replacing the original target image in situ based on the defect local image, the huge memory overhead and computational redundancy caused by directly performing defect generation model inference on the high-resolution full image are avoided, thereby obtaining reliable high-quality, high-resolution defect image samples.

[0041] In some embodiments of this application, see Figure 3 , Figure 3 This is a schematic diagram illustrating some embodiments of the defect image generation method provided in this application. The full-size image is the original target image containing the complete industrial scene background; the local image represents a smaller local image cropped from the full-size image according to a preset cropping region; and the masked local image is the image obtained after masking the original local image. The masked local image is input into a trained defect generation model to generate a defect, resulting in a local image with generated defects. The local image is then fused with the original target image to obtain a full-size target defect image with generated defects.

[0042] See Figure 4 , Figure 4 This is a schematic diagram comparing images of the defect image generation method provided in some embodiments of this application. The original target image 400 defines the area where defects need to be generated using a preset cropping region (rectangular frame shown). An irregular mask (irregular patch shown) is generated within the preset cropping region, resulting in a partial mask image 401, which serves as input to the defect generation model. The defect generation model then generates a defect local image 402 from the masked region in the partial mask image 401. This defect local image 402 is then integrated into the original target image 400 to obtain the target defect image 403.

[0043] See Figure 5 , Figure 5 This is a schematic diagram illustrating the effect of a defect image generation method provided in some embodiments of this application. Figure 5In the diagram, sub-image (a) is an overall view of the original target image. Sub-image (b) is a partial view of the original target image at the preset defect generation location (preset cropping area), including a defect label "Generated defect is here" indicating the preset defect generation coordinates on the original target image. Sub-image (c) is obtained by cropping the original target image according to the preset cropping area. A masked partial image is obtained by masking the selected area in sub-image (c). This masked partial image is then input into the defect generation model to generate a defect partial image. As shown in sub-image (d), the defect partial image is fused into the original target image to obtain the target defect image (partial schematic diagram) shown in sub-image (e).

[0044] As a reference, sub-image (f) shows the result obtained by directly overlaying the defective local image onto the corresponding preset cropping area of ​​the original target image without using the smooth fusion method of this method, serving as a contrast to highlight the smoothness of the fusion of this method. In sub-image (f), the preset cropping area as a whole presents a square bright block with a brightness different from the surrounding background. The boundary of the square bright block is a boundary artifact generated by direct overlay. Although the defect is located within the square area, there is an obvious grayscale step and seam between the square area and the surrounding background.

[0045] In some embodiments of this application, before inputting the preset target defect and masked local image into the preset defect generation model, a series of training steps are required to obtain a defect generation model that meets the requirements. See also... Figure 6 , Figure 6 This is a flowchart illustrating the defect generation model training method provided in some embodiments of this application. Depending on different needs... Figure 6 The order of some steps in the flowchart shown can be changed, and some steps can be omitted. The following section will explain... Figure 6 Each step is explained in detail.

[0046] Step S600: Obtain the original image sample and extract the defect image sample from the original image sample.

[0047] In some embodiments of this application, the original image sample represents a high-resolution (e.g., 8192×8192 pixels) metal surface image containing complete industrial scene background information, covering the texture, gloss and structural features of the metal surface under normal production conditions, such as a complete anodized aluminum metal plate as the acquired image.

[0048] In some embodiments of this application, a predetermined target defect is identified in the original image sample. A defect image sample is then extracted from the original image sample according to predetermined spatial coordinates (e.g., 128×128 or 256×256 pixels). The obtained defect image sample contains at least one predetermined target defect and retains the background texture of the defect area, the shape of the defect itself, and the grayscale distribution characteristics. For example, a 256×256 pixel area with a scratch in the center is cropped from an 8192×8192 full image of a steel plate to obtain a defect image sample.

[0049] Step S601: Determine the defect region of the defect image sample, and perform masking processing on the defect image sample based on the defect region to generate a masked image sample.

[0050] In some embodiments of this application, the defect region is determined by the grayscale distribution features, edge contours, or pre-annotated segmentation information of the preset target defect in the defect image sample. The defect region represents the entire set of pixels occupied by the preset target defect and its corresponding spatial range in the defect image sample. For example, the set of approximately 40×8 pixels of continuous pixels covered by a linear scratch is determined as the defect region in the defect image sample.

[0051] In some embodiments of this application, the grayscale values ​​of all pixels belonging to the defect region in the defect image sample are masked. For example, the grayscale values ​​of all pixels in the defect region are set to 0, or the defect region is selected using a regular-shaped coordinate frame, and then the grayscale values ​​of all pixels within the regular-shaped coordinate frame are set to 0. This creates a blank area with missing information while completely preserving the background texture structure around the blank area, resulting in a masked image sample. For example, after performing masking processing on a 256×256 pixel defect image sample containing scratches, the grayscale values ​​of the 40×8 pixel area where the scratches are located all become 0, while the grayscale values ​​of the remaining background pixels remain the same as before processing.

[0052] Step S602: Based on the preset target defect, mask image sample and defect image sample, the initial adversarial model is iteratively trained. When the initial adversarial model meets the preset iteration conditions, the defect generation model is obtained.

[0053] In some embodiments of this application, the initial adversarial model refers to a Generative Adversarial Network (GAN) architecture that has not yet been fully trained and whose internal network parameters are in a randomly initialized or pre-initialized state. Masked image samples and defective image samples are used as training data input to the initial adversarial model. The initial adversarial model updates its model parameters batch by batch until a pre-set iteration condition is reached to determine whether the initial adversarial model has been sufficiently trained and can terminate iterative training. This pre-set iteration condition can manifest as the number of training epochs reaching a preset upper limit, the model loss changing less than a preset magnitude threshold for several consecutive epochs, or the structural similarity index between the generated image and the target image reaching a preset similarity threshold, etc.

[0054] In some embodiments of this application, the initial adversarial model can adopt a pix2pix framework, which consists of two sub-networks: a generator and a discriminator. The generator, representing the sub-network responsible for image generation in the initial adversarial model, is used to fill the masked region with the grayscale texture of the preset target defect and reconstruct the original background in the non-masked region, outputting a generated image identical to the defect image sample, thus obtaining the defect image to be examined. It typically employs a U-Net structure. The discriminator, representing the sub-network responsible for image authenticity determination in the initial adversarial model, is used to distinguish between the defect image to be examined and the defect image sample, outputting the probability value that each defect image is a real image. It typically employs a convolutional classification network.

[0055] In some embodiments of this application, an initial adversarial model is iteratively trained based on a preset target defect, a mask image sample, and a defect image sample. When the initial adversarial model meets a preset iteration condition, a defect generation model is obtained. This includes: if the preset iteration condition is not met, a first model update is performed, including: inputting the preset target defect and the mask image sample into a generator, generating the preset target defect in the mask region of the mask image sample, and obtaining a defect image to be examined; inputting the defect image to be examined and the defect image sample into a discriminator for image discrimination, generating a first discrimination result, and calculating the model loss of the generator based on the first discrimination result; updating the parameters of the generator based on the model loss, and repeating the first model update steps until the preset iteration condition is met, and determining the iteratively updated generator as the defect generation model.

[0056] In some embodiments of this application, the first model update represents a complete training loop executed when the initial adversarial model does not meet the preset iteration conditions. The generator inputs the category information of the preset target defect and masked image samples. Based on the generator's current network parameters, it performs texture prediction and grayscale filling on the masked regions in the masked image samples to generate the grayscale distribution corresponding to the preset target defect. It also reconstructs the background texture of the non-masked regions and outputs the defect image to be examined.

[0057] In some embodiments of this application, the image of the defect to be tested and the corresponding real defect image sample are input into a discriminator. The discriminator performs image discrimination on the two images respectively, outputting their respective true / false probability values ​​to form a first discrimination result. Based on the first discrimination result and the pixel-by-pixel difference between the image of the defect to be tested and the defect image sample, the model loss of the generator is calculated according to a preset loss function. The model loss reflects the current deviation of the generator in two dimensions: the realism of defect generation and the accuracy of background restoration. Using parameter calculation methods such as backpropagation, the gradient of the model loss with respect to the parameters of each layer of the generator is calculated, and the parameters are updated in the opposite direction of the gradient, so that the output of the generator in the next round of forward propagation is closer to the real defect image sample.

[0058] In some embodiments of this application, calculating the model loss of the generator based on the first discrimination result includes: calculating the adversarial loss based on the first discrimination result, and calculating the background reconstruction loss between the defect image to be tested and the defect image sample; determining the model loss using the adversarial loss, the background reconstruction loss and the preset weight coefficients, updating the parameters of the discriminator based on the adversarial loss, and using the updated discriminator to perform the first model update in the next round.

[0059] In some embodiments of this application, the model loss of the initial adversarial model Expressions can be used Calculate the model loss, where, This indicates finding the minimum value of the generator G. This indicates finding the maximum value of the discriminator D. Indicating resistance to loss, λ represents the background reconstruction loss, and λ is a preset weighting coefficient used to adjust the relative weight of the background reconstruction loss in the model loss.

[0060] In some embodiments of this application, the true / false probability value D(x,G(x,z)) output by the discriminator to the defective image G(x,z) is extracted from the first discrimination result. The true / false probability value D(x,G(x,z)) is substituted into the adversarial loss function L_GAN(D,G) for calculation to obtain the adversarial loss. A pixel-wise absolute difference operation is performed between the defective image G(x,z) to be tested and the defective image sample y to obtain the background reconstruction loss. The adversarial loss and the background reconstruction loss are substituted into the expression... The model loss is calculated.

[0061] In some embodiments of this application, the method further includes: updating the parameters of the discriminator according to the adversarial loss, and using the updated discriminator for the first model update in the next round. The network parameters of the discriminator are updated according to the adversarial loss L_GAN, increasing the probability of the discriminator judging real defect image samples and decreasing the probability of judging defect images to be examined, thereby enhancing the discriminator's ability to distinguish between real and fake images. The discriminator with updated parameters is then used as the first model update in the next round.

[0062] See Figure 7 , Figure 7 This is a schematic diagram illustrating an example of a defect generation model training method provided in some embodiments of this application. The training objective of the defect generation model is to generate a target defect image from an input original target image. During training, the original image sample (Full Size Image) is the original image containing a complete metal surface background; the local image sample (Local Image with Defect) represents a local image extracted from the full-size image according to a preset cropping region, preserving the original defect grayscale texture; and the masked local image (Zero Masked Local Image) is the image obtained after the grayscale of the defect region in the local image with defect is zeroed out.

[0063] In some embodiments of this application, masked image samples are used as input and defective image samples are used as targets. The two are fed into the pixel-to-pixel generative adversarial network (GAN) training stage in a paired manner. A pixel-to-pixel conditional image translation architecture is used to perform adversarial training with the generative adversarial network (GAN). After training, the generator can generate real defects in the zero-set regions and restore the background in the non-masked regions, thereby obtaining a trained defect generation model.

[0064] Reference Figure 8 , Figure 8 This is a flowchart illustrating the adversarial training method for a defect generation model provided in some embodiments of this application. Depending on different needs... Figure 8 The order of some steps in the flowchart shown can be changed, and some steps can be omitted. The following section will explain... Figure 8 Each step is explained in detail.

[0065] Step S800: Perform quality judgment on the target defect image to obtain a second judgment result.

[0066] In some embodiments of this application, quality discrimination refers to determining the authenticity or quality of a target defect image based on quality dimensions such as defect realism, texture continuity between the defect and the surrounding background, grayscale distribution consistency, and the harmony between the defect morphology and its environment, thereby obtaining a second discrimination result. The second discrimination result can take the form of authenticity probability, quality score, or category label given by the discrimination model, and is used to characterize the degree of similarity between the target defect image and the real defect image. In some embodiments of this application, the discriminator obtained after the initial adversarial model training can be used as the discrimination model.

[0067] Step S801: If the target defect image is determined to meet the preset quality standard based on the second discrimination result, the target defect image is added to the machine learning sample set.

[0068] In some embodiments of this application, the preset quality standard refers to a pre-set quality judgment condition or quality threshold used to determine whether a target defect image meets the requirements for inclusion in the database. Its specific form can correspond to the representation of the second discrimination result. For example, when the second discrimination result is represented as the true probability given by the discrimination model, the preset quality standard can be represented as a true probability threshold; when the second discrimination result is represented as a quality score, the preset quality standard can be represented as a score threshold; and when the second discrimination result is represented as a category label, the preset quality standard can be represented as the required target category.

[0069] In some embodiments of this application, the machine learning sample set represents a set of images used to train a machine learning model and enable the model to learn defect recognition capabilities. Each image in the machine learning sample set serves as a training sample for optimizing model parameters. Qualified target defect images are added to the machine learning sample set as new samples. This allows the target defect images, along with existing real defect images or previously stored target defect images in the machine learning sample set, to form the data foundation used for training the machine learning model. Therefore, the quality of the machine learning sample set determines the distribution of defect features that the machine learning model (e.g., a defect recognition model) can learn.

[0070] Step S802: If the target defect image does not meet the preset quality standard based on the second discrimination result, the target defect image is identified as a low-quality defect image and a low-quality prompt message is output.

[0071] In some embodiments of this application, a low-quality defect image refers to a synthesized image whose defect and background fusion quality is substandard at the full-image scale, still exhibiting synthesis traces or boundary artifacts, and whose distribution deviates perceptibly from that of the real defect image. This provides a structured category identifier for subsequent isolation, statistical analysis, or traceability processing of substandard synthesized images. Furthermore, the prompting information output for a target defect image identified as a low-quality defect image is called a low-quality prompt. The content of the low-quality prompt may include the identification information of the low-quality defect image, the corresponding second discrimination result value, the preset quality standard being compared, and the reason for non-compliance, enabling users or monitoring processes to know which target defect image fails to meet the standard in which quality dimension, facilitating user understanding of the image synthesis status.

[0072] In some embodiments of this application, in order to promptly detect an overall decline or degradation in the output quality of the defect generation model, the overall model synthesis capability can be monitored. The method further includes: statistically analyzing the proportion of low-quality defect images in the target defect image to obtain an output low-quality rate; if the output low-quality rate exceeds a preset low-quality rate threshold, a second model update of the defect generation model is triggered.

[0073] In some embodiments of this application, the low-quality output rate can be obtained based on statistically derived quantity proportions. The low-quality output rate is a statistical indicator characterizing the proportion of unqualified images among the target defect images produced by the defect generation model in a current synthesis process or batch. A higher value indicates that the defect generation model produces more low-quality defect images that fail to pass the quality gate during that process; conversely, a lower value indicates a higher proportion of qualified images produced by the defect generation model. The preset low-quality output rate threshold represents a pre-set maximum allowed value for the low-quality output rate.

[0074] In some embodiments of this application, the second model update refers to the model update operation performed in this step to adjust the parameters of the defect generation model in order to improve its subsequent synthesis quality. When the output low-quality rate does not exceed the preset low-quality rate threshold, the overall synthesis quality produced by the defect generation model is considered acceptable, and no update is performed on the defect generation model; the synthesis and discrimination process continues to run with the original parameters. When the output low-quality rate exceeds the preset low-quality rate threshold, it is considered that the proportion of low-quality images produced by the defect generation model has exceeded the tolerance limit, and the overall synthesis quality has deteriorated. This triggers the second model update of the defect generation model, adjusting the parameters of the defect generation model to increase the proportion of subsequent synthesized images passing the quality gate and reduce the subsequent output low-quality rate.

[0075] like Figure 9 As shown, Figure 9This is a virtual structure diagram of a software development apparatus provided in some embodiments of this application, including an image capture module 90, a mask generation module 91, a defect generation module 92, and a defect fusion module 93.

[0076] The image cropping module 90 is used to acquire the original target image and crop the original target image based on a preset cropping area to obtain the original local image. The mask generation module 91 is used to perform masking processing on the original local image based on a preset mask region to obtain a masked local image; The defect generation module 92 is used to perform the following: inputting a preset target defect and a mask local image into a preset defect generation model, generating a preset target defect in a preset mask region in the mask local image, and obtaining a defect local image; The defect fusion module 93 is used to perform the fusion of the local defect image and the original target image based on the preset cropping region to obtain the target defect image.

[0077] It is understood that the module division described above is a logical functional division, and there may be other division methods in actual implementation. Furthermore, the functional modules in the various embodiments of this application can be integrated into the same processing unit, or each module can exist physically separately, or two or more modules can be integrated into the same unit. The integrated modules described above can be implemented in hardware or in a combination of hardware and software functional modules.

[0078] like Figure 10 As shown, Figure 10 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this application. The electronic device 100 can be a computer, mobile phone, tablet computer, laptop computer, server, etc., and the embodiments of this application do not impose any restrictions on the specific type of the electronic device 100.

[0079] exist Figure 10 The electronic device 100 may include a communication module 1001, a memory 1002, a processor 1003, an input / output (I / O) interface 1004, and a bus 1005. The processor 1003 is coupled to the communication module 1001, the memory 1002, and the input / output interface 1004 via the bus 1005.

[0080] Communication module 1001 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as Universal Serial Bus (USB) and Controller Area Network (CAN). The wireless communication module may provide one or more wireless communication solutions such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication networks, Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR).

[0081] The memory 1002 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 1003, and can be used to store executable programs (e.g., machine instructions) of other running programs, as well as user and application data. The RAM may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0082] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 1003. Non-volatile memory can include disk storage devices and flash memory.

[0083] The memory 1002 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 1003. The one or more computer programs include a plurality of instructions, which, when executed by the processor 1003, enable a software development method to be executed on the electronic device 100.

[0084] In other embodiments, such as Figure 10 The electronic device 100 shown also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 100.

[0085] Processor 1003 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0086] The processor 1003 provides computing and control capabilities. For example, the processor 1003 is used to execute computer programs stored in the memory 1002 to implement the software development method described above.

[0087] The input / output interface 1004 is used to provide a channel for user input or output. For example, the input / output interface 1004 can be used to connect various input / output devices, such as a mouse, keyboard, touch device, display screen, etc., so that users can enter information or visualize information.

[0088] Bus 1005 is used at least to provide a channel for communication between communication modules 1001, memory 1002, processor 1003, and input / output interface 1004 in electronic device 100.

[0089] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0090] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. When the program instructions are executed, the method implemented can refer to the methods in the above embodiments of this application.

[0091] The computer-readable storage medium can be the internal memory of the electronic device described in the above embodiments, such as the hard disk or memory of the electronic device. Alternatively, the computer-readable storage medium can be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the electronic device.

[0092] In some embodiments, a computer-readable storage medium may include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of the electronic device, etc.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0094] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0096] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0097] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this application may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A method for generating defect images, characterized in that, The method includes: The original target image is acquired, and the original target image is cropped based on a preset cropping region to obtain an original partial image; The original local image is masked based on a preset mask region to obtain a masked local image. The preset target defect and the masked local image are input into a preset defect generation model, and the preset target defect is generated in the preset masked area in the masked local image to obtain a defect local image; The target defect image is obtained by fusing the local defect image and the original target image with the preset cropping region.

2. The defect image generation method according to claim 1, characterized in that, Before inputting the preset target defect and the masked local image into the preset defect generation model, the method further includes: Obtain an original image sample, and extract a defect image sample from the original image sample; wherein, the defect image sample is an image sample that includes the preset target defect; Determine the defect region of the defective image sample, and perform masking processing on the defective image sample based on the defect region to generate a masked image sample; Based on the preset target defect, the mask image sample, and the defect image sample, the initial adversarial model is iteratively trained. When the initial adversarial model meets the preset iteration conditions, the defect generation model is obtained.

3. The defect image generation method according to claim 2, characterized in that, The initial adversarial model includes a generator and a discriminator; The step of iteratively training an initial adversarial model based on the preset target defect, the mask image sample, and the defect image sample, and obtaining the defect generation model when the initial adversarial model meets preset iteration conditions, includes: If the preset iteration conditions are not met, a first model update is performed, including: The preset target defect and the mask image sample are input into the generator, and the preset target defect is generated in the mask area in the mask image sample to obtain the defect image to be inspected. The defect image to be tested and the defect image sample are input into the discriminator for image discrimination, a first discrimination result is generated, and the model loss of the generator is calculated based on the first discrimination result; The generator is updated with parameters based on the model loss, and the first model update step is repeated until the preset iteration condition is met. The iteratively updated generator is then determined as the defect generation model.

4. The defect image generation method according to claim 3, characterized in that, The step of calculating the model loss of the generator based on the first discrimination result includes: Based on the first discrimination result, the adversarial loss is calculated, and the background reconstruction loss between the defect image to be tested and the defect image sample is calculated; The model loss is determined using the adversarial loss, the background reconstruction loss, and the preset weight coefficients.

5. The method for generating defective images according to claim 4, characterized in that, The method further includes: The discriminator is updated with parameters based on the adversarial loss, and the updated discriminator is used for the first model update in the next round.

6. The defect image generation method according to claim 1, characterized in that, The method further includes: The target defect image is subjected to quality judgment to obtain a second judgment result; If the target defect image is determined to meet the preset quality standard based on the second discrimination result, the target defect image is added to the machine learning sample set; If, based on the second discrimination result, it is determined that the target defect image does not meet the preset quality standard, the target defect image is identified as a low-quality defect image, and a low-quality prompt message is output.

7. The defect image generation method according to claim 6, characterized in that, The method further includes: The proportion of low-quality defect images in the target defect images is statistically analyzed to obtain the low-quality output rate. If the output low quality rate exceeds a preset low quality rate threshold, a second model update of the defect generation model is triggered.

8. A defect image generation apparatus, characterized in that, include: The image cropping module is used to acquire the original target image and crop the original target image based on a preset cropping area to obtain the original partial image. The mask generation module is used to perform masking processing on the original local image based on a preset mask region to obtain a masked local image; The defect generation module is used to input a preset target defect and the masked local image into a preset defect generation model, generate the preset target defect in a preset masked region in the masked local image, and obtain a defect local image. The defect fusion module is used to perform the fusion of the defect local image and the original target image based on the preset cropping region to obtain a target defect image.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the defect image generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the defect image generation method as described in any one of claims 1 to 7.

Images