Method for constructing defect image generation model, defect image generation method and device

Abstract

Embodiments of the present application provide a method for constructing a defect image generation model, a defect image generation method and equipment. The method for constructing a defect image generation model comprises: obtaining a workpiece surface defect image; training the defect image generation model using the workpiece surface defect image to obtain a trained target defect image generation model, wherein the defect image generation model comprises a generator, a fusion attention module is introduced in a multi-scale feature fusion layer of the generator, the fusion attention module comprises a coordinate attention module, and the coordinate attention module is configured to obtain spatial perception features in horizontal and vertical directions by performing global average pooling in the horizontal and vertical directions on feature maps extracted from the workpiece surface defect image. Based on this, embodiments of the present application can enhance the perception ability of the spatial position of the defect and improve the diversity of the generated defect samples.

Description

Technical Field

[0001] This invention relates to the field of image sample generation technology, and in particular to a method for constructing a defect image generation model, a defect image generation method, and a device. Background Technology

[0002] In the industrial manufacturing sector, surface defect detection is a crucial step in ensuring product quality. While deep learning-based defect detection methods exhibit excellent performance, training these models relies on a large number of defect samples. However, in real-world industrial scenarios, the occurrence rate of defect samples is typically low, and the cost of collecting them is high, leading to a common problem of small sample size during model training and limiting the practical application of defect detection technology. In other words, existing image generation methods suffer from insufficient diversity in the generated defect samples. Summary of the Invention

[0003] This invention provides a method for constructing a defect image generation model, a defect image generation method, and an apparatus. By introducing a fusion attention module into the multi-scale feature fusion layer in the generator, the ability to perceive the spatial location of defects is enhanced, thereby improving the diversity of generated defect samples.

[0004] In a first aspect, embodiments of the present invention provide a method for constructing a defect image generation model, comprising: Acquire images of surface defects on the workpiece; The defect image generation model is trained using the workpiece surface defect image to obtain a trained target defect image generation model. The defect image generation model includes a generator, and a fusion attention module is introduced into the multi-scale feature fusion layer of the generator. The fusion attention module includes a coordinate attention module, which is used to obtain the spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image in the horizontal and vertical directions, respectively.

[0005] In some embodiments, the generator includes a random noise vector input layer, an upsampling module, and the fusion attention module; The input to the random noise vector input layer is a random noise vector; The construction formula for the upsampling module is as follows:

[0006] in, Indicates an upsampling operation. This represents an a×a convolution operation, where a is a positive integer. This indicates a batch normalization operation. This represents the Glu activation function. and These represent the input feature map and the output feature map, respectively, obtained through the upsampling module; The construction formula for the fusion attention module is as follows:

[0007] in, This indicates the average pooling operation. This represents a c×c convolution operation, where c is a positive integer. This represents a d×d convolution operation, where d is a positive integer. This indicates a batch normalization operation. Indicates splicing operation, This indicates a splitting operation. This represents the Swish activation function. This represents the Sigmoid activation function. , and These represent the normalized weights in the generated feature map channels, the horizontal direction of the feature map, and the vertical direction of the feature map, respectively. This represents the operation of multiplying the feature map with the generated weights. and These represent the small-scale and large-scale feature maps of the input, respectively. Represents the parameters of two-dimensional spatial fusion perception. and These represent the output feature maps of the channel attention module and the coordinate attention module, respectively.

[0008] In some embodiments, the defective image generation model further includes a discriminator, which includes a feature extraction network, an authenticity judgment module, and an image reconstruction decoder; The construction formula for the feature extraction network is as follows:

[0009] in, This represents the convolution operation. This indicates the jump channel excitation module. and The input image and output feature map of the jump channel excitation module are represented respectively; The construction formula for the jump channel excitation module is as follows:

[0010] in, This indicates a global average pooling operation. This represents the convolution operation. This represents the ReLU activation function. This represents the Sigmoid activation function. This represents the normalized weights on the channels of the generated feature map. This represents the operation of multiplying the feature map with the generated weights. and These represent the small-scale and large-scale feature maps of the input, respectively. This represents the output feature map of the jump channel excitation module; The image reconstruction decoder includes a first decoder and a second decoder for feature maps of different resolutions. The construction formulas for the first decoder and the second decoder are as follows:

[0011] in, Indicates an upsampling operation. This represents a deformable convolution operation. This indicates a batch normalization operation. This represents the ReLU activation function. and These represent the input image and output image of the first decoder, respectively. and These represent the input image and output image of the second decoder, respectively.

[0012] In some embodiments, the target defect image generation model is trained based on a target loss function, the expression of which is as follows:

[0013] in, Represents the target loss function. The adversarial loss function is used to optimize the generator and the discriminator, and the adversarial loss function includes a generator loss function and a discriminator loss function. The reconstruction loss function is used to measure the difference between the generated sample and the original sample. The reconstruction loss function includes the perceptual loss function, the L2 loss function, and the SSIM loss function.

[0014] In some embodiments, the expression for the adversarial loss function is as follows:

[0015] in, This represents the generator loss function, used to maximize the probability of the discriminator misclassifying the generated samples; Indicates the distribution from the standard normal distribution A random noise vector sampled in the middle; This indicates that the generator will generate a random noise vector. The process of converting to generate samples; The discriminator is used to evaluate the authenticity of the input sample; Indicates the expected value; The discriminator loss function is defined as including the loss for real samples, the loss for generated samples, and an additional reconstruction loss. ; This indicates that it is based on real data. Real samples from the middle, This indicates that it is generated by the generator. The generated samples, and These represent the outputs of the discriminator for the real sample and the generated sample, respectively.

[0016] In some embodiments, the expression for the reconstruction loss function is as follows:

[0017] in, Represents the reconstruction loss function. This represents the perceptual loss function, used to ensure the perceptual quality of the generated image; This represents the L2 loss function, used to enhance pixel-level reconstruction accuracy in background regions. These are the loss weights of the L2 loss function, with values ​​ranging from 0 to 1; This represents the SSIM loss function, used to preserve the structural information and texture details of an image. These are the loss weights of the SSIM loss function, with values ​​ranging from 0 to 1.

[0018] Secondly, embodiments of the present invention also provide a method for generating defect images, the method comprising: Obtain the random noise vector; A random noise vector is input into a trained generator to generate a target defect image set, wherein the trained generator is the generator in a trained defect image generation model; the trained defect image generation model is a target defect image generation model obtained by using the defect image generation model construction method as described in the first aspect.

[0019] In some embodiments, the generator includes an upsampling module and a fusion attention module, wherein inputting a random noise vector into the trained generator to generate a target defect image set includes: The random noise vector is converted into a feature map through a deconvolution operation; The size of the feature map is gradually increased by the upsampling module to obtain small-scale feature maps and large-scale feature maps respectively. The semantic information of the small-scale feature map is fused by the fusion attention module, and the large-scale feature map is subjected to global average pooling in the horizontal and vertical directions by the coordinate attention module to obtain the spatial perception features in the horizontal and vertical directions. The spatially perceptual features are channel-converted using a convolution operation to obtain an RGB image. The target defect image set is generated by normalizing the RGB image using an activation function.

[0020] In some embodiments, the method further includes: The average dissimilarity of each defect image in the target defect image set is calculated based on LPIPS distance to obtain a diversity score; Calculate the minimum LPIPS distance between each defect image and the set of real defect images, and then average the results to obtain the quality score. Based on a preset perception distance threshold, a coverage score is obtained according to the proportion of the target defect image set that effectively covers the real defect image set; The target defect image set is evaluated based on the diversity score, the quality score, and the coverage score to obtain the evaluation result.

[0021] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the method for constructing a defect image generation model as described in the first aspect, or the method for generating a defect image as described in the second aspect.

[0022] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for performing the method for constructing a defect image generation model as described in the first aspect, or the method for generating a defect image as described in the second aspect.

[0023] According to embodiments of the present invention, a method for constructing a defect image generation model, a defect image generation method, and an apparatus are provided. The method for constructing the defect image generation model includes: acquiring a workpiece surface defect image; training the defect image generation model using the workpiece surface defect image to obtain a trained target defect image generation model. The defect image generation model includes a generator, and a fusion attention module is introduced into the multi-scale feature fusion layer of the generator. The fusion attention module includes a coordinate attention module, which is used to obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image in both the horizontal and vertical directions. Based on this, the embodiments of the present invention, by introducing a fusion attention module into the multi-scale feature fusion layer of the generator, allows the coordinate attention module in the fusion attention module to obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image in both the horizontal and vertical directions, thereby enhancing the perception of the spatial location of defects and improving the diversity of generated defect samples. Attached Figure Description

[0024] Figure 1 This is the main flowchart of a method for constructing a defect image generation model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a fusion attention module provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of the model training process of a defect image generation model provided in one embodiment of the present invention; Figure 4 This is the main flowchart of a defect image generation method provided in one embodiment of the present invention; Figure 5 This is a sub-flowchart of step S202 in a defect image generation method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the process of generating a defect image using a generator according to an embodiment of the present invention; Figure 7 This is a flowchart of steps S401 to S404 in a defect image generation method provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the following drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0027] In this embodiment of the invention, the terms "furthermore," "exemplarily," or "optionally" are used to indicate examples, illustrations, or explanations, and should not be construed as being more preferred or advantageous than other embodiments or designs. The use of terms such as "furthermore," "exemplarily," or "optionally" is intended to present the relevant concepts in a specific manner.

[0028] To facilitate a more convenient description of the working principle of the embodiments of the present invention, the following introduction of relevant technical scenarios is given first.

[0029] In the industrial manufacturing sector, surface defect detection is a crucial step in ensuring product quality. While deep learning-based defect detection methods exhibit excellent performance, training these models relies on a large number of defect samples. However, in real-world industrial scenarios, the occurrence rate of defect samples is typically low, and the cost of collecting them is high, leading to a common problem of small sample size during model training and limiting the practical application of defect detection technology. In other words, existing image generation methods suffer from insufficient diversity in the generated defect samples.

[0030] To address the aforementioned technical problems, this invention provides a method for constructing a defect image generation model, a defect image generation method, and an apparatus. The method for constructing the defect image generation model includes: acquiring a workpiece surface defect image; training the defect image generation model using the workpiece surface defect image to obtain a trained target defect image generation model. The defect image generation model includes a generator, and a fusion attention module is introduced into the multi-scale feature fusion layer of the generator. The fusion attention module includes a coordinate attention module, which is used to obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image, respectively. Based on this, this embodiment of the invention, by introducing a fusion attention module into the multi-scale feature fusion layer of the generator, allows the coordinate attention module within the fusion attention module to obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image, thereby enhancing the ability to perceive the spatial location of defects and improving the diversity of generated defect samples.

[0031] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0032] like Figure 1 As shown, Figure 1 This is a flowchart of a method for constructing a defect image generation model according to an embodiment of the present invention. The method for constructing the defect image generation model may include, but is not limited to, steps S101 to S102.

[0033] Step S101: Obtain an image of the surface defects of the workpiece.

[0034] It is understandable that a workpiece surface defect image refers to an image of an industrial component with surface defects during the machining process, such as a surface defect image of a solid-state battery. Workpiece surface defect images can be two-dimensional or three-dimensional. For example, a small number of workpiece surface defect depth images can be acquired using a 3D line scan industrial camera, resulting in a three-dimensional workpiece surface defect image. Alternatively, the three-dimensional workpiece surface defect image can be further processed, such as by using pixel normalization methods to process the workpiece surface defect depth image to obtain a single-channel image of the workpiece surface defect, thus obtaining a two-dimensional workpiece surface defect image.

[0035] Step S102: The defect image generation model is trained using the workpiece surface defect image to obtain the trained target defect image generation model. The defect image generation model includes a generator. A fusion attention module is introduced into the multi-scale feature fusion layer of the generator. The fusion attention module includes a coordinate attention module. The coordinate attention module is used to obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image in the horizontal and vertical directions, respectively.

[0036] It is understood that the defect image generation model of the present invention is an adversarial generative model, including but not limited to an improved FastGAN network model. The defect image generation model includes a generator, which introduces a fusion attention module into the multi-scale feature fusion layer of the generator. The coordinate attention module in the fusion attention module can obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the defect image of the workpiece surface in the horizontal and vertical directions, respectively. This enhances the perception and representation ability of the spatial location of the defect and improves the diversity of generated defect samples.

[0037] In some embodiments, the generator includes a random noise vector input layer, an upsampling module, and a fusion attention module. The input to the random noise vector input layer is a random noise vector, for example, a 1x1x256 random noise vector. It should be noted that the number of channels in the random noise vector can be set according to actual conditions, and this embodiment of the invention does not impose a specific limitation on this.

[0038] in, Indicates an upsampling operation. This represents an a×a convolution operation, where a is a positive integer, preferably 3. This indicates a batch normalization operation. This represents the Glu activation function. and These represent the input and output feature maps obtained through the upsampling module, respectively.

[0039] Fusion attention module such as Figure 2 As shown, the construction formula for the fusion attention module is as follows:

[0040] in, This indicates the average pooling operation. This represents a c×c convolution operation, where c is a positive integer, and c is preferably 1. This represents a d×d convolution operation, where d is a positive integer, and the preferred value for d is 4. This indicates a batch normalization operation. Indicates splicing operation, This indicates a splitting operation. This represents the Swish activation function. This represents the Sigmoid activation function. , and These represent the normalized weights in the generated feature map channels, the horizontal direction of the feature map, and the vertical direction of the feature map, respectively. This represents the operation of multiplying the feature map with the generated weights. and These represent the small-scale and large-scale feature maps of the input, respectively. Represents the parameters of two-dimensional spatial fusion perception. and These represent the output feature maps from the channel attention module and the coordinate attention module, respectively. It should be noted that the feature scale ranges for the small-scale and large-scale feature maps can be set according to actual needs.

[0041] Understandably, this invention introduces a fusion attention module into the multi-scale feature fusion layer of the generator. The coordinate attention module in the fusion attention module can obtain spatially perceptual features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the defect images on the workpiece surface, respectively. Spatial attention weights are generated through convolutional coding and the Sigmoid activation function. The generated spatial attention weights are multiplied element-wise with the original feature maps to achieve adaptive modulation of the feature responses at different spatial locations. This enhances the model's precise control over the defect location, shape, and boundary, effectively improving the diversity of generated defects in spatial dimensions such as shape, size, and location. It solves the problem of the original model generating defects with a single shape and lack of spatial variation.

[0042] In some embodiments, the defective image generation model further includes a discriminator, which comprises a feature extraction network, a realism judgment module, and an image reconstruction decoder. The feature extraction network is constructed using the following formula:

[0043] in, This represents the convolution operation. This indicates the jump channel excitation module. and The input image and output feature map of the skip channel excitation module are represented respectively; The construction formula for the jump channel excitation module is as follows:

[0044] in, This indicates a global average pooling operation. This represents the convolution operation. This represents the ReLU activation function. This represents the Sigmoid activation function. This represents the normalized weights on the channels of the generated feature map. This represents the operation of multiplying the feature map with the generated weights. and These represent the small-scale and large-scale feature maps of the input, respectively. This represents the output feature map of the jump channel excitation module; The image reconstruction decoder includes a first decoder and a second decoder for feature maps of different resolutions. The construction formulas for the first decoder and the second decoder are as follows:

[0045] in, Indicates an upsampling operation. This represents a deformable convolution operation. This indicates a batch normalization operation. This represents the ReLU activation function. and These represent the input and output images of the first decoder, respectively. and These represent the input and output images of the second decoder, respectively.

[0046] Understandably, when processing a 1024×1024 resolution input image, the discriminator first extracts the image's feature representation step by step through a feature extraction network. This network consists of multiple convolutional layers and downsampling operations, which work together to progressively transform the high-resolution features of the input image into low-resolution but information-rich feature maps. During this process, the discriminator also introduces a skip channel activation module, which generates channel weights through global average pooling and 1×1 convolution operations. These weights are then applied to the lower-resolution feature maps to enhance the features of key channels. When the image resolution decreases to 8×8, the output feature map of the feature extraction network is input to the realism judgment module. This module further processes the feature map through a series of convolutional layers, batch normalization, and activation function operations, ultimately outputting a scalar value representing the probability that the input image is real. This probability value not only reflects the discriminator's judgment of the input image's realism but also provides important feedback signals to the generator, guiding it to generate more realistic images.

[0047] Furthermore, due to the complexity and diversity of defect feature information, this invention introduces deformable convolution in the image reconstruction decoder to further enhance the discriminator's adaptability to complex geometric structures. Deformable convolution dynamically adjusts the sampling position of the convolution kernel by learning offsets, enabling the kernel to better adapt to geometric deformations in the input feature map. This design allows the image reconstruction decoder to more effectively reconstruct local and global features of the image, providing a stronger self-supervised learning signal for the discriminator.

[0048] It is understandable that, such as Figure 3 As shown, during the model training phase, random noise vectors are used by the generator to generate fake images. The discriminator outputs a true or false judgment on the acquired defective real images and the generated fake images through the feature extraction network and the authenticity judgment module. Based on the loss value between the fake images and the real images, the generator and the discriminator continuously optimize and update their weights through adversarial training.

[0049] In some embodiments, the target defect image generation model is trained based on a target loss function, the expression of which is as follows:

[0050] in, Represents the target loss function. This represents the adversarial loss function, used to optimize the generator and discriminator. The adversarial loss function includes the generator loss function and the discriminator loss function. The reconstruction loss function measures the difference between the generated sample and the original sample. Reconstruction loss functions include perceptual loss, L2 loss, and SSIM loss. Their function is to directly optimize the discriminator's reconstruction capability through supervisory signals, enabling the discriminator to learn more effective feature representations. The model then undergoes adversarial training, indirectly optimizing the generator's generation performance using gradients provided by the discriminator.

[0051] Understandably, this invention introduces an L2 loss function based on pixel-level constraints and an SSIM loss function based on structural similarity into the discriminator loss function. The L2 loss function (mean squared error loss function) strengthens the constraint on accurate reconstruction of non-defective regions (background) by minimizing the Euclidean distance between the generated image and the original image in pixel space, thereby improving the pixel-level fidelity of the generated image. The SSIM loss function calculates the similarity between the generated image and the original image in three dimensions: brightness, contrast, and structure, preserving the global structural information and local texture features of the image, and avoiding the over-smoothing problem caused by L2 loss.

[0052] In some embodiments, the adversarial loss function is expressed as follows:

[0053] in, This represents the generator loss function, used to maximize the probability of the discriminator misclassifying the generated samples; Indicates the distribution from the standard normal distribution A random noise vector sampled in the middle; This indicates that the generator will generate a random noise vector. The process of converting to generate samples; This represents a discriminator used to evaluate the authenticity of input samples; Indicates the expected value; This represents the discriminator loss function, which includes the loss for real samples, the loss for generated samples, and an additional reconstruction loss. ; This indicates that it is based on real data. Real samples from the middle, This indicates that it is generated by the generator. The generated samples, and These represent the discriminator's outputs for real samples and generated samples, respectively. This represents the reconstruction loss function, used to measure the difference between the generated sample and the original sample.

[0054] In some embodiments, the expression for the reconstruction loss function is as follows:

[0055] in, Represents the reconstruction loss function. This represents the perceptual loss function, used to ensure the perceptual quality of the generated image; This represents the L2 loss function, used to enhance pixel-level reconstruction accuracy in background regions. These are the loss weights of the L2 loss function, which can be set in the training parameters, and their values ​​range from 0 to 1. This represents the SSIM loss function, used to preserve the structural information and texture details of an image. These are the loss weights of the SSIM loss function, which can be set in the training parameters, with a value range of 0 to 1.

[0056] Based on this, this invention employs a composite target loss function composed of adversarial loss, perceptual loss, L2 loss, and SSIM loss. This target loss function is used to improve the quality of generated images, enhance defect diversity, and maintain the stability of the training process. Adjustable weight coefficients are also introduced. and It achieves a dynamic balance between adversarial loss, perceptual loss, L2 loss and SSIM loss, taking into account the realism and structural integrity of the generated image, effectively solving the problem of inaccurate background reconstruction outside the defect area in the original model, and significantly improving the overall visual quality of the generated image.

[0057] like Figure 4 As shown, Figure 4 This is a flowchart of a defect image generation method provided in an embodiment of the present invention. The defect image generation method may include, but is not limited to, steps S201 to S202.

[0058] Step S201: Obtain the random noise vector; Step S202: Input the random noise vector into the trained generator to generate a target defect image set. The trained generator is the generator in the trained defect image generation model. The trained defect image generation model is the target defect image generation model obtained by using the above-mentioned defect image generation model construction method.

[0059] Understandably, in the defect image generation stage, a random noise vector is input into a trained generator to generate a target defect image set. Since the trained generator is the generator in the target defect image generation model derived from the aforementioned defect image generation model construction method, it can generate a large number of realistic defect samples. Based on this, the present invention can generate a large number of high-quality fake defect samples, effectively solving the defect image generation problem in small sample scenarios.

[0060] In some embodiments, such as Figure 5 As shown, step S202 includes, but is not limited to, steps S301 to S305.

[0061] Step S301: Convert the random noise vector into a feature map through deconvolution operation; Step S302: The size of the feature map is gradually increased through the upsampling module to obtain small-scale feature maps and large-scale feature maps respectively; Step S303: The semantic information of the small-scale feature map is fused by the fusion attention module, and the large-scale feature map is subjected to global average pooling in the horizontal and vertical directions by the coordinate attention module to obtain the spatial perception features in the horizontal and vertical directions. Step S304: Perform channel conversion on the spatially perceptual features through convolution operation to obtain an RGB image; Step S305: Normalize the RGB image using an activation function to generate a target defect image set.

[0062] Understandably, the specific process by which the generator generates defect images is as follows: (e.g.) Figure 6As shown, the generator first takes a 256-dimensional random noise vector sampled from a normal distribution as input, and converts it into a 4×4×1024 feature map through a deconvolution layer. Then, an upsampling module gradually expands the feature map size. At three key scales—128×128, 256×256, and 512×512—the SCA_Block (fusion attention module) first fuses semantic information from smaller scale feature maps of 4×4, 8×8, 16×16, 32×32, and 64×64 respectively to ensure the richness of the generated content. Then, at 128×128, 2... At the 56×256 and 512*512 scales, a coordinate attention module is further applied. This module performs global average pooling in both the horizontal and vertical directions on the large-scale feature maps of 128×128, 256×256, and 512*512, obtaining spatially perceptual features in the horizontal and vertical directions. This enhances the ability to perceive the spatial location of defects and improves the diversity of generated defects in shape, size, and location. Finally, a 1×1 convolution converts the 512-channel features into a 3-channel RGB image, which is then processed by a Tanh activation function and output to the range [-1, 1], generating the final 512×512 defect image. The specific calculation process is as follows:

[0063] in, This indicates the fusion attention module. Indicates the upsampling module. and These represent the input and output feature maps, respectively, after passing through the fusion attention module and the upsampling module. This indicates the deconvolution operation. This indicates a convolution operation, with SCU representing fusion upsampling. Represents a random noise vector. This represents the defect image output by the generator.

[0064] Understandably, applying the coordinate attention module to large-scale feature maps based on the input image resolution—for example, if the input image resolution is 512×512, applying the coordinate attention module to the three large-scale feature maps of 128×128, 256×256, and 512×512—can avoid introducing unnecessary computational overhead at excessively low or high scales.

[0065] In some embodiments, such as Figure 7 As shown, this also includes, but is not limited to, steps S401 to S404: Step S401: Calculate the average dissimilarity of each defect image in the target defect image set based on LPIPS distance to obtain the diversity score; Step S402: Calculate the minimum LPIPS distance between each defect image and the set of real defect images, and calculate the average value to obtain the quality score; Step S403: Based on a preset perception distance threshold, obtain a coverage score according to the proportion of the target defect image set that effectively covers the real defect image set; Step S404: Evaluate the target defect image set based on diversity score, quality score, and coverage score to obtain the evaluation results.

[0066] Understandably, traditional generated image evaluation metrics (such as FID, IS, and SSIM) are based on large-sample statistical settings, typically requiring at least a thousand real images to obtain reliable evaluation results. However, in real-world industrial defect detection scenarios, obtaining real defect samples is costly, with only a few dozen samples usually available. This sample scarcity causes traditional evaluation metrics to fail in small-sample scenarios: FID and IS metrics cannot construct a stable feature distribution due to insufficient sample size; while the SSIM metric can calculate the similarity between single image pairs, it cannot effectively assess the diversity of generated images and their coverage of the real defect distribution.

[0067] Based on this, the present invention adopts a multi-dimensional quantitative evaluation method based on perceptual similarity to evaluate the generated defective images. Specifically, it achieves effective evaluation of the quality of generated images under small sample conditions by using three complementary score indicators: Diversity Score, Quality Score, and Coverage Score.

[0068] Understandably, the average difference between each defect image in the target defect image set is calculated based on the LPIPS (Learned Perceptual Image Patch Similarity) distance, resulting in a diversity score. The average difference within the generated image set is also calculated based on the LPIPS distance. By randomly sampling image pairs and calculating their perceptual distance, the degree of variation in features such as defect shape, size, and location is quantified. A higher score indicates better diversity in the generated images, effectively avoiding mode collapse problems.

[0069] Understandably, the quality score is calculated by averaging the minimum LPIPS distance between each generated image and the set of real defect images. This metric measures the visual similarity between the generated image and the real defect; a lower score indicates that the generated image is closer to the visual features of the real defect, and thus, the higher the generation quality.

[0070] Understandably, based on a preset perceptual distance threshold, a coverage score is obtained according to the proportion of the target defect image set that effectively covers the real defect image set. Specifically, based on the preset perceptual distance threshold, the proportion of real defect samples that can be effectively covered by the generated image set is statistically analyzed. This metric evaluates the generative model's ability to represent the distribution of real defects; a higher coverage rate indicates that the generated images can cover more types of real defect features.

[0071] This invention introduces the LPIPS metric to evaluate the quality of defect images generated by the model. By randomly sampling image pairs and calculating their perceptual distance, the degree of variation in features such as defect shape, size, and location of the generated images is quantified. This can be extended to calculate the statistical average of the minimum LPIPS distance between each generated image and the set of real defect images, or to statistically determine the proportion of real defect samples that can be effectively covered by the generated image set. This allows for an effective assessment of the diversity of the generated images and their coverage of the real defect distribution.

[0072] Based on the complementary nature of the three indicators—diversity score, quality score, and coverage score—the generated image quality evaluation results are suitable for small sample scenarios, achieving a comprehensive quantitative assessment of the similarity, diversity, and distribution coverage of the generated images.

[0073] In addition, such as Figure 8 As shown, an embodiment of the present invention also discloses an electronic device, including: at least one processor 210; at least one memory 220 for storing at least one program; when the at least one program is executed by the at least one processor 210, it implements a method for constructing a defect image generation model as in any of the preceding embodiments, or a method for generating a defect image as in any of the preceding embodiments.

[0074] In addition, one embodiment of the present invention discloses a computer-readable storage medium storing computer-executable instructions for performing a method for constructing a defect image generation model as described in any of the preceding embodiments, or a method for generating a defect image as described in any of the preceding embodiments.

[0075] The system architecture and application scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0076] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0077] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0078] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process or execution thread, and components may be located on a single computer or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems via signals).

Claims

1. A method for constructing a defect image generation model, characterized in that, The method includes: Acquire images of surface defects on the workpiece; The defect image generation model is trained using the workpiece surface defect image to obtain a trained target defect image generation model. The defect image generation model includes a generator, and a fusion attention module is introduced into the multi-scale feature fusion layer of the generator. The fusion attention module includes a coordinate attention module, which is used to obtain the spatial awareness features in the horizontal and vertical directions by performing global average pooling on the feature maps extracted from the workpiece surface defect image in the horizontal and vertical directions, respectively.

2. The method according to claim 1, characterized in that, The generator includes a random noise vector input layer, an upsampling module, and the fusion attention module; The input to the random noise vector input layer is a random noise vector; The construction formula for the upsampling module is as follows: in, Indicates an upsampling operation. This represents an a×a convolution operation, where a is a positive integer. This indicates a batch normalization operation. This represents the Glu activation function. and These represent the input feature map and the output feature map, respectively, obtained through the upsampling module; The construction formula for the fusion attention module is as follows: in, This indicates the average pooling operation. This represents a c×c convolution operation, where c is a positive integer. This represents a d×d convolution operation, where d is a positive integer. This indicates a batch normalization operation. Indicates splicing operation, This indicates a splitting operation. This represents the Swish activation function. This represents the Sigmoid activation function. , and These represent the normalized weights in the generated feature map channels, the horizontal direction of the feature map, and the vertical direction of the feature map, respectively. This represents the operation of multiplying the feature map with the generated weights. and These represent the small-scale and large-scale feature maps of the input, respectively. Represents the parameters of two-dimensional spatial fusion perception. and These represent the output feature maps of the channel attention module and the coordinate attention module, respectively.

3. The method according to claim 1, characterized in that, The defect image generation model also includes a discriminator, which comprises a feature extraction network, an authenticity judgment module, and an image reconstruction decoder; The construction formula for the feature extraction network is as follows: in, This represents the convolution operation. This indicates the jump channel excitation module. and The input image and output feature map of the jump channel excitation module are represented respectively; The construction formula for the jump channel excitation module is as follows: in, This indicates a global average pooling operation. This represents the convolution operation. This represents the ReLU activation function. This represents the Sigmoid activation function. This represents the normalized weights on the channels of the generated feature map. This represents the operation of multiplying the feature map with the generated weights. and These represent the small-scale and large-scale feature maps of the input, respectively. This represents the output feature map of the jump channel excitation module; The image reconstruction decoder includes a first decoder and a second decoder for feature maps of different resolutions. The construction formulas for the first decoder and the second decoder are as follows: in, Indicates an upsampling operation. This represents a deformable convolution operation. This indicates a batch normalization operation. This represents the ReLU activation function. and These represent the input image and output image of the first decoder, respectively. and These represent the input image and output image of the second decoder, respectively.

4. The method according to claim 3, characterized in that, The target defect image generation model is trained based on a target loss function, the expression of which is as follows: in, Represents the target loss function. The adversarial loss function is used to optimize the generator and the discriminator, and the adversarial loss function includes a generator loss function and a discriminator loss function. The reconstruction loss function is used to measure the difference between the generated sample and the original sample. The reconstruction loss function includes the perceptual loss function, the L2 loss function, and the SSIM loss function.

5. The method according to claim 4, characterized in that, The expression for the adversarial loss function is as follows: in, This represents the generator loss function, used to maximize the probability of the discriminator misclassifying the generated samples; Indicates the distribution from the standard normal distribution A random noise vector sampled in the middle; This indicates that the generator will generate a random noise vector. The process of converting to generate samples; The discriminator is used to evaluate the authenticity of the input sample; Indicates the expected value; The discriminator loss function is defined as including the loss for real samples, the loss for generated samples, and an additional reconstruction loss. ; This indicates that it is based on real data. Real samples from the middle, This indicates that it is generated by the generator. The generated samples, and These represent the outputs of the discriminator for the real sample and the generated sample, respectively.

6. The method according to claim 4, characterized in that, The expression for the reconstruction loss function is as follows: in, Represents the reconstruction loss function. This represents the perceptual loss function, used to ensure the perceptual quality of the generated image; This represents the L2 loss function, used to enhance pixel-level reconstruction accuracy in background regions. These are the loss weights of the L2 loss function, with values ​​ranging from 0 to 1; This represents the SSIM loss function, used to preserve the structural information and texture details of an image. These are the loss weights of the SSIM loss function, with values ​​ranging from 0 to 1.

7. A method for generating defect images, characterized in that, The method includes: Obtain the random noise vector; A random noise vector is input into a trained generator to generate a target defect image set, wherein the trained generator is the generator in a trained defect image generation model; the trained defect image generation model is a target defect image generation model obtained by the construction method of the defect image generation model as described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that, The generator includes an upsampling module and a fusion attention module. The step of inputting a random noise vector into the trained generator to generate a target defect image set includes: The random noise vector is converted into a feature map through a deconvolution operation; The size of the feature map is gradually increased by the upsampling module to obtain small-scale feature maps and large-scale feature maps respectively. The semantic information of the small-scale feature map is fused by the fusion attention module, and the large-scale feature map is subjected to global average pooling in the horizontal and vertical directions by the coordinate attention module to obtain the spatial perception features in the horizontal and vertical directions. The spatially perceptual features are channel-converted using a convolution operation to obtain an RGB image. The target defect image set is generated by normalizing the RGB image using an activation function.

9. The method according to claim 7, characterized in that, The method further includes: The average dissimilarity of each defect image in the target defect image set is calculated based on LPIPS distance to obtain a diversity score; Calculate the minimum LPIPS distance between each defect image and the set of real defect images, and then average the results to obtain the quality score. Based on a preset perception distance threshold, a coverage score is obtained according to the proportion of the target defect image set that effectively covers the real defect image set; The target defect image set is evaluated based on the diversity score, the quality score, and the coverage score to obtain the evaluation result.

10. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements a method for constructing a defect image generation model as described in any one of claims 1 to 6, or a method for generating a defect image as described in any one of claims 7 to 9.

11. A computer-readable storage medium storing computer-executable instructions for performing a method for constructing a defect image generation model as claimed in any one of claims 1 to 6, or a method for generating a defect image as claimed in any one of claims 7 to 9.

Images