Industrial defect image generation method, terminal device, and storage medium

Abstract

The application discloses an industrial defect image generation method, a terminal device and a storage medium. Qualified sample images and real defect images of industrial products are collected, and a training set is constructed by using the qualified sample images and the real defect images. The training set is used as input of an AG-GAN network model, the AG-GAN network model is trained, and an image generation model is obtained. The application can accurately identify rare defects, and greatly improves the reliability of product quality detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an industrial defect image generation method, a terminal device and a storage medium. BACKGROUND

[0002] Industrial defect detection is a key link to ensure product quality. A detection method based on deep learning needs to rely on a large number of labeled defect samples for model training. However, in actual production, qualified products account for a high proportion, defect samples (such as cracks, depressions, scratches, etc.) are few in number and various in type, and some defects (such as small cracks) are difficult to collect. In a single production line, the ratio of defect samples to qualified samples is often less than 1:1000. The model is prone to overfitting, and the recognition accuracy for defects that have not been seen is low. Moreover, the morphology, position and severity of natural defects differ greatly, and existing data enhancement methods (such as rotation and cropping) are difficult to generate new defect samples with real physical characteristics. Defect images generated by the generative adversarial network (GAN) have problems such as blurring, detail distortion, and low background fusion, which cannot effectively improve the performance of the detection model.

[0003] CN120374630A discloses a multi-modal feature and prompt mechanism for over-flow surface defect detection and recognition method and system, which needs to be improved in terms of accurate attention to defect features, model decision stability, physical mechanism compatibility and complex environment adaptability. These defects limit the accuracy and reliability of detection, affecting the quality control and safety protection of industrial production.

[0004] Attention weight explanation has limitations: in the mechanical fault diagnosis scene, there is an asymmetry between attention weight and feature importance. High attention weight may be due to the model's over-response to noise, such as random resonance interference in bearing signals being mislabeled as fault impact, while real fault features such as early cracks may be assigned low weights due to low energy. In solar cell defect detection, if affected by factors such as light and pollutants, the attention mechanism may be disturbed, leading to a decrease in detection accuracy for small defects. This shows that the attention mechanism is sensitive to noise and is prone to misjudging non-defect features as defects, thereby affecting the accuracy of detection.

[0005] Decision vulnerability: the decision vulnerability of the attention mechanism is reflected in the adversarial perturbation experiment, where a slight change in the phase of the input signal can generate a model output with different weight distribution but the same diagnosis result, exposing the sensitivity of the weight to input perturbation far beyond the physical law constraint. This means that even minor environmental changes or data fluctuations in actual application can cause significant changes in the weight distribution of the attention mechanism, thereby affecting the stability and reliability of defect detection.

[0006] Insufficient consideration of physical mechanism: attention weight is essentially a byproduct of model training objectives, and is not an interpretable tool for fault physical mechanism. In bearing outer ring fault diagnosis, the model may assign high weights to the bearing assembly resonance frequency band unrelated to the fault, simply because the band is statistically related to the fault label in the training set, but ignores the actual physical location of the fault characteristic frequency. This shows that the existing defect detection scheme with attention mechanism has shortcomings in understanding and reflecting the physical essence of defects, which may lead to missed detection or misjudgment.

[0007] Insufficient adaptability to complex environment: in complex industrial environments, such as mobile phone cover surface defect detection, the existing deep learning detection model may cause incomplete defect detection when facing interference factors such as shadow shielding and complex background. Although some schemes introduce attention mechanism to improve feature expression ability, there are still limitations in dealing with complex environments, and it is difficult to detect all types of defects comprehensively and accurately. SUMMARY

[0008] The technical problem to be solved by the present application is to provide an industrial defect image generation method, a terminal device and a storage medium to improve the reliability of product quality detection.

[0009] To solve the above technical problems, the technical solution adopted by the present application is as follows: an industrial defect image generation method, comprising the following steps:

[0010] S1, collecting qualified sample images and real defect images of industrial products, and using the qualified sample images and real defect images to construct a training set;

[0011] S2, using the training set as the input of an AG-GAN network model, training the AG-GAN network model to obtain an image generation model;

[0012] The AG-GAN network model comprises:

[0013] a generator for extracting multi-scale feature maps of the input qualified sample images, and generating defect images under the guidance of an attention guiding module using the multi-scale feature maps;

[0014] a discriminator for dividing real defect images and generated defect images into a plurality of sub-regions, and obtaining the authenticity probability of each sub-region.

[0015] The present application uses a generator to generate defect images, i.e. uses generated samples to expand the training set, which can accurately identify rare defects and greatly improves the reliability of product quality detection.

[0016] The attention guiding module comprises a spatial attention submodule and a channel attention submodule; wherein:

[0017] The spatial attention submodule generates a spatial attention map by using the multi-scale features;

[0018] The channel attention submodule performs an all-average pooling operation on the multi-scale feature maps to obtain global statistical information of channels, inputs the global statistical information into a multilayer perceptron, and obtains a channel attention map.

[0019] Under the guidance of the attention guiding module, the specific implementation process of generating a defect image comprises:

[0020] The spatial attention map and the channel attention map are respectively multiplied with the corresponding multi-scale feature maps to obtain a defect image.

[0021] The attention guiding module focuses on the defect area, and the generated defect image is clear in details, for example, the crack edge is continuous, the depth gradient of the depression is natural, and the fusion degree with the background is high.

[0022] The loss function of the AG-GAN network model comprises a generation loss and a discrimination loss , which are defined as follows:

[0023] ;

[0024] ;

[0025] wherein, is an adversarial loss, is an attention guiding loss, is a semantic feature L1 loss, is a physical form constraint loss, and λ1, λ2 and λ3 are weight coefficients, is an adversarial loss in the form of Wasserstein distance, which ensures that the discriminator can effectively distinguish between real defect images and generated defect images. is a gradient penalty term, is a weight of the gradient penalty term, which is used to constrain the discriminator to satisfy the Lipschitz continuity and ensure the stability of training.

[0026] ; wherein, represents the sum of all pixel positions (i,j) on the spatial attention map, i represents the row index of the pixel, j represents the column index of the pixel, M(i,j) is the value of the defect mask obtained by the real defect pattern annotation at the pixel position (i,j), the mask is a binary image, which is used to indicate the defect area and the background area, and A(i,j) is the value of the spatial attention map generated by the generator at the pixel position (i,j).

[0027] ; wherein, ω1, ω2 are weights, , , E is a set of crack edge pixels, and ∇I(x,y) is a crack edge pixel gradient value, is an ideal gradient mean value, D is a set of recessed area pixels, and ∇dG(x,y), ∇dR(x,y) are depth gradients of the generated recessed image and the real recessed image, respectively.

[0028] ; is a feature extraction part of the pre-trained defect classification model, G(z) is a generated defect image, and R is a real defect image.

[0029] The present application can generate defects of different types and different forms through noise adjustment and physical prior constraints, and can cover the generation of defects in electronic components in actual production, and can generate component damage defects of different sizes, shapes and degrees, thereby meeting the demand of detection models for diversified defect samples.

[0030] As an inventive concept, the present application also provides a terminal device comprising a memory, a processor and a computer program stored in the memory; the processor executes the computer program to realize the steps of the above method.

[0031] As an inventive concept, the present application also provides a computer readable storage medium having a computer program / instruction stored thereon; the computer program / instruction is executed by a processor to realize the steps of the above method.

[0032] Compared with the prior art, the application has the beneficial effects that: the application focuses on the defect area through the attention guiding module, the generated defect image is clear in detail, the crack edge is continuous, the depth gradient of the concave is natural, and the fusion degree with the background is high, in the generation of the defect image of the metal component, the generated crack defect edge can keep continuous and smooth, which is very similar to the shape of the real crack, the depth change of the concave defect also conforms to the physical law, and the transition between the defect and the background is natural, and there is no obvious boundary mismatch problem, different types and different forms of defects can be generated through noise adjustment and physical prior constraint, in the generation of the defect of the electronic component in actual production, different sizes, shapes and different degrees of component damage defects can be generated, the demand of the detection model for diversified defect samples is met, after the generated samples are used to expand the training set, rare defects can be accurately recognized, the reliability of product quality detection is greatly improved, and the application can be suitable for metal, electronic, textile and other industrial products, without the need to redesign the network structure for specific scenes, only the physical prior constraint parameters need to be adjusted, and the adaptability is improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The method flowchart of the embodiment of the application is shown in the figure.

[0034] Figure 2 The AG-GAN network model architecture diagram of the embodiment of the application is shown in the figure.

[0035] Figure 3 The encoder and decoder connection structure diagram of the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the embodiment of the application clearer, the technical scheme in the embodiment of the application will be clearly and completely described below in combination with the drawings in the embodiment of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0037] Embodiment 1

[0038] As shown in the figure, the embodiment 1 of the application provides an industrial defect image generation method based on an attention guiding mechanism generative adversarial network, which includes the following steps: Figure 1

[0039] Step 1: build an AG-GAN network architecture (as shown in the figure) Figure 2

[0040] ​​(1) Generator: The encoder-decoder structure is adopted. The encoder extracts multi-scale features of the input qualified sample image through 5 convolution layers, and the resolution is reduced from 256x256 to 8x8. In each convolution operation, the size of the convolution kernel is 7x7, 5x5, 3x3, 3x3, and 3x3, respectively, and the step is 2, 2, 2, 2, respectively. By such setting, the size of the image can be gradually reduced and different scale feature information can be extracted. The output feature map of each layer is processed by batch normalization and LeakyReLU activation. Batch normalization can accelerate the convergence of the model and improve the stability of the model. LeakyReLU activation function can effectively solve the problem of gradient disappearance, and the negative slope is set to 0.2.

[0041] The decoder restores the image resolution through 5 deconvolution layers. The size of the deconvolution kernel is also 7x7, 5x5, 3x3, 3x3, and 3x3, respectively, and the step is 2. A skip connection is set between the encoder and the decoder (as shown in Figure 3 ), which can directly transmit the texture and edge information of the shallow layer in the encoder to the corresponding layer of the decoder, and fuse it with the defect contour and overall structure features of the deep layer semantics, so as to generate more rich and accurate defect images. The output layer uses Tanh activation function to generate defect images with the same size as the input image, and the pixel value range is [-1, 1].

[0042] (2) Discriminator: The PatchGAN structure is adopted, which divides the real defect image or generated defect image into 70x70 local regions, and outputs the authenticity probability of each region. The discriminator receives the defect image output by the generator, the corresponding attention map and the real defect label, and enhances the targeted discrimination of the defect area.

[0043] (3) Attention guiding module: including spatial attention submodule and channel attention submodule.

[0044] Spatial attention submodule: The multi-scale feature map output by the encoder is processed by 1x1 convolution with Sigmoid activation to generate a spatial attention map, and the calculation formula is as follows:

[0045] ;

[0046] Where F is the input feature map, is the 1x1 convolution operation, σ is the Sigmoid activation function, and As(F) is the generated spatial attention map.

[0047] Channel attention sub-module: generate channel weights through global average pooling and multi-layer perceptron, first perform global average pooling operation on the feature map, compress the features of each channel into a value, and obtain the global statistical information of the channel. Then input these statistical information into the multi-layer perceptron, and obtain the weight of each channel after the processing of two fully connected layers, wherein the number of neurons in the first layer is 1 / 16 of the number of channels, and the number of neurons in the second layer is restored to the number of channels, and the calculation formula is:

[0048] ;

[0049] Wherein, GAP is the global average pooling operation, MLP is the multi-layer perceptron, σ is the Sigmoid activation function, and Ac(F) is the generated channel attention map.

[0050] The attention map and the feature map of the generator are fused by element-wise multiplication, which guides the generator to focus on the defect area and suppresses the background irrelevant features. In the fusion process, the spatial attention map and the channel attention map are respectively multiplied with the feature map of the corresponding scale of the generator, so that the generator can pay more attention to the defect area and reduce the interference of background irrelevant information when generating the defect image according to the guidance of the attention map.

[0051] (4) Defect feature constraint module: this module is independent of the discriminator, and the input is the generated defect image and the real defect image itself. This module uses a pre-trained defect classification model as a feature extractor to extract high-order semantic features (such as defect type and shape features) of the generated defect image and the real defect image, and calculates the constraint loss based on the feature difference between the two to ensure that the generated image is consistent with the real defect in the semantic level.

[0052] The L1 distance can measure the difference between the generated defect image and the real defect image in the feature space, and by minimizing this difference, the generated defect image is more similar to the real defect image in the semantic level, and the calculation formula is:

[0053] ;

[0054] Wherein, is the feature extraction part of the pre-trained defect classification model, G(z) is the generated defect image, and R is the real defect image.

[0055] The defect feature constraint module and the discriminator are parallel supervision branches: the discriminator optimizes the pixel-level realism of the generated image through the "real probability", and this module constrains the semantic rationality (such as whether the defect type and shape conform to the real law) of the generated defect through the "semantic feature distance", and the two constitute a multi-dimensional loss function of the generator, and are independent of each other.

[0056] Adding physical prior constraints: For different defect types, design morphological constraint loss (denoted as L morph ), as the core sub-item of defect feature constraint loss, specifically includes:

[0057] Metallic crack defects: The smoothness is measured by calculating the first or second derivative change of the crack edge, and the smoothness loss calculation formula is: ; E is the crack edge pixel set, and ▽I(x,y) is the crack edge pixel gradient value, , which is the ideal gradient mean value. This loss constraint ensures that the crack edge conforms to the physical smoothness of metal crack propagation, avoiding the generation of false cracks with broken edges.

[0058] Concave defects: By comparing the change of pixel gray value in the concave area with the real concave depth gradient, the gradient consistency loss is constructed: ; D is the concave area pixel set, and ▽dG(x,y), ▽dR(x,y) are the generated and real concave depth gradients, respectively. This loss ensures that the concave depth change conforms to the physical logic, avoiding depth mutation.

[0059] Physical morphological constraint loss composed of the above sub-items , ω1, ω2 are weights, which can be adjusted according to the priority of defects. Finally, it is combined with the semantic feature L1 loss to form the total loss of the generator, realizing the dual constraint of physical morphology and semantics.

[0060] This smoothness index is added as a loss term to the total loss, making the generated crack edge more consistent with the actual physical law; for concave defects, by comparing the change of pixel gray value in the concave area with the real concave depth gradient, the gradient consistency loss is constructed, and its calculation formula is:

[0061] ;

[0062] where D is the concave area pixel set, and ▽dG(x,y) is the depth gradient of pixel (x,y) in the generated concave image, and ▽dR(x,y) is the depth gradient of the corresponding pixel in the real concave image.

[0063] Step two: training AG-GAN network

[0064] (1) Data set preparation: Collect qualified sample images and a small amount of real defect images of industrial products to build a training set. When collecting qualified sample images of metal plates, images are obtained from different production batches and different shooting angles to increase the diversity of samples. For a small amount of real defect images, the type, position coordinates in the image, and defect bounding box information of the defect are labeled in detail.

[0065] (2) Initialization parameters: the weights of the generator and the discriminator are initialized by He, which automatically adjusts the initialization parameters according to the input and output dimensions of the network layer, so that the network can converge faster at the beginning of training. The learning rate is set to 0.0002, the batch size is 16, and the training round is 200.

[0066] (3) Alternating training

[0067] A. Fix the discriminator and update the generator: minimize the generation loss to make the generated image as close to the real defect as possible. In the calculation of the adversarial loss, the Wasserstein distance combined with the gradient penalty is used. The Wasserstein distance can measure the difference between the generated distribution and the real distribution, and the gradient penalty ensures the Lipschitz continuity of the discriminator, preventing gradient vanishing or gradient explosion during model training. The attention-guided loss is calculated by comparing the difference between the attention map generated by the generator and the real defect mask. The cross-entropy loss function is used to make the attention map generated by the generator accurately focus on the real defect area. The defect feature constraint loss is as described in step 1, including semantic feature L1 loss and physical form constraint loss. By minimizing these losses, the generator generates defect images that are similar to real defect images in terms of semantics and physical form. The calculation formula of the generation loss is: ; Wherein: is the adversarial loss (Wasserstein distance) that optimizes the pixel-level realism of the generated image. is the attention-guided loss (cross-entropy) that constrains the generator to focus on the real defect area. is the semantic feature L1 loss that ensures the high-order semantics (type, form) of the generated defect consistent with the real defect. is the physical form constraint loss, which contains (crack smoothness) and (depression gradient consistency) sub-items that constrain the generated defect to comply with industrial physical laws; λ1, λ2, λ3 and ω1, ω2 are weights that can be adjusted according to the priority of defect detection in the industrial scene (such as increasing ω1 to strengthen crack constraint in metal component detection).

[0068] B. Fix the generator and update the discriminator: maximize the discrimination loss to make the discriminator accurately distinguish between real defects and generated defects. The loss function of the discriminator is determined by calculating the difference between the discrimination results of real defect images and generated defect images. By maximizing this loss, the discriminator can continuously improve its ability to distinguish between real defects and generated defects, guiding the training of the generator. The calculation formula of the discrimination loss is: ; Wherein, The Wasserstein distance is used as the form of the adversarial loss to ensure that the discriminator can effectively distinguish between real defect images and generated defect images. The gradient penalty term is used as the form of the adversarial loss to ensure that the discriminator can effectively distinguish between real defect images and generated defect images. The gradient penalty term is used as the form of the adversarial loss to ensure that the discriminator can effectively distinguish between real defect images and generated defect images. D(G(z)) represents the authenticity probability distribution calculated by the discriminator when inputting the generated defect image G(z), D(R) represents the authenticity probability distribution calculated by the discriminator when inputting the real defect image R, and E[·] represents the expectation operation on the sample space.

[0069] (4) Loss function design:

[0070] A. Adversarial loss: The Wasserstein distance combined with the gradient penalty (WGAN-GP) method is used to improve the training stability of the generative adversarial network. The core idea is to construct interpolation samples between real defect images and generated defect images to constrain the gradient of the discriminator and ensure that it satisfies the Lipschitz continuity. The specific calculation method is as follows: ; where R is the real defect image, and G(z) is the generated defect image. , represents a random number uniformly sampled on the interval [0, 1]; is the interpolation sample, which is used to construct boundary samples and constrain the discriminator.

[0071] The interpolation sample is input into the discriminator D, and its gradient is calculated, i.e., the gradient of the discriminator output with respect to the input . This gradient reflects the sensitivity of the discriminator to input changes. In order to ensure that the discriminator satisfies the Lipschitz continuity, a gradient penalty term is introduced: ; where: is the expectation of the interpolation sample , and is the L2 norm (Euclidean norm) of the gradient, which is used to measure the length of the gradient vector, and is constrained to be close to 1 to ensure the stability of the discriminator training process.

[0072] B. Attention-guided loss: The attention map generated by the generator is made as consistent as possible with the real defect mask through the cross-entropy loss. The real defect mask is a binary image generated based on the labeled defect position information. The difference between the generated attention map and the real defect mask is calculated through the cross-entropy loss function, which encourages the generator to generate more accurate attention maps. The calculation formula is: ; where represents the attention guidance loss value, which measures the difference between the attention map generated by the generator and the real defect mask. During the training process, the model minimizes this loss value to encourage the generator to generate more accurate attention maps, making the model focus more on the defect area. For example, when the generated attention map is completely consistent with the real defect mask, = 0; the greater the difference, the greater the value of : summation symbol, representing the summation over all pixel positions (i, j) on the attention map. Here, i represents the row index of the pixel, and j represents the column index of the pixel. Assuming the size of the attention map is H x W, the value range of i is 0 to H-1, and the value range of j is 0 to W-1. By calculating and accumulating the results for each pixel position, the loss value of the entire attention map is obtained; M(i, j) is the value of the real defect mask at pixel position (i, j), which is a binary value. In the real defect mask, the pixel value of the defect area is marked as 1, indicating that there is a defect at this position; the pixel value of the background area is marked as 0, indicating that this position is background. For example, in a defect image of a metal plate, the position corresponding to the crack is M(i, j) = 1, while the normal plate area corresponds to M(i, j) = 0; A(i, j) is the value of the attention map generated by the generator at pixel position (i, j), which represents the probability that the generator considers this position to be a defect, with a value range of [0, 1]. The closer the value of A(i, j) is to 1, the greater the probability that the generator considers this position to be a defect; the closer it is to 0, the greater the probability that it is background. For example, A(i, j) = 0.9 indicates that the generator has 90% confidence that this position is a defect area; log: The natural logarithm function is used in this embodiment. In the calculation of cross-entropy loss, the logarithmic function is used to transform the probability value, so that small changes in the probability value can have a greater impact on the loss calculation, thereby enhancing the model's sensitivity to the accuracy of the prediction results. For example, when the predicted probability changes from 0.9 to 0.8, the difference between log(0.9) and log(0.8) will be more obvious than the difference between 0.9 and 0.8 in the loss calculation, allowing the model to pay more attention to changes in the predicted probability; 1 - M(i, j): It has a complementary relationship with M(i, j). When M(i, j) = 1 (i.e., the position is a defect area), 1 - M(i, j) = 0; when M(i, j) = 0 (i.e., the position is a background area), 1 - M(i, j) = 1. It is used to calculate the loss part of the background area, ensuring that the prediction accuracy of the background area is also reflected in the loss; 1 - A(i, j): It has a complementary relationship with A(i, j), representing the probability that the generator considers the position to be background. In the loss calculation, it works with 1 - M(i, j) to measure the accuracy of the generator's prediction of the background area. For example, if the real position is background (M(i, j) = 0) and the generator predicts that the probability of this position being a defect is A(i, j) = 0.2, then 1 - A(i, j) = 0.8, which will be involved in the calculation of the background area loss.

[0073] C. Defect feature constraint loss: as described in step one, including semantic feature L1 loss and physical morphology constraint loss, semantic feature L1 loss is measured by calculating the L1 distance between the generated defect image and the real defect image in the high-order semantic features extracted by the pre-trained defect classification model; the physical morphology constraint loss is designed according to the physical prior knowledge of different defect types, such as the smoothness loss of metal cracks, which calculates the gradient change of crack edge pixels and compares it with the ideal smoothness index to determine the loss value.

[0074] Step three: generating industrial defect images

[0075] (1) input the qualified sample image into the trained generator, and the generator generates defects in the area prone to defects under the guidance of attention, and for metal plate images, generate corresponding defects in the positions prone to defects such as the edge of the plate and the welding position according to the knowledge learned before;

[0076] (2) adjust the input noise Z vector of the generator to control the diversity of defects, the input noise Z vector is a randomly generated vector, by changing the value of the Z vector, the generator can generate defect images of different shapes and sizes, and increase the diversity of defects;

[0077] (3) post-processing of the generated defect image, such as contrast adjustment and random addition of slight background noise, to further improve the authenticity, wherein the contrast adjustment is realized by linear transformation of the pixel value of the image to enhance the contrast between the defect and the background in the image; random addition of slight background noise simulates the noise interference that may occur in the actual shooting process, so that the generated image is closer to the image in the real industrial scene;

[0078]

[0079] The embodiment scheme focuses on the defect area through the space+channel attention module, and combines the physical prior constraint (such as crack smoothness and depression gradient), so that the structural consistency and detail authenticity of the generated image are significantly better than those of the traditional method. The traditional GAN is prone to defect blur and background fragmentation, and the single attention model has weak adaptability to multi-scale defects;

[0080]

[0081] The defect samples generated by the embodiment scheme have stronger semantic consistency (pre-trained model constraint) and physical authenticity (morphological loss), and after being used to expand the data set, the recognition ability of the detection model for rare defects and small-scale defects can be effectively improved, and the risk of overfitting is lower;

[0082] The embodiment scheme adopts Wasserstein distance + gradient penalty adversarial loss, combines He initialization and batch normalization, and has significantly better training stability than traditional GANs; at the same time, the attention guiding mechanism reduces invalid feature learning and accelerates convergence. Noise Z vector adjustment and multi-scale feature fusion also improve the diversity of generated samples.

[0083] The above data reflect the advantages of the AG-GAN scheme in terms of generation quality, detection gain, and training stability. The core lies in the precise focus of the attention guiding mechanism, the authenticity guarantee of multi-dimensional constraints, and the optimized training strategy.

[0084] The embodiment overcomes the defects of existing defect detection schemes that introduce attention mechanisms from multiple aspects such as network architecture design, training strategy optimization, and generated image post-processing.

[0085] Precise focus on defect areas: The spatial attention submodule and the channel attention submodule generate spatial attention maps and channel attention maps, respectively, and multiply them element by element with the feature maps of the corresponding scale of the generator, so that the generator can precisely focus on the defect area and suppress background irrelevant features. Compared with some existing schemes where the attention mechanism does not precisely focus on the defect area, this can more effectively highlight the defects and reduce background interference, thereby improving the accuracy of defect detection. For example, in the generation of metal plate defect images, it can clearly highlight the features of positions prone to defects such as edges and welds, avoiding misjudgment of background features as defects.

[0086] Guarantee the semantic and physical authenticity of generated images: A pre-trained defect classification model is introduced to extract high-order semantic features of generated defect images and real defect images, and the L1 distance is calculated as a constraint loss to ensure semantic consistency of generated defects; physical prior constraints are added for different defect types, such as calculating the edge derivative to measure the smoothness of metal cracks and constructing a loss based on the relationship between the gray value change and the depth gradient for concave defects. This solves the problem of poor semantic consistency and lack of physical prior constraints in existing schemes, making the generated defect images highly similar to real defect images in terms of semantics and physical form, and improving the quality of training data for defect detection models.

[0087] Improve training stability and adversarial effect: During training, the Wasserstein distance is combined with gradient penalty to calculate the adversarial loss, ensuring the Lipschitz continuity of the discriminator and preventing gradient vanishing or gradient explosion during model training; the attention guiding loss is calculated through the cross-entropy loss, making the attention map generated by the generator accurately focus on the real defect area. This training method solves the problems of poor training stability and poor adversarial effect in existing schemes, enabling the generator and discriminator to better collaborate in training and generate images closer to real defects.

[0088] Enhance defect diversity and image authenticity: by adjusting the input noise Z vector of the generator, the diversity of defects can be controlled, different morphological and size defect images can be generated, and the demand of the detection model for diversified defect samples can be met; the generated defect image is subjected to post-processing operations such as contrast adjustment and random addition of slight background noise, and the authenticity of the image is further improved, so that it is closer to the image in the real industrial scene. This makes up for the shortcomings of the existing scheme in terms of defect diversity and image authenticity, and helps to improve the generalization ability of the detection model.

[0089] Embodiment 2

[0090] Embodiment 2 of the present application provides a terminal device corresponding to the above-mentioned embodiment 1, which can be a processing device for a client, such as a mobile phone, a notebook computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0091] The terminal device of the present embodiment includes a memory, a processor and a computer program stored on the memory; the processor executes the computer program on the memory to realize the steps of the method of embodiment 1.

[0092] In some implementations, the memory can be a high-speed random access memory (RAM: Random Access Memory), and can also include a non-volatile memory, such as at least one disk memory.

[0093] In other implementations, the processor can be a central processing unit (CPU), a digital signal processor (DSP) or various types of general-purpose processors, which are not limited here.

[0094] Embodiment 3

[0095] Embodiment 3 of the present application provides a computer readable storage medium corresponding to the above-mentioned embodiment 1, which stores computer programs / instructions. The computer program / instruction is executed by the processor to realize the steps of the method of embodiment 1.

[0096] The computer readable storage medium can be a tangible device that maintains and stores instructions for use by an instruction execution device. The computer readable storage medium can be, for example but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the above.

[0097] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code. Embodiments of the present application can be implemented with various computer program products, such as an object oriented programming language, e.g., Java, and a transitory scripting language, e.g., JavaScript.

[0098] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in one or more of the flow diagrams and / or block diagrams. Figure 1 one or more functions specified in one or more of the flow diagrams and / or block diagrams.

[0099] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in one or more of the flow diagrams and / or block diagrams. Figure 1 one or more functions specified in one or more of the flow diagrams and / or block diagrams.

[0100] While preferred embodiments of the application have been described, modifications and variations can be apparent to those skilled in the art once aware of the general underlying concepts. Therefore, the appended claims as typically interpreted in the art should not be construed as limited to the preferred embodiments but should be interpreted to include all embodiments and variations that fall within the true spirit and scope of the application.

[0101] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An industrial defect image generation method characterized by comprising: The method comprises the following steps: S1, collecting qualified sample images and real defect images of industrial products, and constructing a training set by using the qualified sample images and real defect images; S2, taking the training set as an input of an AG-GAN network model, training the AG-GAN network model, and obtaining an image generation model; The AG-GAN network model comprises: a generator, configured to extract a multi-scale feature map of an input qualified sample image, and generate a defect image under the guidance of an attention guiding module by using the multi-scale feature map; a discriminator, configured to divide a real defect image and a generated defect image into a plurality of sub-regions, and obtain a truth probability of each sub-region; The loss function of the AG-GAN network model comprises a generation loss and a discrimination loss , which are respectively defined as follows: ； ； wherein, is an adversarial loss, is an attention-guided loss, is a semantic feature L1 loss, is a physical form constraint loss, λ1, λ2, λ3 are weight coefficients, is an adversarial loss in the form of Wasserstein distance, is a gradient penalty term, is a weight of the gradient penalty term, D(G(z)) represents a realness probability distribution calculated by the discriminator when inputting the generated defect image G(z), D(R) represents a realness probability distribution calculated by the discriminator when inputting the real defect image R, and E[·] represents an expectation operation on the sample space.

2. The industrial defect image generation method according to claim 1, characterized by, The attention guiding module comprises a spatial attention submodule and a channel attention submodule; wherein: The spatial attention submodule generates a spatial attention map by using the multi-scale feature map; The channel attention submodule performs an all-average pooling operation on the multi-scale feature map to obtain global statistical information of a channel, inputs the global statistical information into a multi-layer perceptron, and obtains a channel attention map.

3. The industrial defect image generation method according to claim 2, characterized in that, Under the guidance of the attention guiding module, the specific implementation process of generating a defect image comprises: The spatial attention map and the channel attention map are respectively multiplied with the corresponding multi-scale feature map to obtain a defect image.

4. The industrial defect image generation method of claim 1, wherein, ; wherein, represents the sum of all pixel positions (i, j) on the spatial attention map, i represents the row index of the pixel, j represents the column index of the pixel, M(i, j) is the value of the defect mask obtained by the real defect pattern annotation at the pixel position (i, j), and A(i, j) is the value of the spatial attention map generated by the generator at the pixel position (i, j).

5. The industrial defect image generation method of claim 1, wherein, ; where ω1, ω2 are weights, , , E is the set of crack edge pixels, ∇I(x, y) is the crack edge pixel gradient value, is the ideal gradient mean value, D is the set of concave region pixels, ∇dG(x, y), ∇dR(x, y) are the depth gradients of the generated concave image and the real concave image, respectively.

6. The industrial defect image generation method of claim 1, wherein, ; G(z) is the generated defect image, and R is the real defect image.

7. The industrial defect image generation method of claim 1, wherein, Gradient penalty term The expression for the gradient penalty term is: ； wherein, is a real defect image and a generated defect image are linearly interpolated between boundary samples, is the expectation, is the gradient of the discriminator output with respect to the input, is the L2 norm of the gradient.

8. A terminal device comprising a memory, a processor, and a computer program stored on the memory; characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-7.

9. A computer readable storage medium having stored thereon computer programs / instructions; characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method of any one of claims 1-7.

Images