Scratch defect data augmentation method and system based on gan and spatial reverse attention mechanism
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV
- Filing Date
- 2024-02-19
- Publication Date
- 2026-08-07
AI Technical Summary
[0004]本发明旨在解决现有技术中存在的技术问题，提供了一种基于GAN及空间反向注意力机制的划痕数据扩充方法及系统，以解决现有深度学习方法无法很好生成细小划痕(问题和在较暗背景下无法生成划痕的问题
[0031]提取出的编码器和生成器结构将划痕语义图、具有或没有划痕的真实图像生成对应真实图像风格的真实划痕图像，对真实划痕图像进行数据扩充。
Smart Images
Figure CN117934447B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of defect image generation technology, specifically relating to a method and system for augmenting scratch defect data based on GAN and spatial inverse attention mechanism. Background Technology
[0002] Defect detection is a crucial part of product manufacturing and maintenance. Scratches are a special type of surface defect, making their detection particularly important. However, real-world scratch defects are relatively rare in industrial settings, and the limited scratch data available is insufficient for training supervised deep learning models for scratch defect detection. Therefore, it is necessary to augment the existing scratch data.
[0003] Data augmentation can enhance known scratch samples by rotating, translating, flipping, adjusting contrast, and adding noise. However, this method is limited to the feature space of known scratch data and cannot fully represent the features of samples with complex textures. In recent years, CNNs (Convolutional Neural Networks) have seen numerous applications in data augmentation due to their excellent image generation capabilities, but they have lacked a good solution for generating images with complex textures. The advent of GANs (Generative Adversarial Networks) addressed this issue. GANs no longer attempt to manually design evaluation criteria to quantify the realism of images; instead, they delegate the evaluation task to CNNs. Through training, the discrimination ability of the evaluation network is gradually improved, thereby simultaneously enhancing the generation ability of the generative network. However, existing GAN-based deep learning methods have consistently fallen short in generating scratch defect images, failing to generate images with small scratches or scratches against dark backgrounds. Summary of the Invention
[0004] This invention aims to solve the technical problems existing in the prior art and provides a scratch data augmentation method and system based on GAN and spatial reverse attention mechanism to solve the problems that existing deep learning methods cannot generate fine scratches well and cannot generate scratches in dark backgrounds.
[0005] To achieve the above-mentioned objectives of the present invention, according to a first aspect of the present invention, the present invention provides a scratch data augmentation method based on GAN and spatial reverse attention mechanism, which includes the following steps:
[0006] Obtain a sample set of real scratch images, preprocess the sample set to obtain a training set and a test set;
[0007] A GAN-based data augmentation network is constructed and trained. The data augmentation network includes an encoder, a generator, and a discriminator. The training process is as follows: the encoder learns from real scratch image samples to generate the style of real scratch images; the generator learns from the semantic map of scratches and the style of real scratch images generated by the encoder to generate simulated image samples; the discriminator uses a prediction module with multiple feature layers to distinguish the authenticity of simulated image samples and real scratch image samples at different scales; simultaneously, the generated image and label alignment module performs dot product operations with the labels on the generated image and the real scratch image, and then sends them to the discriminator to determine authenticity. When the discriminator cannot distinguish authenticity, the encoder and generator structures are extracted based on the characteristics of the data augmentation network at this time.
[0008] The encoder and generator structures are extracted by inputting a scratch semantic map and real images with or without scratches, and then generate real scratch images with the corresponding real image style, and augment the real scratch images with data.
[0009] The data augmentation method in this invention learns the overall distribution of real images through an encoder, generates simulated image samples through a generator, and incorporates a generated image and label alignment module into the discriminator. This effectively solves the problems of existing deep learning methods being unable to generate fine scratches well and being unable to generate scratches against a dark background, thereby achieving the generation of realistic scratch images.
[0010] According to a preferred embodiment of the present invention, the encoder processing method is as follows:
[0011] Adjust the size of the input real scratch image;
[0012] After multiple sequentially stacked feature extraction modules, each feature extraction module includes a convolutional layer as well as two-dimensional channel normalization function and activation function processing;
[0013] The features processed by the feature extraction module are flattened using a flattening operation.
[0014] Then, the vectors are fed into two fully connected layers to obtain two vectors, which serve as the mean μ and variance σ, respectively. 2 A Gaussian distribution is obtained.
[0015] The encoder processing method of this invention enables the model to learn richer and more abstract feature representations, thereby improving the model's performance and generalization ability.
[0016] According to another preferred embodiment of the present invention, the processing method of the generator method is as follows:
[0017] The scratch semantic map and Gaussian distribution are fed into the generator network, and the input Gaussian distribution is flattened through a linear layer.
[0018] The Gaussian distribution is transformed into a tensor by reshaping the layer;
[0019] The scratch semantic map and a random vector are concatenated along the channel dimension. The size of the random vector is the same as the feature size of the scratch semantic map. The concatenated image and the Gaussian distribution after the reshaping layer are fed into the SRA (Spatial Reduction Attention) module and the upsampling module array. The SRA module and the upsampling module are stacked in sequence. Each stacked SRA module receives the scratch semantic map and the upsampling result image of the previous layer as input.
[0020] It accepts a random vector as input, which ensures that even if the input is the same scratch semantic map, the generated scratch real image can still be random and diverse.
[0021] According to another preferred embodiment of the present invention, the SRA module adjusts the size of the scratch semantic map to be the same as the size of the input features of the SRA module, and then performs convolution, dot product and addition operations on the semantic map and the input features within the SRA module.
[0022] Each SRA module contains a spatial inverse attention module, which adjusts feature weights to make the generator focus more on scratch generation.
[0023] According to another preferred embodiment of the present invention, the processing method of the discriminator is as follows:
[0024] The generated image and label are then concatenated and fed into the discriminator network;
[0025] The generated image and the real scratch image are fed into the discriminator network after the dot product operation is performed between the generated image and the label and the label, respectively, using the generated image and label alignment module.
[0026] The prediction module, utilizing the intermediate and final feature layers of the discriminator, distinguishes between the authenticity of simulated image samples and real scratch image samples at different scales.
[0027] Taking scratch processing as an example, the discriminator not only evaluates the generated image and the real image from different feature scales on the whole image (scratches + background), but also focuses on evaluating the details of scratch generation from different feature scales.
[0028] To achieve the above-mentioned objectives of the present invention, according to a first aspect of the present invention, the present invention provides a scratch data augmentation system based on GAN and spatial reverse attention mechanism, comprising:
[0029] The preprocessing module preprocesses the sample set to obtain a training set and a test set, wherein the sample set is composed of real scratch images.
[0030] A data augmentation network, comprising an encoder, a generator, and a discriminator, is used. Real scratch image samples are learned in the encoder to generate the style of real scratch images. The semantic map of the scratch and the style of the real scratch images generated by the encoder are learned in the generator to generate simulated image samples. The prediction modules of the intermediate and final feature layers of the discriminator distinguish the authenticity of simulated image samples and real scratch image samples at different scales. At the same time, the generated image and the real scratch image are fed into the discriminator after performing dot product operations with the labels using the generated image and label alignment modules to determine authenticity. When the discriminator cannot distinguish authenticity, the encoder and generator structures are extracted based on the characteristics of the data augmentation network at this time.
[0031] The extracted encoder and generator structures generate real scratch images in the style of real images from scratch semantic maps and real images with or without scratches, and augment the real scratch images with data.
[0032] This invention learns the overall distribution of real images through an encoder, generates simulated image samples in a generator, and incorporates a generated image and label alignment module into the discriminator. This effectively solves the problems of existing deep learning methods being unable to generate fine scratches well and being unable to generate scratches in dark backgrounds, thereby achieving the generation of realistic scratch images.
[0033] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0034] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:
[0035] Figure 1 This is a flowchart of a preferred embodiment of the scratch defect data augmentation method based on GAN and spatial reverse attention mechanism of the present invention.
[0036] Figure 2 This is a schematic diagram of the overall structure of the data augmentation network in a preferred embodiment of the present invention;
[0037] Figure 3 This is a schematic diagram of a data augmentation network encoder in a preferred embodiment of the present invention;
[0038] Figure 4 This is a schematic diagram of a data augmentation network generator in a preferred embodiment of the present invention;
[0039] Figure 5 This is a schematic diagram of a spatial reverse attention module in a preferred embodiment of the present invention;
[0040] Figure 6 This is a schematic diagram of a data augmentation network discriminator in a preferred embodiment of the present invention;
[0041] Figure 7 This is a schematic diagram of a data augmentation comparison experiment, in which... Figure 7 (a) is a tag. Figure 7 (b) is a real image. Figure 7 (c) is the SPADE expansion effect. Figure 7 (d) is the Pi x2Pi xHD expansion effect. Figure 7 (e) represents the extension effect of the method presented in this paper. Detailed Implementation
[0042] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0043] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "vertical", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0044] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.
[0045] This invention provides a method for augmenting scratch defect data based on GAN and spatial inverse attention mechanism, such as... Figure 1 As shown, it includes the following steps:
[0046] A sample set of real scratch images is obtained, and the sample set is preprocessed to obtain a training set and a test set.
[0047] This invention can be applied to the generation of data related to scratches or other defects. By providing the network with a semantic map and a ground truth map, it can generate a corresponding real image of the semantic map. Furthermore, this invention is particularly suitable for generating scratch images. While GANs tend to focus on generating complex background textures for realistic scratch defects, combining this invention with a spatial inverse attention mechanism allows GANs to pay more attention to generating scratch details.
[0048] In this embodiment, preprocessing mainly involves cropping the images to the same size. In a more preferred embodiment, the scratch image samples come from a factory, totaling 20 images acquired by an industrial camera using a ring light, with a resolution of 5472*3648. Since the image resolution is too high to be directly used for training the convolutional neural network model, each image is cropped to 512*512 pixels, and images containing surface scratches are selected as training and testing data. A total of 80 images are used; 70 are randomly selected for training, and 10 for testing.
[0049] like Figure 2 As shown, a GAN-based data augmentation network is constructed and trained. The data augmentation network includes an encoder, a generator, and a discriminator. The training process is as follows: the encoder learns from real scratch image samples to generate the style of real scratch images; the generator learns from the scratch semantic map and the style of the real scratch images generated by the encoder to generate simulated image samples; the discriminator uses a prediction module with multiple feature layers to distinguish the authenticity of simulated image samples and real scratch image samples at different scales; simultaneously, the generated image and label alignment module performs dot product operations with the labels on the generated image and the real scratch image, and then sends them to the discriminator to determine authenticity. When the discriminator cannot distinguish authenticity, the encoder and generator structures are extracted based on the characteristics of the data augmentation network at this time.
[0050] In this embodiment, such as Figure 3 As shown, the encoder includes:
[0051] The resize layer adjusts the size of the input real scratch image, for example, to 256*256.
[0052] Multiple feature extraction modules are stacked sequentially. Each feature extraction module includes a convolutional layer and a two-dimensional channel normalization function and activation function processing. For example, after processing, the multiple stacked feature extraction modules become [1,512,4,4].
[0053] The reshape layer flattens the features output by the feature extraction module through a flattening operation, for example, flattening them to a size of [8192,1,1].
[0054] Two fully connected layers process the features output by the reshaping layer, resulting in two vectors (e.g., of size [1, 256]), which serve as the mean μ and variance σ, respectively. 2 We obtain a Gaussian distribution (e.g., of size [1,256]).
[0055] In a preferred embodiment of the present invention, the encoder processing method is as follows:
[0056] Resize the input real scratch image to 256*256;
[0057] After multiple stacked feature extraction modules, each feature extraction module includes a convolutional layer and two-dimensional channel normalization and activation functions. After processing by multiple stacked feature extraction modules, the result is a size of [1,512,4,4].
[0058] After flattening, the resulting size is [8192,1,1].
[0059] Then, the vectors are fed into two fully connected layers to obtain two vectors of [1, 256], which are used as the mean μ and variance σ, respectively. 2 We obtain a Gaussian distribution with a size of [1, 256].
[0060] In this embodiment, scratch semantic maps are generated using OpenCV, the PIL library, and the matplotlib library to simulate realistic scratch-annotated images. After preprocessing the generated scratch semantic maps, input variables are obtained. These input variables are then fed into an encoder and generator to obtain the generated result. The generated result and the scratch semantic maps fed into the network form complete training data, thus realizing the generation of scratch defect images.
[0061] In this embodiment, the environment and implementation details for training the data augmentation network are shown in Table 1 and Table 2.
[0062] Table 1. Training Environment for Data Augmentation Networks
[0063] project parameter operating system Windows 10 Professional Edition 1019043.1586 processor Intel(R)Core(TM)i7-8700CPU@3.20GHz Memory 64G graphics card NVIDIA GeForce GTX 1080Ti*2 Image accelerator CUDA 10.0 + cuDNN 7.6 Deep learning framework PyTorch 1.7.1
[0064] Table 2. Implementation details of data augmentation networks
[0065] project parameter Input dimensions 512*512 loss function Cross-entropy Optimizer Adam(betas=(0.0,0.9)) Number of training rounds 300 Learning rate 0.0002 Batch size 6
[0066] like Figure 4 As shown, the generator includes:
[0067] Linear layer, which flattens the input Gaussian distribution;
[0068] The reshaping layer converts the Gaussian distribution output by the linear layer into a tensor, for example, with a size of [1, 1024, 16, 16].
[0069] The SRA module and upsampling module array concatenate the scratch semantic map and a random vector along the channel dimension. The size of the random vector is the same as the feature size of the semantic map. The concatenated image and the Gaussian distribution after the reshaping layer are fed into the SRA module and upsampling module array. The SRA module and upsampling module are stacked in sequence. Each stacked SRA module receives the scratch semantic map and the upsampling result image of the previous layer as input, and the resulting image size is, for example, [1,3,512,512].
[0070] A 3×3 convolutional layer and a ReLU activation function layer are used to resize the scratch semantic map to match the upsampling result of the SRA module. Figure 1 Sample size.
[0071] like Figure 5 As shown, the SRA module includes:
[0072] The first convolutional layer performs a convolution operation on the semantic graph;
[0073] The first dot product unit is used to perform dot product between the features output by the first convolutional layer and the input features of the SRA module;
[0074] The second convolutional layer performs a convolution operation on the semantic graph;
[0075] The addition unit is used to add the features output by the second convolutional layer and the output of the first dot product unit.
[0076] The third convolutional layer is used to perform convolution operations on the output of the addition unit;
[0077] The maximum tensor layer (Torch.max layer) and the average tensor layer (Torch.mean layer) are used to calculate the maximum return tensor and the average return tensor of the output features of the third convolutional layer;
[0078] The concat layer connects the maximum return tensor and the average return tensor.
[0079] The fourth convolutional layer and sigmoid activation function are used to convolve the output of the connected layer and map it to the [0,1] interval, thereby converting the features into probability values;
[0080] The inversion layer is used to invert the result after processing by the fourth convolutional layer and the sigmoid activation function; inversion makes the generator network pay more attention to the generation of scratch details.
[0081] The second dot product unit is used to multiply the output image of the inverted layer with the output image of the summation unit. After passing through the second dot product unit, the features with adjusted weights are obtained, making the network pay more attention to the generation of scratch details.
[0082] The reverse attention module extends from the third convolutional layer to the second dot product unit. This module includes the third convolutional layer, the maximum tensor layer, the average tensor layer, the connection layer, the fourth convolutional layer, the activation function, inversion, and the final second dot product unit. The features initially input at the third convolutional layer undergo a series of operations to obtain weights. Multiplying the features by these weights yields the weighted features.
[0083] This invention feeds the scratch semantic map and Gaussian distribution into the generator network, and flattens the input Gaussian distribution through a linear layer;
[0084] The Gaussian distribution is transformed into a tensor with a size of [1, 1024, 16, 16] through a reshaping layer. The scratch semantic map and a random vector are concatenated along the channel dimension. The size of the random vector is the same as the feature size of the scratch semantic map. The concatenated image and the Gaussian distribution after the reshaping layer are fed into an array of SRA and upsampling modules to obtain an image with a size of [1, 3, 512, 512]. The SRA and upsampling modules are stacked sequentially. Each stacked SRA module receives the scratch semantic map and the upsampling result image from the previous layer as input. The SRA module resizes the scratch semantic map to the same size as the input features of the SRA module. Then, within the SRA module, convolution, dot product, and sum operations are performed on the semantic map and the input features. Each SRA module contains a spatial inverse attention module, which adjusts the feature weights (calculates the maximum and average values of the input features in the channel dimension, concatenates the maximum and average values, uses convolution to adjust the number of channels, and then uses the sigmoid function to obtain the feature weights. In order to make the generator network pay more attention to the generation of scratch details, the weights are inverted and multiplied with the input features to obtain the output), making the generator pay more attention to the generation of scratches.
[0085] like Figure 6 As shown, the discriminator includes:
[0086] The first connection layer is used to concatenate the generated image and the label and then feed them into the discriminator network;
[0087] The generated image and label alignment module is used to perform dot product operations between the generated image and the real scratch image and the label respectively, and then feed the concatenated image into the discriminator network.
[0088] The discriminator network has a prediction module in each layer (specifically, both the generated image and the real image are fed into the discriminator simultaneously; feature maps corresponding to the generated and real images exist in different feature layers, and the loss of the two feature maps is calculated in each layer). The prediction module distinguishes the authenticity of simulated image samples and real scratch image samples at different scales. In the discriminator, the feature maps are downsampled in each layer. Therefore, in each layer, the discriminator judges the authenticity of the image at five different resolutions: 512*512, 256*256, 128*128, 64*64, and 32*32, thereby realizing the distinction between simulated image samples and real scratch image samples at different scales.
[0089] Finally, the encoder and generator structures extracted from the scratch semantic map and real images with or without scratches are input to generate real scratch images with the corresponding real image styles, and the real scratch images are augmented with data.
[0090] To demonstrate the superiority of the proposed method, some comparative experiments were conducted. In addition to the mainstream Pix2PixHD method, the more advanced SPADE method was also selected for comparison. The results are as follows: Figure 7 As shown. From Figure 7 It can be seen that the generated images from Pix2PixHD and SPADE have unnatural textures, with a large number of repeated textures, severe meshing, and uneven lighting, failing to meet realistic lighting conditions. Furthermore, they are essentially unable to generate small scratches or scratches in dark backgrounds. This embodiment does not exhibit these problems. The method used in this embodiment produces natural image texture transitions, moving from light to dark along the lighting direction. This demonstrates that the network has learned the data distribution of a real image, rather than averaging and repeating pixel values within each semantic object. Moreover, it generates very natural and complete scratches, even those in dark backgrounds, indicating the significant advancement of this invention in focusing on scratch detail generation. The quantitative results are shown in Table 3. The data shows that the technical solution of this invention still demonstrates superiority. Although it is inferior to SPADE in LP IPS metrics, the technical solution of this invention is significantly better than SPADE in terms of generated image results. In summary, the experimental results prove that the technical solution of this invention possesses certain advantages.
[0091] Table 3. Comparison of Quantitative Results
[0092]
[0093] This invention also provides a scratch data augmentation system based on GAN and spatial inverse attention mechanism, comprising a preprocessing module, wherein the preprocessing module preprocesses a sample set to obtain a training set and a test set, the sample set being a sample set composed of real scratch images; a data augmentation network, wherein the data augmentation network includes an encoder, a generator, and a discriminator, real scratch image samples are learned in the encoder to generate the style of real scratch images; the semantic map of scratches and the style of real scratch images generated by the encoder are learned in the generator to generate simulated image samples; the discriminator... The prediction modules of the intermediate and final feature layers distinguish the authenticity of simulated image samples and real scratch image samples at different scales. At the same time, the generated image and label alignment modules perform dot product operations on the generated image and real scratch image with the label image respectively and then send them to the discriminator to determine authenticity. When the discriminator cannot distinguish authenticity, the encoder and generator structures are extracted according to the characteristics of the data augmentation network at this time. The extracted encoder and generator structures generate real scratch images with the corresponding real image style from the scratch semantic map and real images with or without scratches, and perform data augmentation on the real scratch images.
[0094] This invention learns the overall distribution of real images through an encoder, generates simulated image samples in a generator, and incorporates a generated image and label alignment module into the discriminator. This effectively solves the problems of existing deep learning methods being unable to generate fine scratches well and being unable to generate scratches in dark backgrounds, thereby achieving the generation of realistic scratch images.
[0095] In the description of this specification, references to terms such as "preferred embodiment," "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0096] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims
1. A method for augmenting scratch defect data based on GAN and spatial reverse attention mechanism, characterized in that, Includes the following steps: Obtain a sample set of real scratch images, preprocess the sample set to obtain a training set and a test set; A GAN-based data augmentation network is constructed and trained. The data augmentation network includes an encoder, a generator, and a discriminator. The training process is as follows: the encoder learns from real scratch image samples to generate the style of real scratch images; the generator learns from the scratch semantic map and the style of the real scratch images generated by the encoder to generate simulated image samples; the discriminator uses a prediction module with multiple feature layers to distinguish the authenticity of simulated image samples and real scratch image samples at different scales; simultaneously, the generated image and the real scratch image are dot-producted with the labels using a generated image and label alignment module and then fed into the discriminator to determine authenticity; when the discriminator cannot distinguish authenticity, the encoder and generator structures are extracted based on the characteristics of the data augmentation network at this time. The encoder and generator structures are extracted by inputting a scratch semantic map and real images with or without scratches to generate real scratch images with the style of the real images, and the real scratch defect images are augmented with data. The encoder's processing method is as follows: Adjust the size of the input real scratch image; After multiple sequentially stacked feature extraction modules, each feature extraction module includes a convolutional layer as well as two-dimensional channel normalization function and activation function processing; The features processed by the feature extraction module are flattened using a flattening operation. Then, the vectors are fed into two fully connected layers to obtain two vectors, which serve as the mean μ and variance σ, respectively. 2 A Gaussian distribution is obtained; The processing method of the generator method is as follows: The scratch semantic map and Gaussian distribution are fed into the generator network, and the input Gaussian distribution is flattened through a linear layer. The Gaussian distribution is transformed into a tensor by reshaping the layer; The scratch semantic map and a random vector are concatenated along the channel dimension. The size of the random vector is the same as the feature size of the scratch semantic map. The concatenated image and the Gaussian distribution after the reshaping layer are sent to the SRA module and the upsampling module array. The SRA module and the upsampling module are stacked in sequence. Each stacked SRA module receives the scratch semantic map and the upsampling result image of the previous layer as input. The SRA module resizes the scratch semantic map to the same size as the input features of the SRA module, and then performs convolution, dot product, and addition operations on the semantic map and the input features within the SRA module. Each SRA module contains a spatial inverse attention module, which adjusts feature weights to make the generator focus more on scratch generation.
2. The scratch defect data augmentation method based on GAN and spatial reverse attention mechanism according to claim 1, characterized in that, The processing method of the discriminator is as follows: The generated image and label are then concatenated and fed into the discriminator network; The generated image and the real scratch image are fed into the discriminator network after the dot product operation is performed between the generated image and the label image, respectively, using the generated image and label alignment module. The prediction module, utilizing the intermediate and final feature layers of the discriminator, distinguishes between the authenticity of simulated image samples and real scratch image samples at different scales.
3. A scratch defect data augmentation system based on GAN and spatial reverse attention mechanism according to the method of claim 1 or 2, characterized in that, include: The preprocessing module preprocesses the sample set to obtain a training set and a test set, wherein the sample set is composed of real scratch images. A data augmentation network, comprising an encoder, a generator, and a discriminator, is used. Real scratch image samples are learned in the encoder to generate the style of real scratch images. The semantic map of the scratch and the style of the real scratch images generated by the encoder are learned in the generator to generate simulated image samples. The prediction module of each layer of the discriminator distinguishes the authenticity of simulated image samples and real scratch image samples at different scales. At the same time, the generated image and the real scratch image are fed into the discriminator after performing a dot product operation with the label using the generated image and label alignment module to determine authenticity. When the discriminator cannot distinguish authenticity, the encoder and generator structures are extracted based on the characteristics of the data augmentation network at this time. The extracted encoder and generator structures generate real scratch images in the style of real images from scratch semantic maps and real images with or without scratches, and augment the real scratch images with data.
4. The scratch defect data augmentation system based on GAN and spatial reverse attention mechanism according to claim 3, characterized in that, The encoder includes: A resizing layer that adjusts the size of the input real scratch image; Multiple feature extraction modules are stacked sequentially, and each feature extraction module includes a convolutional layer as well as a two-dimensional channel normalization function and activation function processing; The reshaping layer flattens the features output by the feature extraction module through a flattening operation. Two fully connected layers process the features output by the reshaping layer, respectively, to obtain two vectors, which serve as the mean μ and variance σ. 2 A Gaussian distribution is obtained.
5. The scratch defect data augmentation system based on GAN and spatial reverse attention mechanism according to claim 3, characterized in that, The generator includes: A linear layer that flattens the input Gaussian distribution; The reshaping layer converts the Gaussian distribution output by the linear layer into a tensor. The SRA module and upsampling module array concatenate the scratch semantic map and a random vector along the channel dimension. The size of the random vector is the same as the feature size of the scratch semantic map. The concatenated image and the Gaussian distribution after the reshaping layer are fed into the SRA module and upsampling module array. The SRA module and upsampling module are stacked in sequence. Each stacked SRA module receives the scratch semantic map and the upsampling result image of the previous layer as input. A 3×3 convolutional layer and a ReLU activation function layer are used to resize the scratch semantic map to the same size as the upsampled result map of the SRA module.
6. The scratch defect data augmentation system based on GAN and spatial reverse attention mechanism according to claim 3, characterized in that, The SRA module includes: The first convolutional layer performs a convolution operation on the semantic graph; The first dot product unit is used to perform dot product between the features output by the first convolutional layer and the input features; The second convolutional layer performs a convolution operation on the semantic graph; The addition unit is used to add the features output by the second convolutional layer and the output of the first dot product unit. The third convolutional layer is used to perform convolution operations on the output of the addition unit; The maximum tensor layer and the average tensor layer are used to calculate the maximum return tensor and the average return tensor of the output features of the third convolutional layer. A connection layer that concatenates the maximum return tensor and the average return tensor; The fourth convolutional layer and sigmoid activation function are used to convolve the output of the connected layer and map it to the [0, 1] interval, thereby converting the features into probability values; The inverting layer is used to invert the result after processing by the fourth convolutional layer and the sigmoid activation function. The second dot product unit is used to perform a dot product between the output of the inverted layer and the output of the summation unit.
7. The scratch defect data augmentation system based on GAN and spatial reverse attention mechanism according to claim 3, characterized in that, The discriminator includes: The first connection layer is used to concatenate the generated image and the label and then feed them into the discriminator network; The generated image and label alignment module is used to perform dot product operations on the generated image, the real scratch image and the label image respectively, and then feed the concatenated image into the discriminator network. The discriminator network has prediction modules in its intermediate and final feature layers. These prediction modules distinguish the authenticity of simulated image samples from real scratch image samples at different scales.