An industrial defect image generation method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING FOCUSIGHT TECH
- Filing Date
- 2022-11-01
- Publication Date
- 2026-05-29
Smart Images
Figure CN115661578B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image visual processing technology, and in particular to a method for generating industrial defect images. Background Technology
[0002] Deep learning models have matured significantly in mainstream computer vision, but their performance is less than ideal in industrial defect detection scenarios. Training a deep learning model often requires a large amount of rich data. In industrial defect detection, the number of defect images is limited, their structures are often simple, and data imbalance hinders the improvement of deep learning model performance. Traditional data augmentation methods, such as rotating, translating, and adding noise, can effectively increase the number of defect samples and enrich their appearance. However, these methods only increase the amount of data to a certain extent; the augmented defect images remain similar to the original ones, offering limited help in improving the performance of deep learning models.
[0003] In recent years, adversarial training, especially generative adversarial models (GANs), has played an increasingly important role in anomaly detection for imbalanced images. GANs train a generator using raw data, which produces new samples. The generator data and real training data are then fed into a discriminator, which is trained to correctly distinguish data types. The discriminator and generator compete against each other; the generator learns to produce new samples that more closely approximate real data to deceive the discriminator, while the discriminator also needs to better distinguish between generated and real data.
[0004] CGAN is an improvement on GAN, adding additional conditional information y to the generator (G) and discriminator (D) of the original GAN to achieve a conditional generative model. The additional conditional information in CGAN can be class labels or other auxiliary information. Therefore, CGAN allows GANs to be trained using images and their corresponding labels, and to generate specific images using given labels during the testing phase.
[0005] The core operation of CGAN is to add conditional information to G and D (which is actually to add class label y to both G and D). (1) The input of the original GAN generator is a noise signal. In CGAN, the class label can be combined with the noise signal as a latent space representation. (2) The input of the original GAN discriminator is image data (real image and generated image). In CGAN, the class label and image data also need to be concatenated as the discriminator input.
[0006] In CGAN, the generator takes noise z and condition y as input and outputs generated samples G(z|y). The discriminator takes training samples x and condition y as input and outputs the probability D(x|y) that a training sample belongs to a real sample.
[0007] However, the defect images generated by the CGAN model show blurry backgrounds and defects. This is mainly because the generator network in CGAN has a shallow structure and lacks sufficient expressive power to generate realistic images, resulting in blurry defect images. Furthermore, the CGAN model cannot control the size and location of generated defects. The CGAN model generator does not incorporate the location and size information of defects when generating defect images; it only incorporates defect category information. This means the CGAN model generator can only generate different categories of defects based on the category information, but cannot control the size and location of the generated defects. In industrial defect images, some defects appear within a relatively fixed range and do not appear arbitrarily in the image. Therefore, it is necessary to control the generated defects to appear in a reasonable location. While the defect locations generated by the CGAN model generator are relatively fixed, this fixed location may not be where the defect should appear. In addition, defects in industrial defect images have different sizes, and directly using the CGAN model generator results in relatively fixed defect sizes, making it impossible to generate defects of different sizes. Summary of the Invention
[0008] The technical problem to be solved by the present invention is to provide an industrial defect image generation method, which solves the problem of generating relatively realistic defect samples when there are only a few defect samples, and rapidly expands the quantity and richness of defect samples.
[0009] The technical solution adopted by this invention to solve its technical problem is: an industrial defect image generation method, comprising the following steps,
[0010] S1. Collect and create the FSGAN model dataset;
[0011] S2. Perform data augmentation and data normalization preprocessing on the data;
[0012] S3. Construct the FSGAN model generator structure;
[0013] S4. Set the hyperparameters of the model and train the model;
[0014] S5. Analyze the results of the experiment.
[0015] Furthermore, in step S1 of the present invention, different positive sample images are collected, and a mask image is provided for each positive sample image. Each mask image is marked with the location and size information of the expected defect.
[0016] Furthermore, in step S2 of this invention, the data augmentation method includes random rotation, random cropping, and Gaussian blur; the data normalization method is: X' i =X i / 255.0, where X i It is the input feature, X' i It is the output feature.
[0017] Furthermore, in step S3 of this invention, the FSGAN model generator consists of 13 convolutional layers, 2 deconvolutional layers, 11 normalization layers, and 12 activation function layers. After the input image is fed into the model, it passes through 3 convolutional-deconvolutional modules, 3 residual modules, 2 upsampling modules, and 4 convolutional-deconvolutional modules before outputting. Each convolutional-deconvolutional module consists of one convolutional layer, one normalization layer, and one activation function layer. The convolutional kernel size in the first convolutional-deconvolutional module is 7×7, and the stride is 1, used to extract features from the input feature map. The convolutional kernel size in the other two convolutional-deconvolutional modules is 3×3, and the stride is 2, used for downsampling to increase the receptive field of the model. The residual module consists of one convolutional-deconvolutional module and one convolutional layer. The first convolutional layer consists of a normalization layer and a convolutional kernel. All kernels have a size of 3×3 and a stride of 1. This layer extracts features from the input feature map to obtain high-level semantic features. The second convolutional layer consists of a deconvolutional layer, a normalization layer, and an activation function layer. The deconvolutional layer has a kernel size of 4×4 and a stride of 2. This layer upsamples the input feature map. The final four convolutional layers each consist of a convolutional layer, a normalization layer, and an activation function layer. Each kernel has a size of 1×1, 3×3, 5×5, and 7×7, and a stride of 1. These layers extract features from the input feature map, making the defective images generated by the model clearer.
[0018] Furthermore, in step S4 of this invention, the model is trained using pre-trained model weights; the SGD gradient update strategy is adopted, and the learning rate update rule uses a fixed learning rate.
[0019] Furthermore, in step S5 of this invention, if the generated defect is similar to the real defect, the training ends; if the generated defect differs significantly from the real defect, the hyperparameters of the model are adjusted and training continues.
[0020] The beneficial effect of this invention is that it solves the defects existing in the prior art.
[0021] The structure of the CGAN model was reconstructed by removing the fully connected structure and adding convolutional excitation, residual, and upsampling modules to form the FSGAN model. The FSGAN model has a deeper number of network layers, a larger model capacity, and generates clearer defect images. The FSGAN model takes positive sample images, mask images, noisy images, and class images as input. It directly generates defects at specified locations, transforming a positive sample image into an image with defects. The generated defects have different locations, sizes, categories, and appearances, which quickly expands the quantity and richness of the dataset. It eliminates the need to generate backgrounds, reduces the difficulty of generating defect images, and reduces the time required for defect sample collection. Attached Figure Description
[0022] Figure 1 This is a flowchart of the defect image generation process of the present invention;
[0023] Figure 2 This is a structural diagram of the FSGAN model generator of this invention;
[0024] Figure 3 This is a schematic diagram of the category location input image for the FSGAN model of this invention. Detailed Implementation
[0025] The present invention will now be described in further detail with reference to the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0026] like Figures 1-3 The method for generating industrial defect images, as shown, includes the following steps:
[0027] S1. Collect and create the FSGAN model dataset. Collect different positive sample images and provide a mask image for each positive sample image. Each mask image is labeled with the location and size information of the expected defect. Provide a category image for each positive sample image, which records the category of the defect. Provide a noise image for each positive sample image, which contains randomly generated Gaussian noise. Combine the positive sample images, mask images, noise images, and category images to generate the category location input image for the FSGAN model. The generated category location input image needs to undergo data preprocessing before being fed into the generator. Data preprocessing includes data augmentation and data normalization. The generated category location input image also needs to undergo data augmentation to further expand the number and richness of input images and improve the ability of the FSGAN model to generate defect images. Since the data distribution of input images at different category locations is inconsistent, normalization is also required for the input images.
[0028] S2. Data preprocessing includes data augmentation and data normalization. Training the FSGAN model requires a large number of training samples; therefore, data augmentation methods are used to expand the FSGAN model dataset. The data augmentation methods used include random rotation, random cropping, and Gaussian blurring. Defects in industrial images generally have different tilt angles; during data augmentation, rotation is used, with the tilt angle ranging from -10 to 10 degrees. Defects in industrial images have different sizes; a region of interest is randomly cropped from the original image and then scaled to a fixed size, thus scaling the defects in the image. Defects in industrial images have different levels of sharpness; Gaussian blurring can blur the defects in the dataset to different degrees, which is beneficial for the FSGAN model's ability to generate defects with varying degrees of blur.
[0029] Data preprocessing refers to normalizing the range of values in the input image to [0,1]. The input image data is normalized according to the following formula:
[0030] X' i =X i / 255.0
[0031] Among them, X i It is the input feature, X' i It is the output feature.
[0032] Without data normalization, the distribution of training data in each batch may differ. Because the input data for each layer of the network is constantly changing, it becomes difficult to find a suitable balance point, ultimately making it difficult for the constructed neural network model to converge.
[0033] S3. Construct the FSGAN model generator structure. See [link to FSGAN model generator structure] for details. Figure 2The entire FSGAN model generator consists of 13 convolutional layers, 2 deconvolutional layers, 11 normalization layers, and 12 activation function layers. After the input image is fed into the model, it passes through 3 convolutional-to-activation modules, 3 residual modules, 2 upsampling modules, and 4 convolutional-to-activation modules before outputting. Each convolutional-to-activation module consists of one convolutional layer, one normalization layer, and one activation function layer. The convolutional layer in the first convolutional-to-activation module has a 7×7 kernel size and a stride of 1, primarily used to extract features from the input feature map. The convolutional layers in the following two convolutional-to-activation modules have a 3×3 kernel size and a stride of 2, primarily used for downsampling to increase the model's receptive field. The residual module consists of one convolutional-to-activation module, one convolutional layer, and one normalization layer, all with a 3×3 kernel size and a stride of 1, primarily used to extract features from the input feature map to obtain high-level semantic features. The upsampling module consists of a deconvolution layer, a normalization layer, and an activation function layer. The deconvolution layer has a 4×4 kernel size and a stride of 2, primarily used to upsample the input feature map. The final four convolutional-normalization modules each consist of a convolutional layer, a normalization layer, and an activation function layer. The kernel sizes are 1×1, 3×3, 5×5, and 7×7, respectively, with a stride of 1. These modules are mainly used to extract features from the input feature map, making the defective images generated by the model clearer. In Figure 2, convolutional-normalization ×3 represents three consecutive convolutional-normalization modules, residual ×3 represents three consecutive residual modules, upsampling ×2 represents two consecutive upsampling modules, and concatenation represents combining features from two different channels.
[0034] The FSGAN model generator uses a different input than the CGAN model generator. The FSGAN model generator incorporates a mask image into its input, which is labeled with the size and location of the defects expected to be generated by the model. The FSGAN model can then generate defect images based on the size and location information in the mask image.
[0035] The FSGAN model generator has deeper network layers, greater model capacity, and generates clearer defect images.
[0036] The CGAN model cannot control the size and location of generated defects. When generating defect images, the CGAN model generator only incorporates defect category information, not location and size information. This means the CGAN model generator can only generate defects of different categories based on the category information, but cannot control the size and location of the generated defects. In industrial defect images, some defects appear within a relatively fixed range and do not appear arbitrarily in the image. Therefore, it is necessary to control the location of the generated defects to be reasonable. While the defect locations generated by the CGAN model generator are relatively fixed, this fixed location may not be where the defect should appear. Furthermore, defects in industrial defect images have different sizes, and directly using the CGAN model generator results in relatively fixed defect sizes, making it impossible to generate defects of different sizes.
[0037] To address the issue of CGAN models' inability to control the size and location of generated defects, the FSGAN model generator employs a different input method. The FSGAN model generator incorporates a mask image into its input, which is labeled with the desired size and location of the defects to be generated. The FSGAN model generator can then generate defect images based on this size and location information from the mask image. The input to the FSGAN model generator includes positive sample images, a mask image, a noise image, and a class image. This invention refers to the input image of the FSGAN model generator as the class location input image. The structure of the class location input image in the FSGAN model is shown below. Figure 3 .
[0038] The FSGAN model generator requires positive sample images, mask images, noisy images, and class images as input images for class location. Positive sample images are images without defects, and each pixel in the image is represented by an Si symbol. i,j In this diagram, i represents the height of the positive sample image, and j represents the width of the positive sample image; the mask image is an image with the defect locations marked, where pixels at the defect locations are represented by 1, and pixels at the background locations are represented by 0; the noise image is a randomly generated Gaussian noise image, where each pixel is represented by Z. i,j Let i represent the height of the positive sample image and j represent the width of the positive sample image; the category image is an image containing the defect category, and each pixel in the image is represented by C. The positive sample image, mask image, noise image, and category image are fused to obtain the category location input image for the FSGAN model generator. The background region in the input image is represented by S. i,j The defect area is represented by C*Z. i.jThis indicates that when the FSGAN model generator generates defect images based on the input image with the category location, it only needs to fit the features of the defect region, without needing to fit the features of the background region. The resulting defect images are clearer and more similar to real defects. Furthermore, the FSGAN model generator can generate defect images based on the defect location and size provided in the mask image, allowing control over the size and location of the generated defects.
[0039] S4. Set the model's hyperparameters and train the model. Set the batch size to 128, the number of iterations to 2000 epochs, the learning rate to 0.0002, and use the SGD gradient update strategy with a fixed learning rate. Train the model using pre-trained model weights.
[0040] S5. Analyze the experimental results. If the generated defects are similar to the real defects, end the training. If the generated defects are significantly different from the real defects, adjust the model's hyperparameters and continue training.
[0041] The above description is only a specific embodiment of the present invention. Various examples and illustrations do not constitute a limitation on the substantive content of the present invention. Those skilled in the art can make modifications or variations to the above-described specific embodiments after reading the specification without departing from the substance and scope of the invention.
Claims
1. A method for generating industrial defect images, characterized in that: Includes the following steps, S1. Collect and create the FSGAN model dataset, collect different positive sample images, and provide a mask image for each positive sample image. Each mask image is labeled with the location and size information of the expected defect. For each positive sample image, a category image is provided, which records the category of the defect; for each positive sample image, a noise image is provided, which contains randomly generated Gaussian noise; the positive sample image, mask image, noise image and category image are fused to obtain the category location input image of the FSGAN model generator; S2. Perform data augmentation and data normalization preprocessing on the data; S3. Construct the FSGAN model generator structure. The FSGAN model generator consists of 13 convolutional layers, 2 deconvolutional layers, 11 normalization layers, and 12 activation function layers. After the input image is fed into the model, it passes through 3 convolutional-deconvolutional modules, 3 residual modules, 2 upsampling modules, and 4 convolutional-deconvolutional modules before outputting. The convolutional-deconvolutional module consists of one convolutional layer, one normalization layer, and one activation function layer. The convolutional layer in the first convolutional-deconvolutional module has a 7×7 kernel size and a stride of 1, used to extract features from the input feature map. The convolutional layers in the other two convolutional-deconvolutional modules have a 3×3 kernel size and a stride of 2, used for downsampling to increase the receptive field of the model. The residual module consists of one convolutional-deconvolutional module, one convolutional layer, and one normalization layer, both with a 3×3 kernel size and a stride of 1, used to extract features from the input feature map to obtain high-level semantic features. The upsampling module consists of a deconvolution layer, a normalization layer, and an activation function layer. The deconvolution layer has a kernel size of 4×4 and a stride of 2, and is used to upsample the input feature map. The last four convolutional and normalized modules consist of a convolutional layer, a normalization layer, and an activation function layer. The sizes of each convolutional kernel are 1×1, 3×3, 5×5, and 7×7, respectively, and the convolutional stride is 1. They are used to extract features from the input feature map. S4. Set the hyperparameters of the model and train the model; S5. Analyze the results of the experiment.
2. The method for generating industrial defect images as described in claim 1, characterized in that: In step S2, the data augmentation methods include random rotation, random cropping, and Gaussian blur; the data normalization method is as follows: ,in, These are input features. It is the output feature.
3. The industrial defect image generation method as described in claim 1, characterized in that: In step S4, the model is trained using pre-trained model weights; the SGD gradient update strategy is adopted, and the learning rate update rule uses a fixed learning rate.
4. The method for generating industrial defect images as described in claim 1, characterized in that: In step S5, if the generated defect is similar to the real defect, the training ends; if the generated defect differs significantly from the real defect, the hyperparameters of the model are adjusted and training continues.