Industrial defect sample generation method based on generative model

Abstract

The invention provides an industrial defect sample generation method based on a generative model, and the method comprises the steps: enabling the generative model to learn the distribution characteristics of image data, then reconstructing new image data which is the same as an original image in distribution and is different from the original image in sampling, and generating new defect sample images of the same type and different forms. The defect sample generation method based on the generative model can efficiently generate a large number of defect sample images in various forms, can be used for solving the problems of difficulty in defect sample collection and insufficient diversity in a product appearance quality inspection link, and enables training data to cover defect samples in all forms as much as possible, thereby improving the detection capability of the model and improving the detection efficiency. The number of times of iterative optimization of the model is reduced; the invention belongs to the technical field of visual detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual inspection, and in particular relates to a method for generating industrial defect samples based on a generative model. Background Art

[0002] Product appearance defect detection is a key link in the manufacturing industry, and also a difficult link. The accuracy of product appearance quality inspection will directly affect the quality level of the product and the production efficiency of the factory. The rapid development of deep learning technology has enabled it to be widely used in the field of industrial product quality inspection, and has been effectively applied in the quality inspection links of industries such as consumer electronics, semiconductors, electronic components, automotive parts, and medical equipment. Among them, the semantic segmentation task is mainly used to locate the product defect position and segment the defect contour, and can also identify the defect category, which can realize refined quality inspection and is the most commonly used method in the quality inspection link; while the target detection task is mainly used to locate the defect position and identify the defect category, which is also commonly used in the quality inspection link; in addition, the classification task is mainly used to identify the defect category, and it is impossible to locate the defect position and segment the defect contour; and the anomaly detection task is mainly used to determine whether the product is NG, which can locate the defect position but cannot identify the defect category. The above deep learning methods can all realize automatic and efficient quality inspection tasks, which are of vital significance to improving production efficiency and reducing production costs.

[0003] Appearance quality inspection methods based on deep learning technology often require a large number of defect sample images to train neural network models. In practical applications, it is difficult to obtain defect sample images. Common scenarios generally have many types of defects, but the amount of defect sample data is small and the diversity is insufficient. Therefore, at different stages of the project, there are challenges in defect data. In the early stages of the project, the detection accuracy of some categories of defects often cannot meet the requirements due to insufficient collection of defect samples. However, through continuous iterative optimization of the model, the model can already detect known defects well in the later stages of the project. Due to production process problems, it is inevitable to produce various new forms of defects, which may lead to model omissions, so it is necessary to expand defect samples.

[0004] Artificially creating defects in product appearance is a method, but it is more subjective and does not match the defects produced under actual production conditions; amplifying the morphology and color of defective sample images is a common and effective method, but the model trained on this basis still has the problem of not being able to adapt to new forms of defects; although new defects of various forms can be PSed out through Photoshop software, it is also more subjective and less efficient. Summary of the invention

[0005] The purpose of the present invention is to provide a method for generating industrial defect samples based on a generative model to solve the technical problems described in the background technology.

[0006] In the above-mentioned method for generating industrial defect samples based on a generative model, the generative model is a generative adversarial network, which includes a generator model for generating defect sample images and a discriminator model for distinguishing between real images and images generated by the generator model; The industrial defect sample generation method comprises the following steps: S1, obtaining defect sample images and preprocessing them, and dividing the sample data according to the defect categories; S2. Build a generator model, including five generator modules Generator, each generator module contains convolution, convolutional attention module CBAM, deconvolution and instance normalization; S3. Build a discriminator model, adopt the structure of the EfficientNet classification model, and prune and compress it; S4. Train the generative adversarial network and save the generator model. In addition, the generator model and the discriminator model are trained alternately. The generator model is first trained for one or more iterations, and then the discriminator model is trained for one or more iterations. This training process is repeated continuously until the Nash equilibrium state is finally reached. At this time, the generator model can be saved for subsequent defect sample generation. S5. Generate defect sample images using the generator model.

[0007] In some embodiments, the S1 specifically comprises the following steps: S1-1, collect defect sample images and perform preprocessing. Because different categories of defects of the same product may have large differences in imaging and morphology, if all categories of defects share the same generative model, it may not be possible to generate the desired defect samples. Therefore, the sample data is divided according to the defect category, and the generative models are trained separately according to the defect category. Each generative model only generates defect samples of the corresponding category. This can improve the effectiveness of the generative model and ensure the authenticity of the generated defect samples.

[0008] In some embodiments, S2 specifically includes the following steps: S2-1, the generator model takes standard normal distribution data as input, and generates defect sample images through five generator modules and convolution output layers; S2-2. The generator module Generator includes two convolutional layers, two convolutional attention modules CBAM, a deconvolution layer and an instance normalization layer. The CBAM module adaptively learns channel and spatial attention weights to improve the feature expression ability of the model. Channel attention can enhance the information of key feature channels and suppress the information of irrelevant feature channels through weight distribution. Spatial attention can highlight important image areas and reduce the influence of unimportant areas through weight distribution. In addition, when calculating the normalized statistics in instance normalization, only a single sample is considered, without cross-sample and cross-channel calculations, so as to be more suitable for use scenarios with higher requirements for single pixels. Compared with batch normalization, it improves the quality of generated images and achieves high-quality reconstruction of defect sample images. S2-3, the convolution output layer only has simple convolution operations, which mainly convolutes the input feature map to output a defect sample image of a specified size and number of channels.

[0009] In some implementations, the S3 specifically includes the following steps: S3-1. EfficientNet coordinates and balances the depth, width, and resolution of the model, achieving relatively excellent performance in image classification tasks. EfficientNet mainly includes the MBConv structure. stage operate Number of feature map channels Repetitions 1 Conv3×3 32 1 2 MBConv1, k3×3 32 1 3 MBConv2, k3×3 64 2 4 MBConv3, k5×5 64 2 5 MBConv4, k3×3 64 2 6 MBConv5, k5×5 64 2 7 Conv1×1&Pooling&FC 256 1 .

[0010] In some embodiments, the S4 specifically includes the following steps: S4-1. The basic idea of ​​the generative adversarial network is to use the generator model to generate forged data, and then use the discriminator model to identify the authenticity of the data. In addition, through iterative training, the discriminator model can classify the defective samples in the training set as real data, and classify the defective samples generated by the generator model as false data. The generator model can generate defective sample data that is indistinguishable from the real thing to deceive the discriminator model.

[0011] In some implementations, the S5 specifically includes the following steps: S5-1. Randomly generate normal distribution data and input it into the generator model to generate defect sample images, and control the number of generated defect samples by setting parameters; S5-2. Since it cannot be guaranteed that every defect sample image generated is usable, browse the generated defect sample images and manually delete the unusable samples.

[0012] According to the above technical solution, the present invention can achieve the following beneficial effects: The generative model learns the distribution characteristics of image data and then reconstructs new image data with the same distribution but different sampling as the original image, that is, it can generate new defect sample images of the same category but different forms. The defect sample generation method based on the generative model can efficiently and massively generate defect sample images of various forms, which can be used to solve the problems of difficulty in collecting defect samples and insufficient diversity in the product appearance quality inspection link, so that the training data can cover all forms of defect samples as much as possible, thereby improving the detection ability of the model and reducing the number of model iteration optimizations. Therefore, it can be used to solve the problems of difficulty in collecting defect samples and insufficient diversity in the product appearance quality inspection link, so that the training data can cover all forms of defect samples as much as possible, thereby improving the detection ability of the model and reducing the number of model iteration optimizations, speeding up the progress of project launch and acceptance. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the specific implementation of the present invention, the following is a brief description of the drawings and reference numerals required to be used in describing the specific implementation.

[0014] Figure 1 It is a structural diagram of the generator model of the present invention; Figure 2 It is a structural diagram of the generator module of the present invention; Figure 3 is a structural diagram of MBConv according to the present invention; Figure 4 This is the adversarial network structure diagram of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in combination with specific implementation methods and with reference to the accompanying drawings.

[0016] This specific embodiment provides a method for generating industrial defect samples based on a generative model.

[0017] The generative model may be a generative adversarial network, and the generative adversarial network may be composed of a generator model for generating defect sample images and a discriminator model for distinguishing between real images and images generated by the generator model.

[0018] The industrial defect sample generating method comprises the following steps.

[0019] S1, obtaining defect sample images and preprocessing them, and dividing the sample data according to the types of defects; S1 is specifically as follows: Collect defect sample images and perform preprocessing. Different categories of defects of the same product may have large differences in imaging and morphology. If all categories of defects share the same generative model, it may not be possible to generate the desired defect samples. The sample data is divided according to the defect category, and the generative models are trained according to the defect category. Each generative model only generates defect samples of the corresponding category. This can improve the effectiveness of the generative model and ensure the authenticity of the generated defect samples.

[0020] S2. Construct a generator model, which consists of five generator modules Generator. Each generator module includes convolution, convolutional attention module CBAM, deconvolution and instance normalization. The details of S2 are as follows: S2-1, the generator model takes the standard normal distribution data as input, and generates defect sample images through five generator modules and convolution output layers. The generator model is shown in Figure 1; S2-2. The generator module Generator consists of two convolutional layers, two convolutional attention modules CBAM, a deconvolution layer and an instance normalization layer. The generator module is shown in Figure 2. The CBAM module adaptively learns channel and spatial attention weights to improve the feature expression ability of the model. Channel attention can enhance the information of key feature channels and suppress the information of irrelevant feature channels through weight distribution. Spatial attention can highlight important image areas and reduce the influence of unimportant areas through weight distribution. In addition, instance normalization only considers a single sample when calculating the normalized statistics, without cross-sample and cross-channel calculations. It is more suitable for use scenarios with higher requirements for single pixels. Compared with batch normalization, it can improve the quality of generated images and achieve high-quality reconstruction of defect sample images. S2-3, the convolution output layer only has simple convolution operations, which mainly convolutes the input feature map to output a defect sample image of a specified size and number of channels.

[0021] S3, build a discriminator model, adopt the structure of the EfficientNet classification model, and prune and compress it; S3 is as follows: S3-1, EfficientNet coordinates and balances the depth, width and resolution of the model, and achieves excellent performance in image classification tasks. EfficientNet is mainly composed of MBConv structure. MBConv structure is as follows Figure 3 As shown; stage operate Number of feature map channels Repetitions 1 Conv3×3 32 1 2 MBConv1, k3×3 32 1 3 MBConv2, k3×3 64 2 4 MBConv3, k5×5 64 2 5 MBConv4, k3×3 64 2 6 MBConv5, k5×5 64 2 7 Conv1×1&Pooling&FC 256 1 Table 1 S4, training a generative adversarial network and saving the generator model; S4 is specifically as follows: S4-1. The basic idea of ​​the generative adversarial network is to use the generator model to generate forged data, and then use the discriminator model to identify the authenticity of the data. The generative adversarial network is shown in Figure 4. In addition, through iterative training, the discriminator model can classify the defective samples in the training set as real data, and classify the defective samples generated by the generator model as false data. The generator model can generate defective sample data that is indistinguishable from the real thing to deceive the discriminator model. In addition, the generator model and the discriminator model are trained alternately. The generator model is first trained for one or more iterations, and then the discriminator model is trained for one or more iterations. The training process is repeated continuously until the Nash equilibrium state is finally reached. At this time, the generator model can be saved for subsequent defective sample generation.

[0022] S5, using the generator model to generate a defect sample image; S5 is specifically as follows: S5-1. Randomly generate normal distribution data and input it into the generator model to generate defect sample images, and control the number of generated defect samples by setting parameters; S5-2. Browse the generated defect sample images and manually delete unusable samples to ensure that each generated defect sample image is usable.

[0023] In the above-mentioned industrial defect sample generation method, the generative model learns the distribution characteristics of the image data, and then reconstructs new image data with the same distribution but different sampling as the original image, that is, it can generate new defect sample images of the same category but different forms. The defect sample generation method based on the generative model can efficiently and massively generate defect sample images of various forms, which can be used to solve the problems of difficulty in collecting defect samples and insufficient diversity in the product appearance quality inspection link, so that the training data can cover defect samples of all forms as much as possible, thereby improving the detection ability of the model and reducing the number of model iteration optimizations. In addition, the industrial defect sample generation method can be used to solve the problems of difficulty in collecting defect samples and insufficient diversity in the product appearance quality inspection link, so that the training data can cover defect samples of all forms as much as possible, thereby improving the detection ability of the model and reducing the number of model iteration optimizations. This industrial defect sample generation method is an important auxiliary means in the product quality inspection link, which can speed up the progress of project launch and acceptance.

Claims

1. A method for generating industrial defect samples based on a generative model, characterized in that: The generative model is a generative adversarial network, which includes a generator model for generating defect sample images and a discriminator model for distinguishing between real images and images generated by the generator model; The industrial defect sample generation method comprises the following steps: S1, obtaining defect sample images and preprocessing them, and dividing the sample data according to the defect categories; S2. Build a generator model, including five generator modules Generator, each generator module contains convolution, convolutional attention module CBAM, deconvolution and instance normalization; S3. Build a discriminator model, adopt the structure of the EfficientNet classification model, and prune and compress it; S4. Train the generative adversarial network and save the generator model. In addition, the generator model and the discriminator model are trained alternately. The generator model is first trained for one or more iterations, and then the discriminator model is trained for one or more iterations. This training process is repeated continuously until the Nash equilibrium state is finally reached. At this time, the generator model can be saved for subsequent defect sample generation. S5. Generate defect sample images using the generator model.

2. The method for generating industrial defect samples according to claim 1, characterized in that: The S1 specifically includes the following steps: S1-1, collect defect sample images and perform preprocessing. Because different categories of defects of the same product may have large differences in imaging and morphology, if all categories of defects share the same generative model, it may not be possible to generate the desired defect samples. Therefore, the sample data is divided according to the defect category, and the generative models are trained separately according to the defect category. Each generative model only generates defect samples of the corresponding category. This can improve the effectiveness of the generative model and ensure the authenticity of the generated defect samples.

3. The method for generating industrial defect samples according to claim 1, characterized in that: The S2 specifically includes the following steps: S2-1, the generator model takes standard normal distribution data as input, and generates defect sample images through five generator modules and convolution output layers; S2-2. The generator module Generator includes two convolutional layers, two convolutional attention modules CBAM, a deconvolution layer and an instance normalization layer. The CBAM module adaptively learns channel and spatial attention weights to improve the feature expression ability of the model. Channel attention can enhance the information of key feature channels and suppress the information of irrelevant feature channels through weight distribution. Spatial attention can highlight important image areas and reduce the influence of unimportant areas through weight distribution. In addition, when calculating the normalized statistics in instance normalization, only a single sample is considered, without cross-sample and cross-channel calculations, so as to be more suitable for use scenarios with higher requirements for single pixels. Compared with batch normalization, it improves the quality of generated images and achieves high-quality reconstruction of defect sample images. S2-3, the convolution output layer only has simple convolution operations, which mainly convolutes the input feature map to output a defect sample image of a specified size and number of channels.

4. The method for generating industrial defect samples according to claim 1, characterized in that: The S3 specifically includes the following steps: S3-1. EfficientNet coordinates and balances the depth, width, and resolution of the model, achieving relatively excellent performance in image classification tasks. EfficientNet mainly includes the MBConv structure. S3-2. The discriminator model only performs binary classification and does not require a large network structure. Therefore, pruning and compression operations are performed on the depth, width, and resolution of EfficientNet. The discriminator model is as follows: 。 5. The method for generating industrial defect samples according to claim 1, characterized in that: The S4 specifically comprises the following steps: S4-1. The basic idea of ​​the generative adversarial network is to use the generator model to generate forged data, and then use the discriminator model to identify the authenticity of the data. In addition, through iterative training, the discriminator model can classify the defective samples in the training set as real data, and classify the defective samples generated by the generator model as false data. The generator model can generate defective sample data that is indistinguishable from the real thing to deceive the discriminator model.

6. The method for generating industrial defect samples according to claim 1, characterized in that: The S5 specifically includes the following steps: S5-1. Randomly generate normal distribution data and input it into the generator model to generate defect sample images, and control the number of generated defect samples by setting parameters; S52. Since it cannot be guaranteed that every defect sample image generated is usable, browse the generated defect sample images and manually delete unusable samples.

Images