Appearance defect detection method based on unsupervised learning

Abstract

The invention aims to provide an appearance defect detection method based on unsupervised learning, which can reduce tedious data set collection and preparation work, and can solve the problems of difficulty in data acquisition, time consumption in data labeling and easiness in model missing detection of a supervised learning appearance defect detection method. The method comprises the following steps that S1, normal samples are obtained and preprocessed, the number of the normal samples is set according to requirements, and abnormal samples are generated in combination with Perlin noise; s2, performing down-sampling feature extraction on the generated abnormal sample by using a convolutional neural network to obtain a hidden variable of a specified dimension, and encoding the hidden variable into a discrete variable by using a codebook mechanism; s3, performing up-sampling reduction on the discrete variable by using a deconvolution neural network, and further reconstructing a normal sample; and S4, inputting a test sample into the trained model to obtain an abnormal region in the test sample. The method is applied to the technical field of appearance defect detection.

Description

Technical Field

[0001] The present invention is applied to the technical field of appearance defect detection, and in particular relates to an appearance defect detection method based on unsupervised learning. Background Art

[0002] In the process of display panel manufacturing, defects such as scratches, indentations, broken pieces, dust, stains, etc. on the display panel often affect the quality of the display panel and the output results of the finished product grade. Therefore, in the process of display panel manufacturing, it is necessary to perform defect detection on the various components that make up the display panel and the surface of the successfully assembled display panel. At present, the detection of the components of the display panel and its display panel mainly relies on human eye observation and the use of traditional image processing algorithms for detection. Defect detection based on the human eye not only has a strong subjective factor; in addition, the human eye will also suffer from visual fatigue after long-term detection; and the panel defect detection algorithm based on the traditional image processing algorithm has poor generalization ability, redundant parameters to be adjusted, low intelligence, and high over-detection rate and missed detection rate.

[0003] Product quality is a very critical factor in the manufacturing industry. In the process of industrial production, various defects are inevitable due to production process reasons. Severe defects will directly affect the use of products. Although manual quality inspection has a certain degree of accuracy, it has high labor costs and low efficiency; traditional visual algorithms can realize automatic defect detection, but they are highly dependent on human professional ability, sensitive to environmental changes and lack generalization; deep learning algorithms have powerful automatic feature extraction capabilities, which can realize automatic and efficient defect detection and have been widely used in various industrial scenarios, such as 3C, electronic components, medical devices and semiconductors. The detection scheme based on supervised learning first needs to collect a large number of abnormal samples and annotate the data, and then train the model and deploy it on the machine. In actual industrial scenarios, it is often difficult to collect a large number of abnormal samples, and data annotation is very time-consuming. When the production environment or product changes, the above data set preparation process needs to be repeated. In reality, it is impossible to predict what kind of abnormal samples will appear, and the model is prone to the risk of missed detection. Therefore, it is necessary to provide an appearance defect detection method based on unsupervised learning, which can reduce the tedious data set collection and preparation work, and can solve the problems of data collection difficulties, time-consuming data annotation, and easy missed detection of the model in the supervised learning appearance defect detection method. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an appearance defect detection method based on unsupervised learning, which can reduce the tedious data set collection and preparation work, and can solve the problems of difficult data collection, time-consuming data labeling, and easy model omissions in supervised learning appearance defect detection methods.

[0005] The technical solution adopted by the present invention is: the method comprises the following steps: Step S1: obtain normal samples and perform preprocessing. The number of normal samples is set according to demand, and abnormal samples are generated by combining Perlin noise; Perlin noise (also known as Perlin noise) refers to a natural noise generation algorithm invented by Ken Perin, which has continuity in functions and can give consistent values ​​when called multiple times. In the field of video games, Perlin noise can be used to generate continuous terrain; or in the field of art, Perlin noise can be used to generate patterns; Step S2: Use the convolutional neural network to perform downsampling feature extraction on the generated abnormal samples to obtain latent variables of the specified dimension, and use the codebook mechanism to encode the latent variables into discrete variables; Step S3: Use a deconvolutional neural network to upsample and restore discrete variables, and then reconstruct normal samples; Step S4: Input the test sample into the trained model to obtain the abnormal area in the test sample.

[0006] It can be seen from the above scheme that the appearance defect detection method based on unsupervised learning can solve the problems of difficult data collection, time-consuming data labeling, and easy model omissions in the supervised learning appearance defect detection method; it is more in line with actual usage scenarios and can realize end-to-end appearance defect detection; it can reduce the tedious data set collection and preparation work.

[0007] A preferred solution is that step S1 includes the following subdivision steps: Step S11: Generate a Perlin noise image of the same size according to the size of the normal sample and perform threshold segmentation and binarization processing. The Perlin noise image generated each time is different, and the size and number of white areas are controlled by parameters; Step S12: randomly select a normal sample image, slice it, shuffle it, and then splice it, and then perform calculations based on PM to obtain a noise defect map; PM refers to Figure 1 Image logo of Step S13: first invert PM, then perform an AND operation with the normal sample image to blacken the noise area, then perform an addition operation with the noise defect image and add a transparency factor to obtain the abnormal sample corresponding to the normal sample; since the Perlin noise image generated each time is different, all normal sample images can be repeatedly selected; A preferred solution is that step S2 includes the following subdivision steps: Step S21: Downsampling feature extraction uses a combination of convolutional layer, batch normalization layer and activation function layer, and needs to be downsampled to 1 / 16 of the original image; Step S22: Use the codebook mechanism to encode the latent variables into discrete variables and reconstruct a more realistic normal sample image.

[0008] A preferred solution is that step S3 includes the following subdivision steps: Step S31: Upsampling restoration can be performed using a combination of a deconvolution layer, a batch normalization layer, and an activation function layer. The image needs to be restored to the size of a normal sample image. The model is trained by minimizing the error between the reconstructed normal sample image and the model input normal sample image as the loss function. The loss function is an MSE function or a SSIM function.

[0009] A preferred solution is that step S4 includes the following subdivision steps: Step S41: input the test sample into the trained model to reconstruct a normal sample, and subtract the test sample from the reconstructed normal sample to obtain the abnormal area of ​​the test sample; Step S42: Filter out the desired abnormal area by setting the pixel number range and pixel value range of the abnormal area. One pixel number range corresponds to one pixel value range. For example, the pixel number range of the abnormal area is 10~20, and the pixel value range is 30~50. Multiple range correspondences can be set.

[0010] A preferred solution is that step S42 includes the following subdivision steps: Step S421: Input the test image into the neural network model, reconstruct the normal sample, subtract the test image from the normal sample to obtain the abnormal area, screen the number of pixels and pixel values, and output the defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is the Perlin noise image of step S11 and the noise image subjected to threshold segmentation and binarization processing; Figure 2 is the overall structure diagram of the model of step S31; Figure 3 This is a test flow chart of step S42. DETAILED DESCRIPTION

[0012] In this embodiment, the present invention includes the following steps: Step S1: Get normal samples and preprocess them. The number of normal samples is set according to the requirements. Combined with Perlin noise, abnormal samples are generated. Perlin noise refers to a natural noise generation algorithm invented by Ken Perin. It has continuity in function and can give consistent values ​​when called multiple times. In the field of electronic games, Perlin noise can be used to generate continuous terrain; or in the field of art, Perlin noise can be used to generate patterns. Step S2: Use the convolutional neural network to perform downsampling feature extraction on the generated abnormal samples to obtain latent variables of the specified dimension, and use the codebook mechanism to encode the latent variables into discrete variables; Step S3: Use a deconvolutional neural network to upsample and restore discrete variables, and then reconstruct normal samples; Step S4: Input the test sample into the trained model to obtain the abnormal area in the test sample.

[0013] In this embodiment, step S1 includes the following subdivision steps: Step S11: Generate a Perlin noise image of the same size according to the size of the normal sample and perform threshold segmentation and binarization processing. The Perlin noise image generated each time is different, and the size and number of white areas are controlled by parameters; Step S12: randomly select a normal sample image, slice it, shuffle it, and then splice it, and then perform calculations based on PM to obtain a noise defect map; PM refers to Figure 1 Image logo of Step S13: first invert PM, then perform an AND operation with the normal sample image to blacken the noise area, then perform an addition operation with the noise defect image and add a transparency factor to obtain the abnormal sample corresponding to this normal sample; since the Perlin noise image generated each time is different, all normal sample images can be selected repeatedly.

[0014] In this embodiment, step S2 includes the following subdivision steps: Step S21: Downsampling feature extraction uses a combination of convolutional layer, batch normalization layer and activation function layer, and needs to be downsampled to 1 / 16 of the original image; Step S22: Use the codebook mechanism to encode the latent variables into discrete variables and reconstruct a more realistic normal sample image.

[0015] In this embodiment, step S3 includes the following subdivision steps: Step S31: Upsampling restoration can be performed using a combination of a deconvolution layer, a batch normalization layer, and an activation function layer. The image needs to be restored to the size of a normal sample image. The model is trained by minimizing the error between the reconstructed normal sample image and the model input normal sample image as a loss function. The loss function is an MSE function or a SSIM function, as shown in FIG2 .

[0016] In this embodiment, step S4 includes the following subdivision steps: Step S41: input the test sample into the trained model to reconstruct a normal sample, and subtract the test sample from the reconstructed normal sample to obtain the abnormal area of ​​the test sample; Step S42: The desired abnormal area is screened out by setting the pixel quantity range and pixel value range of the abnormal area. One pixel data quantity range corresponds to one pixel value range. For example, the pixel quantity range of the abnormal area is 10-20, and the pixel value range is 30-50. Multiple range correspondences can be set. As shown in FIG3 .

[0017] In this embodiment, step S42 includes the following subdivision steps: Step S421: Input the test image into the neural network model, reconstruct the normal sample, subtract the test image from the normal sample to obtain the abnormal area, screen the number of pixels and pixel values, and output the defects.

Claims

1. A method for detecting appearance defects based on unsupervised learning, characterized in that: The method comprises the following steps: Step S1: obtain normal samples and perform preprocessing. The number of normal samples is set according to demand, and abnormal samples are generated by combining Perlin noise; Step S2: Use the convolutional neural network to perform downsampling feature extraction on the generated abnormal samples to obtain latent variables of the specified dimension, and use the codebook mechanism to encode the latent variables into discrete variables; Step S3: Use a deconvolutional neural network to upsample and restore discrete variables, and then reconstruct normal samples; Step S4: Input the test sample into the trained model to obtain the abnormal area in the test sample.

2. The appearance defect detection method based on unsupervised learning according to claim 1, characterized in that: The step S1 includes the following subdivision steps: Step S11: Generate a Perlin noise image of the same size according to the size of the normal sample and perform threshold segmentation and binarization processing. The Perlin noise image generated each time is different, and the size and number of white areas are controlled by parameters; Step S12: randomly select a normal sample image, slice it, shuffle it, and then splice it, and then perform calculations according to PM to obtain a noise defect map, where PM refers to the image identifier of Figure 1; Step S13: First, PM is inverted, and then an AND operation is performed with the normal sample image to blacken the noise area. Then, an addition operation is performed with the noise defect image and a transparency factor is added to obtain the abnormal sample corresponding to the normal sample.

3. The appearance defect detection method based on unsupervised learning according to claim 1, characterized in that: The step S2 includes the following subdivision steps: Step S21: Downsampling feature extraction uses a combination of convolutional layer, batch normalization layer and activation function layer, and needs to be downsampled to 1 / 16 of the original image; Step S22: Use the codebook mechanism to encode the latent variables into discrete variables and reconstruct a more realistic normal sample image.

4. The appearance defect detection method based on unsupervised learning according to claim 1, characterized in that: The step S3 includes the following subdivision steps: Step S31: Upsampling restoration can be performed using a combination of a deconvolution layer, a batch normalization layer, and an activation function layer. The image needs to be restored to the size of a normal sample image. The model is trained by minimizing the error between the reconstructed normal sample image and the model input normal sample image as the loss function. The loss function is an MSE function or a SSIM function.

5. The appearance defect detection method based on unsupervised learning according to claim 1, characterized in that: The step S4 includes the following subdivision steps: Step S41: input the test sample into the trained model to reconstruct a normal sample, and subtract the test sample from the reconstructed normal sample to obtain the abnormal area of ​​the test sample; Step S42: Filter out the desired abnormal area by setting the pixel number range and pixel value range of the abnormal area. One pixel number range corresponds to one pixel value range. For example, the pixel number range of the abnormal area is 10~20, and the pixel value range is 30~50. Multiple range correspondences can be set.

6. The appearance defect detection method based on unsupervised learning according to claim 5, characterized in that: The step S42 includes the following subdivision steps: Step S421: Input the test image into the neural network model, reconstruct the normal sample, subtract the test image from the normal sample to obtain the abnormal area, screen the number of pixels and pixel values, and output the defects.

Images