Industrial visual defect detection method

Abstract

The invention provides an industrial visual defect detection method, which realizes defect (flaw) detection of a product by using a neural network automatic learning threshold value so as to obviously improve the detection accuracy and reduce the defect omission ratio and over-killing rate, and in addition, as the industrial visual defect detection method has automatically learned a binary approximate image, the detection accuracy is greatly improved, and the detection efficiency is greatly improved. The binary approximate image can be binarized into a standard binary image only through a simple threshold value, so that defect detection is realized; the invention belongs to the technical field of visual detection.

Description

Technical Field

[0001] The invention belongs to the technical field of visual inspection, and in particular relates to an industrial visual defect inspection method. Background Art

[0002] In the process of industrial manufacturing, due to process reasons, the products produced will inevitably have defects (blemishes), which will cause the product to be unqualified. In order to enable the machine to automatically detect unqualified products, it is necessary to detect whether there are defects on the product.

[0003] At present, a commonly used defect detection method is mainly to use an autoencoder network for detection. The autoencoder network is a deep learning neural network that is often used in scenarios such as image restoration or image reconstruction. The process of using an autoencoder network for defect detection includes: using an industrial camera to collect images of products that need to be detected for defects, inputting the collected images into a trained (learned) autoencoder network, and the autoencoder network outputs a reconstructed image. In theory, if there is no defect in the input image, the output reconstructed image is exactly the same as the input image; if there is a defect in the input image, then the defect is repaired in the reconstructed image, that is, the defect does not exist in the reconstructed image; therefore, by comparing the input image and the reconstructed image, the area where the difference between the two is in the image indicates that the area is a defect, wherein the specific method is: subtracting the input image and the reconstructed image to obtain the subtracted image, then taking the absolute value of each value in the subtracted image to obtain a single-channel difference image, and then manually setting a threshold to binarize the difference image into a binary image, and extracting the highlight area from the binary image through some image processing methods, that is, the area where the difference exists, thereby realizing defect detection.

[0004] In the above method, the detection of defects is determined by binary images, so whether a reasonable and more accurate binary image can be generated is very critical. However, in conventional defect detection, the binary image is obtained by manually setting a threshold to binarize the difference image, which is highly dependent on the accuracy of the threshold. Therefore, whether the threshold can be set correctly is very critical. However, due to the variety of product defect types and the complex and changeable appearance characteristics of the defects such as color and texture, it is difficult to use a threshold to correctly separate all defects from the binary image, which makes it very difficult to manually set the threshold. Summary of the invention

[0005] The purpose of the present invention is to provide an industrial visual defect detection method to achieve product defect (flaw) detection through automatic learning thresholds of a neural network, thereby solving the technical defects described in the background technology.

[0006] The industrial visual defect detection method is applied to a neural network, wherein the neural network includes an autoencoder network and a threshold learning network; Wherein, the autoencoder network is mainly used to reconstruct an image. In addition, the autoencoder network includes an encoder and a decoder; The threshold learning network is used to learn the threshold to automatically binarize the difference image, and then output the binary approximate image. The binary approximate image is binarized into a standard binary image by setting a simple threshold. The highlight area in the binary image is the area corresponding to the defect, thereby realizing defect detection. In addition, the threshold learning network includes convolution module 1, convolution module 2, convolution module 3, convolution module 4 and a differentiable binarization module; The industrial visual defect detection method comprises the following steps: First, input the image, collect the grayscale image or color image of the product by the industrial camera, pre-process and normalize it to between 0 and 1, and input it to the encoder and convolution module 2; The encoder extracts features from the input image through several convolutional layers and inputs them to the decoder and convolution module 3; The decoder reconstructs the image from the extracted features through several convolutional layers and upsampling layers and transmits it to the convolution module 1; Convolution module 1, which is used to extract reconstructed image features F1 from the reconstructed image, and shares weights with convolution module 2; Convolution module 2, used to extract input image feature F2 from the input image, which shares weights with convolution module 1; Difference image: reconstruct the image feature F1 minus the input image feature F2, and take the absolute value of the subtracted feature to get the difference image; Convolution module 3, which consists of several convolutional layers and upsampling layers, and forms a U-shaped structure with the encoder, which is used to extract the features of the input image; Convolution module 4, which includes two upsampling layers and a 1x1 convolution layer with 1 convolution kernel, wherein the two upsampling layers are used to upsample the feature resolution extracted by convolution module 3 to the same resolution as the input image, and the convolution layer is used to learn and predict the threshold to obtain a threshold map, and the resolution of the threshold map is the same as the resolution of the difference image; Differentiable binarization module, which is a differentiable binarization function, such as , in, P refers to the difference image, T refers to the threshold map, [ i, j ] refers to the coordinate index, [ Pi,j ] refers to the difference image with coordinate index [ i, j] eigenvalue at [ Ti,j ] refers to the threshold map where the coordinate index is [ i, j ], k is the amplification factor, which is used to control the degree of change of the function. B is a binary approximate image; The binary approximate image is a single-channel image finally predicted by the network, in which the pixel values ​​are composed of numbers between 0 and 1. By setting a simple threshold, the binary approximate image can be binarized into a binary image composed of 0 and 1. The location information of the defect can be calculated from the binary image through some image processing methods.

[0007] In some embodiments, the industrial visual defect detection method further comprises the following steps: The true value mask image is the learning label of the threshold learning network. It is used to supervise the learning of the threshold learning network. It is a standard binary mask image composed of 0 and 1. It is the mask of the input image, where 0 represents the background area and 1 represents the defect area. The true value mask image is obtained by the binary approximation image to obtain Loss_mask, which is the cross entropy loss of the two classifications.

[0008] In some embodiments, the industrial visual defect detection method further comprises the following steps: The true value image is the learning label of the autoencoder network, which is used to supervise the learning of the autoencoder network. The true value image is exactly the same as the input image. The true value image is obtained by reconstructing the image to obtain Loss_image, which is the cross entropy loss of the two classifications.

[0009] In some embodiments, in order to learn the parameters in each module, the total loss function of the network is Loss = Loss_mask + * Loss_image, where is the weight coefficient.

[0010] In some embodiments, the convolution module 1 is composed of a 3x3 convolution layer with 1 convolution kernel, and the activation function is Sigmoid.

[0011] In some embodiments, the convolution module 2 is composed of a 3x3 convolution layer with 1 convolution kernel, and the activation function is Sigmoid.

[0012] In some implementations, the number of convolutional layers and upsampling layers of the convolution module 3 is determined by an encoder.

[0013] According to the technical solution, the present invention can achieve the following beneficial effects: When detecting product defects, the present invention reconstructs images through an autoencoder network. At the same time, the threshold is learned through a threshold learning network to automatically binarize the difference image, and then a binary approximate image is output, so that only a simple threshold needs to be set to binarize the binary approximate image into a standard binary image. The highlighted area in the binary image is the area corresponding to the defect, thereby realizing defect detection. Therefore, in the present industrial visual defect detection method, there is no need to carefully adjust the threshold through subjective experience to take into account the ability to detect various defects. Instead, only a threshold needs to be simply set to binarize the binary approximate image into a standard binary image, thereby improving the detection accuracy and reducing the defect missed detection rate and over-kill rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation of the present invention, the following is a brief description of the drawings and reference numerals required to be used in describing the specific implementation.

[0015] Figure 1 is a neural network structure diagram of the present invention; Figures 2 to 7 It is a schematic diagram of the effect of an application example of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in combination with specific implementation methods and with reference to the accompanying drawings.

[0017] like Figure 1 As shown, this specific embodiment provides an industrial visual defect detection method, which is applied to a neural network, wherein the neural network includes an autoencoder network and a threshold learning network; Wherein, the autoencoder network is mainly used to reconstruct an image. In addition, the autoencoder network includes an encoder and a decoder; Among them, the threshold learning network is used to learn the threshold to automatically binarize the difference image, and then output the binary approximate image. By setting a simple threshold, the binary approximate image is binarized into a standard binary image. The highlighted area in the binary image is the area corresponding to the defect, thereby realizing defect detection. In addition, the threshold learning network includes convolution module 1, convolution module 2, convolution module 3, convolution module 4 and a differentiable binarization module.

[0018] The industrial visual defect detection method comprises the following steps: First, input the image, collect the grayscale image or color image of the product by the industrial camera, pre-process and normalize it to between 0 and 1, and input it to the encoder and convolution module 2; Encoder, which consists of several convolutional layers. The number of convolutional layers can be specified according to the difficulty of the specific problem. The encoder extracts features from the input image and inputs them to the decoder and convolution module 3; Decoder, the decoder consists of several convolutional layers and upsampling layers. The number of convolutional layers can be specified according to the difficulty of the specific problem. The decoder reconstructs the image using the features extracted by the encoder and transmits it to the convolution module 1; Convolution module 1, consisting of a 3x3 convolution layer with 1 convolution kernel and a sigmoid activation function, is used to extract the reconstructed image feature F1 from the reconstructed image and shares weights with convolution module 2; Convolution module 2, which consists of a 3x3 convolution layer with 1 convolution kernel and a sigmoid activation function, is used to extract input image features F2 from the input image and shares weights with convolution module 1; Difference image: reconstruct the image feature F1 minus the input image feature F2, and take the absolute value of the subtracted feature to get the difference image; Convolution module 3, which consists of several convolutional layers and upsampling layers. The number of layers is determined by the encoder. It forms a U-shaped structure with the encoder and is used to extract the features of the input image for subsequent learning of the threshold. The resolution of the extracted features should be one quarter of the resolution of the input image. Convolution module 4, which includes two upsampling layers and a 1x1 convolution layer with 1 convolution kernel, wherein the two upsampling layers are used to upsample the feature resolution extracted by convolution module 3 to the same resolution as the input image, and the convolution layer is used to learn and predict the threshold to obtain a threshold map. Therefore, the threshold map is composed of a series of thresholds learned by the network, and the resolution of the threshold map is the same as the resolution of the difference image; Differentiable binarization module, which is a differentiable binarization function, such as , in, P refers to the difference image, T refers to the threshold map, [ i, j ] refers to the coordinate index, [ Pi,j ] refers to the difference image with coordinate index [ i, j ] eigenvalue at [ Ti,j ] refers to the threshold map where the coordinate index is [ i, j ], k is the amplification factor, which is used to control the degree of change of the function. B is a binary approximate image; The binary approximate image is a single-channel image predicted by the network. The pixel values ​​are composed of numbers between 0 and 1. By setting a simple threshold, the binary approximate image can be binarized into a binary image composed of 0 and 1. The location information of the defect can be calculated from the binary image through some image processing methods. The true value mask image is the learning label of the threshold learning network. It is used to supervise the learning of the threshold learning network. It is a standard binary mask image composed of 0 and 1. It is the mask of the input image, where 0 represents the background area and 1 represents the defect area. The true value mask image is obtained by the binary approximation image to obtain Loss_mask. Loss_mask is the cross entropy loss of the binary classification. In addition, the true value mask image is only needed during network learning (training). It is not needed for defect detection after learning. The true value image is the learning label of the autoencoder network. It is used to supervise the learning of the autoencoder network. The true value image is exactly the same as the input image. The true value image is obtained by reconstructing the image to obtain Loss_image. Loss_image is the cross entropy loss of the two classifications. In addition, the true value image is only needed when the network is learning (training). It is not needed for defect detection after the learning is completed.

[0019] In order to learn the parameters in each module, the total loss function of the network is: Loss = Loss_mask + * Loss_image, where is the weight coefficient.

[0020] In summary, the industrial visual defect detection method described in this specific embodiment realizes product defect (flaw) detection by using a neural network to automatically learn thresholds, thereby significantly improving detection accuracy and reducing defect missed detection rate and over-kill rate; in addition, since the industrial visual defect detection method has automatically learned a binary approximate image, only a simple threshold is needed to binarize the binary approximate image into a standard binary image, thereby realizing defect detection.

[0021] In addition, in order to further illustrate the industrial visual defect detection method, the following application examples are listed.

[0022] S1, image acquisition and preprocessing, which uses high-precision image acquisition equipment to obtain the appearance image of the product, and performs filtering and enhancement preprocessing operations on the appearance image to enhance image quality and reduce noise interference; Collect the cropped small picture of the original picture, such as Figure 2 shown.

[0023] S2, key edge point detection, which combines the gradient information and local features of the image, sets a threshold according to the strength and gradient amplitude attributes of the edge points, retains edge points with larger strength or gradient amplitude, and accurately detects key edge points with significant features; The image after edge detection is as follows Figure 3 shown.

[0024] S3, Blob analysis, which performs binarization on the image containing key edge points, identifies different blobs through the connected component labeling algorithm, and calculates the features of each blob; Threshold segmentation, such as Figure 4 As shown; The connected areas are divided into different blobs, such as Figure 5 shown.

[0025] S4, screening and determination of key contour points, which sets rules to screen potential contour points according to the area or perimeter characteristics of the Blob; First calculate the center point coordinates of each blob. Take the defect as an example. The center point coordinates of the defect are (18.55, 164.09). Calculate the distance from the defect contour point to the center point, the area of ​​the defect, set the threshold, find the edge key points that meet the threshold conditions, retain them, and output the edge key point set of the defect{} Filter results such as Figure 6 shown.

[0026] S5, post-processing and optimization, which further smoothes, denoises and optimizes the extracted key contour points to improve their accuracy and stability.

[0027] Post-processing, smoothed defects, such as Figure 7 shown.

Claims

1. An industrial visual defect detection method, characterized in that: It is applied to a neural network, which includes an autoencoder network and a threshold learning network; Wherein, the autoencoder network is mainly used to reconstruct an image. In addition, the autoencoder network includes an encoder and a decoder; The threshold learning network is used to learn the threshold to automatically binarize the difference image, and then output the binary approximate image. The binary approximate image is binarized into a standard binary image by setting a simple threshold. The highlight area in the binary image is the area corresponding to the defect, thereby realizing defect detection. In addition, the threshold learning network includes convolution module 1, convolution module 2, convolution module 3, convolution module 4 and a differentiable binarization module; The industrial visual defect detection method comprises the following steps: First, input the image, collect the grayscale image or color image of the product by the industrial camera, pre-process and normalize it to between 0 and 1, and input it to the encoder and convolution module 2; The encoder extracts features from the input image through several convolutional layers and inputs them to the decoder and convolution module 3; The decoder reconstructs the image from the extracted features through several convolutional layers and upsampling layers and transmits it to the convolution module 1; Convolution module 1, which is used to extract reconstructed image features F1 from the reconstructed image, and shares weights with convolution module 2; Convolution module 2, used to extract input image feature F2 from the input image, which shares weights with convolution module 1; Difference image: reconstruct the image feature F1 minus the input image feature F2, and take the absolute value of the subtracted feature to get the difference image; Convolution module 3, which consists of several convolutional layers and upsampling layers, and forms a U-shaped structure with the encoder, which is used to extract the features of the input image; Convolution module 4, which includes two upsampling layers and a 1x1 convolution layer with 1 convolution kernel, wherein the two upsampling layers are used to upsample the feature resolution extracted by convolution module 3 to the same resolution as the input image, and the convolution layer is used to learn and predict the threshold to obtain a threshold map, and the resolution of the threshold map is the same as the resolution of the difference image; Differentiable binarization module, which is a differentiable binarization function, such as ， in, P refers to the difference image, T refers to the threshold map, [ i, j ] refers to the coordinate index, [ Pi,j ] refers to the difference image with coordinate index [ i, j ] eigenvalue at [ Ti,j ] refers to the threshold map where the coordinate index is [ i, j ], k is the amplification factor, which is used to control the degree of change of the function. B is a binary approximate image; The binary approximate image is a single-channel image finally predicted by the network, in which the pixel values ​​are composed of numbers between 0 and 1. By setting a simple threshold, the binary approximate image can be binarized into a binary image composed of 0 and 1. The location information of the defect can be calculated from the binary image through some image processing methods.

2. The industrial visual defect detection method according to claim 1, characterized in that: The following steps are also included: The true value mask image is the learning label of the threshold learning network. It is used to supervise the learning of the threshold learning network. It is a standard binary mask image composed of 0 and 1. It is the mask of the input image, where 0 represents the background area and 1 represents the defect area. The true value mask image is obtained by the binary approximation image to obtain Loss_mask, which is the cross entropy loss of the two classifications.

3. The industrial visual defect detection method according to claim 2, characterized in that: The following steps are also included: The true value image is the learning label of the autoencoder network, which is used to supervise the learning of the autoencoder network. The true value image is exactly the same as the input image. The true value image is obtained by reconstructing the image to obtain Loss_image, which is the cross entropy loss of the two classifications.

4. The industrial visual defect detection method according to claim 3, characterized in that: In order to learn the parameters in each module, the total loss function of the network is Loss = Loss_mask + * Loss_image, where is the weight coefficient.

5. The industrial visual defect detection method according to claim 1, characterized in that: The convolution module 1 consists of a 3x3 convolution layer with 1 convolution kernel, and the activation function is Sigmoid.

6. The industrial visual defect detection method according to claim 5, characterized in that: Convolution module 2 consists of a 3x3 convolution layer with 1 convolution kernel and the activation function is Sigmoid.

7. The industrial visual defect detection method according to claim 1, characterized in that: The number of convolutional layers and upsampling layers of the convolution module 3 is determined by the encoder.

Images