Unsupervised industrial product defect detection method and device based on deep learning

Abstract

This invention discloses an unsupervised industrial product defect detection method and apparatus based on deep learning, as well as a computer-readable storage medium. The method includes the following steps: training a first autoencoder using training samples and obtaining the latent representation of the training samples during the training process; performing dimensionality reduction on the latent representation of the training samples to obtain the latent representation of normal samples; initializing the memory module in a second autoencoder using the latent representation of normal samples, and training the second autoencoder using the training samples; inputting test samples into the trained second autoencoder to obtain reconstructed samples; calculating a defect mask using the test samples and the reconstructed samples, and determining whether the test samples have defects based on the defect mask. This invention achieves effective defect detection using only defect-free image samples for model training, thus improving the defect detection performance.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to an unsupervised industrial product defect detection method and device based on deep learning, and a computer-readable storage medium. Background Art

[0002] In the production process of industrial products, due to the limitations of process level and the influence of environmental factors, defective products will inevitably be produced. If these products cannot be detected at the source of defects as early as possible, they will flow into the subsequent production steps, making subsequent detection more difficult and increasing the cost of repair. What is more serious is that if these products are accidentally sold on the market, it will have a great impact on the product image. In order to detect these defective products, traditional technology adopts the method of deploying AOI (Automatic Optic Inspection) equipment on the production line for automatic defect detection. However, due to the limitations of its own system design, AOI equipment often has a large number of false alarms (i.e., judging non-defective products as defective products). Therefore, technical workers are required to conduct re-inspections, which greatly reduces the efficiency of detection.

[0003] In recent years, with the development of deep learning, convolutional neural networks, as an end-to-end model that can automatically extract image features, have been increasingly studied and applied. In order to detect whether there are defects in industrial product images, traditional deep learning-based defect detection methods usually use supervised models (i.e., collect a large number of defective and non-defective image samples for training). However, in practical applications, due to the high cost of obtaining defective samples and the constant changes in defect types, it is usually difficult to collect enough defective image samples, while non-defective image samples account for the majority. This will lead to insufficient model training or inability to cope with defect types that have never appeared in the training set during detection, greatly weakening the defect detection effect of the model. Summary of the invention

[0004] The embodiments of the present application provide a method and device for unsupervised industrial product defect detection based on deep learning, and a computer-readable storage medium, thereby solving the problem that traditional technologies have poor defect detection effects due to the difficulty in obtaining defective image samples. This method achieves effective defect detection when only defect-free image samples are used to train the model, thereby improving the effect of defect detection.

[0005] The present application embodiment provides an unsupervised industrial product defect detection method based on deep learning, the method comprising:

[0006] Using training samples to train the first autoencoder, and obtaining implicit expressions of the training samples during the training process; wherein the training samples are defect-free images;

[0007] Performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample;

[0008] Initializing a memory module in a second autoencoder using the implicit expression of the normal sample, and training the second autoencoder using the training sample;

[0009] Input the test sample into the trained second autoencoder to obtain a reconstructed sample;

[0010] The test sample and the reconstructed sample are used to calculate and obtain a defect mask, and whether the test sample has a defect is determined according to the defect mask.

[0011] In one embodiment, the method further comprises:

[0012] Acquire a preset number of defect-free images;

[0013] The defect-free images are scaled to the same size and normalized to obtain training samples.

[0014] In one embodiment, the step of training the first autoencoder using the training sample and obtaining the implicit expression of the training sample during the training process includes:

[0015] Inputting the training samples one by one into the first autoencoder to train the first autoencoder;

[0016] After the first autoencoder is trained, a feature vector output by the encoder module of the first autoencoder of the training sample is obtained, and the feature vector is used as an implicit expression of the training sample.

[0017] In one embodiment, the step of performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample includes:

[0018] The implicit expression of the training sample is subjected to dimensionality reduction processing using principal component analysis to obtain the implicit expression after dimensionality reduction, and the implicit expression is used as the implicit expression of the normal sample.

[0019] In one embodiment, the step of inputting the test sample into the trained second autoencoder to obtain the reconstructed sample includes:

[0020] Inputting the test sample into the trained second autoencoder to obtain a reconstructed image, and calculating the mean square error between the test sample and the reconstructed image;

[0021] If the mean square error between the test sample and the reconstructed image is less than or equal to a preset threshold, the reconstructed image is used as the reconstructed sample.

[0022] In one embodiment, the step of inputting the test sample into the trained second autoencoder to obtain a reconstructed sample further includes:

[0023] If the mean square error between the test sample and the reconstructed image is greater than a preset threshold, the reconstructed image is re-input into the trained second autoencoder.

[0024] In one embodiment, the step of using the test sample and the reconstructed sample to calculate and obtain a defect mask, and judging whether the test sample has a defect according to the defect mask includes:

[0025] Calculating a structure-similarity-weighted pixel-to-pixel error using the test sample and the reconstructed sample, and using the pixel-to-pixel error as a defect mask;

[0026] The sum of all pixel values ​​of the defect mask is used as a defect index, and whether the test sample has a defect is determined according to the defect index.

[0027] In one embodiment, the step of judging whether the test sample has defects according to the defect index includes:

[0028] If the defect index is greater than a preset threshold, it is determined that the test sample has a defect;

[0029] If the defect index is less than or equal to a preset threshold, it is determined that the test sample has no defects.

[0030] An embodiment of the present application also provides a device, which includes a processor, a memory, and a defect detection program stored in the memory and executable on the processor. When the defect detection program is executed by the processor, the steps of the unsupervised industrial product defect detection method based on deep learning as described above are implemented.

[0031] An embodiment of the present application also provides a computer-readable storage medium, on which a defect detection program is stored. When the defect detection program is executed by a processor, the steps of the unsupervised industrial product defect detection method based on deep learning as described above are implemented.

[0032] The technical solutions of a method and device for unsupervised industrial product defect detection based on deep learning and a computer-readable storage medium provided in the embodiments of the present application have at least the following technical effects:

[0033] The invention adopts the method of using training samples to train the first autoencoder, and obtaining the implicit expression of the training samples during the training process; wherein the training samples are defect-free images; performing dimensionality reduction processing on the implicit expression of the training samples to obtain the implicit expression of normal samples; using the implicit expression of the normal samples to initialize the memory module in the second autoencoder, and using the training samples to train the second autoencoder; inputting the test samples into the trained second autoencoder to obtain the reconstructed samples; using the test samples and the reconstructed samples to calculate the defect mask, and judging whether the test sample has defects according to the defect mask. Therefore, the problem that the traditional technology has poor defect detection effect due to the difficulty in obtaining defective image samples is effectively solved, and the effective detection of defects is achieved when only defect-free image samples are used to train the model, thereby improving the effect of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the structure of the device involved in the embodiment of the present application;

[0035] Figure 2 This is a flowchart of the first embodiment of the unsupervised industrial product defect detection method based on deep learning of the present application;

[0036] Figure 3 This is a flow chart of a second embodiment of the unsupervised industrial product defect detection method based on deep learning of the present application;

[0037] Figure 4 This is a flow chart of the third embodiment of the unsupervised industrial product defect detection method based on deep learning of the present application. DETAILED DESCRIPTION

[0038] In order to solve the problem that the traditional technology has a poor defect detection effect due to the difficulty in obtaining defective image samples, the present application adopts a technical solution of using training samples to train the first autoencoder and obtaining the implicit expression of the training samples during the training process; wherein the training samples are defect-free images; the implicit expression of the training samples is subjected to dimensionality reduction processing to obtain the implicit expression of normal samples; the memory module in the second autoencoder is initialized using the implicit expression of the normal samples, and the second autoencoder is trained using the training samples; the test samples are input into the trained second autoencoder to obtain the reconstructed samples; the defect mask is calculated using the test samples and the reconstructed samples, and the test sample is judged based on the defect mask whether there is a defect. The effective detection of defects is achieved when only defect-free image samples are used to train the model, thereby improving the effect of defect detection.

[0039] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0040] Combination Figure 1 As shown, it is a schematic diagram of a hardware structure of the device involved in each embodiment of the present application, and the device may include: a processor 101, a memory 102, an input unit 103, an output unit 104 and other components. Those skilled in the art can understand that Figure 1 The hardware structure of the device shown does not constitute a limitation of the device, and the device may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0041] Combine the following Figure 1 The various components of the device are specifically introduced:

[0042] The processor 101 is the control center of the device, connecting the various parts of the entire device, and performing various functions of the device or processing data by running or executing programs stored in the memory 102 and calling data stored in the memory 102, thereby monitoring the device as a whole. In addition, the processor 101 includes at least a graphics processor GPU.

[0043] The memory 102 can be used to store various programs and various data of the device. The memory 102 mainly includes a program storage area and a data storage area, wherein the program storage area at least stores the program required for defect detection; the data storage area can store various data of the device. In addition, the memory 102 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0044] The input unit 103 may be used to input data required for autoencoder training and data required for defect detection from outside the device.

[0045] The output unit 104 may be configured to output the defect detection result of the data that needs to be defect detected.

[0046] In the embodiment of the present application, the processor 101 may be used to call the defect detection program stored in the memory 102 and perform the following operations:

[0047] Using training samples to train the first autoencoder, and obtaining implicit expressions of the training samples during the training process; wherein the training samples are defect-free images;

[0048] Performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample;

[0049] Initializing a memory module in a second autoencoder using the implicit expression of the normal sample, and training the second autoencoder using the training sample;

[0050] Input the test sample into the trained second autoencoder to obtain a reconstructed sample;

[0051] The test sample and the reconstructed sample are used to calculate and obtain a defect mask, and whether the test sample has a defect is determined according to the defect mask.

[0052] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0053] Acquire a preset number of defect-free images;

[0054] The defect-free images are scaled to the same size and normalized to obtain training samples.

[0055] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0056] Inputting the training samples one by one into the first autoencoder to train the first autoencoder;

[0057] After the first autoencoder is trained, a feature vector output by the encoder module of the first autoencoder of the training sample is obtained, and the feature vector is used as an implicit expression of the training sample.

[0058] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0059] The implicit expression of the training sample is subjected to dimensionality reduction processing using principal component analysis to obtain the implicit expression after dimensionality reduction, and the implicit expression is used as the implicit expression of the normal sample.

[0060] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0061] Inputting the test sample into the trained second autoencoder to obtain a reconstructed image, and calculating the mean square error between the test sample and the reconstructed image;

[0062] If the mean square error between the test sample and the reconstructed image is less than or equal to a preset threshold, the reconstructed image is used as the reconstructed sample.

[0063] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0064] If the mean square error between the test sample and the reconstructed image is greater than a preset threshold, the reconstructed image is re-input into the trained second autoencoder.

[0065] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0066] Calculating a structure-similarity-weighted pixel-to-pixel error using the test sample and the reconstructed sample, and using the pixel-to-pixel error as a defect mask;

[0067] The sum of all pixel values ​​of the defect mask is used as a defect index, and whether the test sample has a defect is determined according to the defect index.

[0068] In one embodiment, the processor 101 may be used to call a defect detection program stored in the memory 102 and perform the following operations:

[0069] If the defect index is greater than a preset threshold, it is determined that the test sample has a defect;

[0070] If the defect index is less than or equal to a preset threshold, it is determined that the test sample has no defects.

[0071] According to the above technical solution, this embodiment uses the following method: use training samples to train the first autoencoder, and obtain the implicit expression of the training samples during the training process; wherein the training samples are defect-free images; perform dimensionality reduction processing on the implicit expression of the training samples to obtain the implicit expression of normal samples; use the implicit expression of the normal samples to initialize the memory module in the second autoencoder, and use the training samples to train the second autoencoder; input the test sample into the trained second autoencoder to obtain the reconstructed sample; use the test sample and the reconstructed sample to calculate the defect mask, and use the defect mask to determine whether the test sample has defects. Therefore, the problem that the traditional technology has poor defect detection effect due to the difficulty in obtaining defective image samples is effectively solved, and effective detection of defects is achieved when only defect-free image samples are used to train the model, thereby improving the effect of defect detection.

[0072] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0073] Combination Figure 2 As shown, in the first embodiment of the present application, the unsupervised industrial product defect detection method based on deep learning of the present application specifically includes the following steps:

[0074] Step S110, using the training samples to train the first autoencoder, and obtaining the implicit expression of the training samples during the training process.

[0075] In this embodiment, the first autoencoder includes an encoder module and a decoder module. The encoder and the decoder are both convolutional neural networks. The training sample is a defect-free image. Assuming that the encoder is E, the decoder is D, and the training sample is x, then after the training sample is input into the first autoencoder, the output image can be obtained as The first autoencoder can be trained by optimizing the loss function of the first autoencoder. Wherein E(x) is the feature vector of the training sample x output by the encoder E, that is, the implicit expression of the training sample x. After all the training samples are used to train the first autoencoder, the implicit expression of all samples can be obtained during the training process.

[0076] Step S120, performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample.

[0077] In this embodiment, assuming that the number of the training samples is N, N implicit expressions of the N training samples can be obtained in step S110, that is, N-dimensional implicit expressions. However, in order to make the data easier to process and use and remove data noise, it is also necessary to perform dimensionality reduction processing on the N-dimensional implicit expressions. After the dimensionality reduction processing on the N-dimensional implicit expressions, low-dimensional implicit expressions can be obtained, that is, the implicit expressions of normal samples.

[0078] Step S130: Initialize the memory module in the second autoencoder using the implicit expression of the normal sample, and train the second autoencoder using the training sample.

[0079] In this embodiment, the second autoencoder includes an encoder module, a memory module and a decoder module. Assuming that the implicit expression of the normal sample obtained in step S120 is M-dimensional, that is, the implicit expression of M normal samples is obtained, the memory module in the second autoencoder is initialized using the implicit expression of the M normal samples. After the memory module in the second autoencoder is initialized using the implicit expression of the normal sample, the second autoencoder can be trained using the training sample.

[0080] In one embodiment, assuming that the image size of the training sample is W×H×3, the dimension of the implicit expression is D l , then the network structure of the second autoencoder can be shown in the following table. Wherein, the format of the convolution layer (Conv) and the deconvolution layer (Deconv) is out_channels×kernel_size×stride×padding.

[0081]

[0082]

[0083] Assuming that the encoder is E, the decoder is D, and the training sample is x, the step of training the second autoencoder using the training sample may include:

[0084] Step a: forward propagation of the second autoencoder. Input the training sample into the second autoencoder, and obtain the implicit expression of the training sample as z=E(x) through the encoder forward propagation. Calculate the implicit expression z and the implicit expression m of each normal sample in the memory module. i The cosine similarity of Then use the softmax function to normalize it and get the weight of the implicit expression of each normal sample in the memory module In order to reconstruct the training samples using a small number of most similar normal samples, wi Sort them from large to small, and take the value of their q quantile (0<q<1, the smaller q is, the fewer normal samples are used) as the threshold λ for truncation, that is, Then the implicit expression used by the decoder is The final reconstructed image is

[0085] Step b, back-propagating the second autoencoder. The loss function of the second autoencoder is The first term is the reconstruction error, and the second term is The entropy of The parameters in the second autoencoder are optimized by optimizing the loss function.

[0086] During the training process, the memory module in the second autoencoder also needs to be updated in real time, and the parameters of the memory module are fixed after the training is completed.

[0087] Step S140: input the test sample into the trained second autoencoder to obtain a reconstructed sample.

[0088] In this embodiment, the test sample is a sample that may have defects, that is, it may be a non-defective image or a defective image. Given a test sample, after the test sample is input into the trained second autoencoder, the trained second autoencoder can be used to reconstruct the test sample through an iterative optimization method to obtain a reconstructed sample corresponding to the test sample, that is, the corresponding non-defective sample. By adopting the iterative optimization method, the reconstructed sample can only contain normal samples.

[0089] Step S150 , using the test sample and the reconstructed sample to calculate and obtain a defect mask, and judging whether the test sample has a defect according to the defect mask.

[0090] In this embodiment, the defect mask refers to the segmentation result of defects in the image, and different areas are represented by different pixel values. For example, the pixel value of the defective position can be 255, and the pixel value of the non-defective position can be 0. The defect mask of the test sample can be calculated by using the test sample and the reconstructed sample. In one embodiment, the defect mask can be obtained by calculating the pixel-to-pixel error weighted by the structural similarity of the test sample and the reconstructed sample. After obtaining the defect mask of the test sample, the defect mask of the test sample can be analyzed to determine whether the test sample has defects. In one embodiment, it can be determined whether the test sample has defects by calculating the sum of the pixel values ​​of the defect mask.

[0091] The beneficial effect of the above method is that the first autoencoder is trained using training samples, and the implicit expression of the training samples is obtained during the training process; wherein the training samples are defect-free images; the implicit expression of the training samples is subjected to dimensionality reduction processing to obtain the implicit expression of normal samples; the memory module in the second autoencoder is initialized using the implicit expression of the normal samples, and the second autoencoder is trained using the training samples; the test samples are input into the trained second autoencoder to obtain the reconstructed samples; the defect mask is calculated using the test samples and the reconstructed samples, and the technical solution of judging whether the test sample has defects based on the defect mask is adopted. Therefore, the problem that the traditional technology has poor defect detection effect due to the difficulty in obtaining defective image samples is effectively solved, and effective detection of defects is achieved when only defect-free image samples are used to train the model, thereby improving the effect of defect detection.

[0092] Combination Figure 3 As shown, in the second embodiment of the present application, the unsupervised industrial product defect detection method based on deep learning of the present application specifically includes the following steps:

[0093] Step S211, obtaining a preset number of defect-free images.

[0094] In this embodiment, in order to fully train the autoencoder, a sufficient number of training samples need to be obtained. Since the training samples used to train the autoencoder in this application are defect-free images, a sufficient number of defect-free images need to be obtained as training samples. Accordingly, the preset number needs to be set to a value that matches the number of defect-free images required.

[0095] Step S212: scaling the defect-free images to the same size and performing normalization processing on them to obtain training samples.

[0096] In this embodiment, after a sufficient number of defect-free images are obtained, the defect-free images need to be preprocessed. First, the defect-free images need to be scaled to the same size so that the size of the images input to the autoencoder can be unified. After scaling the defect-free images, they need to be further normalized, that is, the data on each pixel of the defect-free image is mapped to a floating point number between 0 and 1. By normalizing the defect-free images, the data processing in the autoencoder training process can be simplified. After processing the defect-free images, the defect-free images can be used as training samples for autoencoder training.

[0097] Step S221: input the training samples one by one into the first autoencoder to train the first autoencoder.

[0098] In this embodiment, it is assumed that the image size of the training sample is W×H×3, and the dimension of the implicit expression is D l , then the network structure of the first autoencoder can be shown in the following table. Among them, the format of the convolution layer (Conv) and the deconvolution layer (Deconv) is out_channels×kernel_size×stride×padding.

[0099]

[0100]

[0101] Assuming that the encoder is E, the decoder is D, and the training sample is x, after the training sample is input into the first autoencoder, the output image can be obtained as In order to make the first autoencoder learn a purer normal sample, the output image Afterwards, the output image can be used as the input of the first autoencoder to forward propagate again to obtain the output image The loss function is The first autoencoder is trained by optimizing the loss function. After all training samples are input into the first autoencoder one by one to train the first autoencoder, the training of the first autoencoder can be completed.

[0102] Step S222: After the first autoencoder is trained, a feature vector output by the encoder module of the first autoencoder of the training sample is obtained, and the feature vector is used as an implicit expression of the training sample.

[0103] In this embodiment, assuming that the training sample is x, after the first autoencoder is trained, the feature vector z=E(x) output by the encoder module of the training sample x through the first autoencoder can be obtained, which is the implicit expression of the training sample x. Assuming that the dimension of the implicit expression is D l , the number of training samples is N, then the implicit expression matrix composed of the implicit expressions of all training samples can be finally obtained, and the dimension of the implicit expression matrix is ​​N×D l .

[0104] Step S231, using principal component analysis to perform dimensionality reduction processing on the implicit expression of the training sample, to obtain the implicit expression after dimensionality reduction, and use it as the implicit expression of the normal sample.

[0105] In this embodiment, after the implicit expressions of all training samples are obtained and the implicit expression matrix is ​​obtained, the implicit expression matrix can be transposed to change its dimension to D l ×N. Assuming that the number of normal samples finally obtained is M, after using principal component analysis to reduce the dimension of the transposed implicit expression matrix, a dimension of D can be obtained. l ×M implicit expression matrix. Then again for the dimension D l ×M implicit expression matrix is ​​transposed to obtain a dimension of M×D l The implicit expression matrix of is the implicit expression matrix of normal samples.

[0106] Step S240: Initialize the memory module in the second autoencoder using the implicit expression of the normal sample, and train the second autoencoder using the training sample.

[0107] Step S250: input the test sample into the trained second autoencoder to obtain a reconstructed sample.

[0108] Step S260: Use the test sample and the reconstructed sample to calculate and obtain a defect mask, and determine whether the test sample has a defect according to the defect mask.

[0109] The beneficial effect of the above method is that the step of obtaining the trained second autoencoder is supplemented and refined on the basis of the first embodiment. Therefore, the problem that the traditional technology has poor defect detection effect due to the difficulty in obtaining defective image samples is further effectively solved, and effective defect detection is achieved when only defect-free image samples are used to train the model, thereby improving the effect of defect detection.

[0110] Combination Figure 4 As shown, in the third embodiment of the present application, the unsupervised industrial product defect detection method based on deep learning of the present application specifically includes the following steps:

[0111] Step S310: Use the training samples to train the first autoencoder, and obtain the implicit expression of the training samples during the training process.

[0112] Step S320, performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample.

[0113] Step S330: Initialize the memory module in the second autoencoder using the implicit expression of the normal sample, and train the second autoencoder using the training sample.

[0114] Step S341: input the test sample into the trained second autoencoder to obtain a reconstructed image, and calculate the mean square error between the test sample and the reconstructed image.

[0115] In this embodiment, given a test sample, assuming that the test sample is x, after the test sample x is input into the trained second autoencoder, the reconstructed image can be obtained as However, after obtaining the reconstructed image, it is still necessary to determine whether the reconstructed image contains only normal samples. In this case, the mean square error between the test sample and the reconstructed image can be calculated first, and then the calculated mean square error can be used to determine whether the reconstructed image contains only normal samples. The mean square error can reflect the degree of difference between the test sample and the reconstructed image.

[0116] Step S342: If the mean square error between the test sample and the reconstructed image is less than or equal to a preset threshold, the reconstructed image is used as the reconstructed sample.

[0117] In this embodiment, the preset threshold is a value corresponding to the mean square error that can be pre-set according to actual needs and can be used to determine that the difference between the test sample and the reconstructed image is small. If the mean square error between the test sample and the reconstructed image is less than or equal to the preset threshold, it means that the difference between the test sample and the reconstructed image is small, and it can be determined that the reconstructed image only contains normal samples. Therefore, there is no need to continue to reconstruct the reconstructed image, and the reconstructed image can be directly used as the reconstructed sample.

[0118] Step S343: if the mean square error between the test sample and the reconstructed image is greater than a preset threshold, the reconstructed image is re-input into the trained second autoencoder.

[0119] In this embodiment, if the mean square error between the test sample and the reconstructed image is greater than a preset threshold, it means that the difference between the test sample and the reconstructed image is still large. At this time, it cannot be determined that the reconstructed image only contains normal samples. Therefore, it is necessary to continue to reconstruct the reconstructed image, that is, to re-input the reconstructed image into the trained second autoencoder, and then continue to calculate the mean square error between the input image and the reconstructed image, so as to determine whether the reconstructed image can be used as a reconstructed sample.

[0120] Step S351 : Calculate the structure similarity weighted pixel-to-pixel error using the test sample and the reconstructed sample, and use the pixel-to-pixel error as a defect mask.

[0121] In this embodiment, in order to eliminate the influence of small positioning errors in the reconstruction process and improve the accuracy of the defect mask image, the present application uses the pixel error weighted by structural similarity as the result of the defect mask. Assume that the test sample is x and the reconstructed sample is The calculation method of structural similarity is Among them, μ x and are x and The average value, δ x and are x and The variance of , c1 and c2 are constants used to maintain stability.

[0122] The calculation method for the pixel error at position (i, j) is Where C is the number of channels of the image. In summary, the calculation method of the pixel-to-pixel error weighted by structural similarity can be obtained as Among them, α is the coefficient that controls the proportion of inter-pixel error in the defect mask. The larger α is, the greater the weight of inter-pixel error is. Since the structural similarity is calculated for the image, the structural similarity index of the pixel point at position (i, j) is defined as the square small image with a side length of l pixels centered on this pixel point (i.e. x′ ij and ). When this pixel is located at the edge of the image, the part beyond the image is filled with 0 pixel value.

[0123] After calculation, the defect mask of the test sample can be obtained.

[0124] Step S352: taking the sum of all pixel values ​​of the defect mask as a defect index, and judging whether the test sample has defects according to the defect index.

[0125] In this embodiment, according to the defect mask K obtained in step S351, it can be determined whether the test sample x has defects. First, all pixel values ​​in the defect mask K are summed to obtain the defect index I of the defect mask = ∑ i ∑ j M ij, and then judging whether the test sample has defects according to the defect index. The defect index identifies the possibility that the test sample has defects. The larger the defect index, the greater the possibility that the test sample has defects; the smaller the defect index, the smaller the possibility that the test sample has defects. In one embodiment, the step of judging whether the test sample has defects according to the defect index may include: if the defect index is greater than a preset threshold, judging that the test sample has defects; if the defect index is less than or equal to the preset threshold, judging that the test sample does not have defects. The preset threshold is a value corresponding to the defect index that is pre-set according to actual conditions and can be used to judge whether the test sample has defects.

[0126] The beneficial effect of the above method is that the steps of detecting the test sample are supplemented and refined on the basis of the first embodiment. Therefore, the problem that the traditional technology has a poor defect detection effect due to the difficulty in obtaining defective image samples is further effectively solved, and effective defect detection is achieved when only defect-free image samples are used to train the model, thereby improving the effect of defect detection.

[0127] Based on the same inventive concept, an embodiment of the present application also provides a device, which includes a processor, a memory, and a defect detection program stored in the memory and run on the processor. When the defect detection program is executed by the processor, it implements the various processes of the above-mentioned unsupervised industrial product defect detection method embodiment based on deep learning, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0128] Since the device provided in the embodiment of the present application is a device used to implement the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the device, so it is not repeated here. All devices used in the method of the embodiment of the present application belong to the scope of protection of this application.

[0129] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, on which a defect detection program is stored. When the defect detection program is executed by a processor, the various processes of the above-mentioned unsupervised industrial product defect detection method embodiment based on deep learning are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0130] Since the computer-readable storage medium provided in the embodiments of the present application is a computer-readable storage medium used to implement the method of the embodiments of the present application, based on the method introduced in the embodiments of the present application, those skilled in the art can understand the specific structure and deformation of the computer-readable storage medium, so it is not repeated here. All computer-readable storage media used in the methods of the embodiments of the present application belong to the scope of protection of this application.

[0131] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0136] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An unsupervised industrial product defect detection method based on deep learning, characterized in that: The method comprises: Using training samples to train the first autoencoder, and obtaining implicit expressions of the training samples during the training process; wherein the training samples are defect-free images; Performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample; Initializing a memory module in a second autoencoder using the implicit expression of the normal sample, and training the second autoencoder using the training sample; Input the test sample into the trained second autoencoder to obtain a reconstructed sample; Using the test sample and the reconstructed sample to calculate and obtain a defect mask, and judging whether the test sample has a defect according to the defect mask; The step of training the first autoencoder using the training sample and obtaining the implicit expression of the training sample during the training process includes: Inputting the training samples one by one into the first autoencoder to train the first autoencoder; After the first autoencoder is trained, a feature vector output by the encoder module of the first autoencoder is obtained for the training sample, and the feature vector is used as an implicit expression of the training sample; The step of performing dimensionality reduction processing on the implicit expression of the training sample to obtain the implicit expression of the normal sample includes: The implicit expression of the training sample is subjected to dimensionality reduction processing using principal component analysis to obtain the implicit expression after dimensionality reduction, and the implicit expression is used as the implicit expression of the normal sample.

2. The unsupervised industrial product defect detection method based on deep learning according to claim 1, characterized in that: The method further comprises: Acquire a preset number of defect-free images; The defect-free images are scaled to the same size and normalized to obtain training samples.

3. The unsupervised industrial product defect detection method based on deep learning according to claim 1, characterized in that: The step of inputting the test sample into the trained second autoencoder to obtain the reconstructed sample comprises: Inputting the test sample into the trained second autoencoder to obtain a reconstructed image, and calculating the mean square error between the test sample and the reconstructed image; If the mean square error between the test sample and the reconstructed image is less than or equal to a preset threshold, the reconstructed image is used as the reconstructed sample.

4. The unsupervised industrial product defect detection method based on deep learning according to claim 3, characterized in that: The step of inputting the test sample into the trained second autoencoder to obtain a reconstructed sample also includes: If the mean square error between the test sample and the reconstructed image is greater than a preset threshold, the reconstructed image is re-input into the trained second autoencoder.

5. The unsupervised industrial product defect detection method based on deep learning according to claim 1, characterized in that: The step of using the test sample and the reconstructed sample to calculate and obtain a defect mask, and judging whether the test sample has a defect according to the defect mask comprises: Calculating a structure-similarity-weighted pixel-to-pixel error using the test sample and the reconstructed sample, and using the pixel-to-pixel error as a defect mask; The sum of all pixel values ​​of the defect mask is used as a defect index, and whether the test sample has a defect is determined according to the defect index.

6. The unsupervised industrial product defect detection method based on deep learning according to claim 5, characterized in that: The step of judging whether the test sample has defects according to the defect index comprises: If the defect index is greater than a preset threshold, it is determined that the test sample has a defect; If the defect index is less than or equal to a preset threshold, it is determined that the test sample has no defects.

7. A device, characterized in that: The device includes a processor, a memory, and a defect detection program stored in the memory and executable on the processor. When the defect detection program is executed by the processor, the steps of the unsupervised industrial product defect detection method based on deep learning as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a defect detection program, and when the defect detection program is executed by the processor, the steps of the unsupervised industrial product defect detection method based on deep learning as described in any one of claims 1 to 6 are implemented.

Images