Silk screen defect detection method, device and storage medium based on unsupervised learning

Abstract

This invention relates to the field of screen printing defect detection technology. It discloses a screen printing defect detection method, apparatus, and storage medium based on unsupervised learning. The screen printing defect detection method based on unsupervised learning includes: acquiring an abnormal image and inputting the abnormal image into a reconstruction network to obtain a defect-free reconstructed image; comparing the difference between the abnormal image and the reconstructed image to obtain a residual image that initially locates the defect; and inputting the abnormal image, the reconstructed image, and the residual image into a discriminant network to obtain an anomaly score map that precisely locates the defect. This invention improves the accuracy of screen printing defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of screen printing defect detection, and in particular to a screen printing defect detection method, device and storage medium based on unsupervised learning. Background Art

[0002] With the rapid development of modern industrialization, people have higher and higher requirements for printing industry products. However, in the actual printing process, there are often many missing and wrong printing phenomena. Due to the low efficiency, high labor cost, fatigue and false detection of traditional manual inspection, visual inspection systems are now widely used to detect defects online.

[0003] With the continuous development of computer vision technology, defect recognition technology based on deep learning has become increasingly mature, effectively improving the recognition speed and accuracy of defects. Among them, supervised learning models are the most common in the detection of defect samples, and a large number of defect annotated samples are usually used to train the model. However, in the actual silk screen inspection industry, the defect samples have diverse shapes, and the image texture transformation is complex and irregular, which makes it difficult to detect defects. Summary of the invention

[0004] The main purpose of the present invention is to provide a silk screen defect detection method, device and storage medium based on unsupervised learning, aiming to solve the technical problem that it is difficult to realize defect detection under the conditions that the silk screen defect samples have diverse morphologies and complex image texture transformations.

[0005] To achieve the above object, the present invention provides a silk screen defect detection method based on unsupervised learning, and the silk screen defect detection method based on unsupervised learning comprises the following steps:

[0006] Acquire an abnormal image, and input the abnormal image into a reconstruction network to obtain a defect-free reconstructed image;

[0007] Comparing the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminary defect location;

[0008] The abnormal image, the reconstructed image and the residual image are input into the discriminant network to obtain an abnormal score map for accurately locating defects.

[0009] Optionally, the method further comprises generating a simulated abnormal image, and the step of generating a simulated abnormal image further comprises:

[0010] A noise image is randomly generated by a Perlin noise generator, and the noise image is binarized to obtain a simulated abnormal image mask and a reverse mask;

[0011] Sampling an abnormal texture source image from a preset source image data set, and performing data enhancement on the abnormal texture source image;

[0012] A normal sample image is acquired, and a simulated abnormal image is obtained based on the normal sample image, the simulated abnormal image mask and the inverse mask, and the abnormal texture source image after data enhancement.

[0013] Optionally, the step of performing data enhancement on the abnormal texture source image includes:

[0014] One or more methods are randomly selected from a preset set to enhance sampling of the abnormal texture source image, wherein the preset set at least includes brightness change, sharpness, exposure, color change and tone separation.

[0015] Optionally, the method further comprises generating a simulated abnormal image, and the step of generating a simulated abnormal image further comprises:

[0016] Selecting a source image from a preset source image data set, and randomly generating a patch based on the source image;

[0017] A normal sample image is obtained, and the patch is pasted onto the normal sample image to obtain a simulated abnormal image.

[0018] Optionally, it also includes:

[0019] Based on the simulated abnormal image, the reconstruction network is trained, the reconstruction error between the abnormal image and the reconstructed image is used as the reconstruction loss function, and the reconstruction network is trained until the reconstruction loss function tends to converge.

[0020] Optionally, the step of comparing the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminarily locating defects includes:

[0021] Using a regional analysis method to compare the difference between the abnormal image and the reconstructed image, and determine the defective image block where the defect is located;

[0022] The abnormal image and the reconstructed image are compared pixel by pixel, and the defect is preliminarily located in the defective image block to obtain a residual image.

[0023] Optionally, it also includes:

[0024] The abnormal image, the reconstructed image and the residual image are used to perform initial training on the discriminant network, and the total loss function of the reconstructed network and the discriminant network is calculated until the value of the total loss function is minimized.

[0025] Optionally, the reconstruction network includes an encoder and a decoder, the discriminant network includes a convolutional network, and the discriminant network shares the encoder of the reconstruction network to complete feature extraction.

[0026] In addition, to achieve the above-mentioned purpose, the present invention also provides a screen printing defect detection device based on unsupervised learning, characterized in that the screen printing defect detection device comprises:

[0027] A reconstruction module, used for acquiring an abnormal image and inputting the abnormal image into a reconstruction network to obtain a defect-free reconstructed image;

[0028] A preliminary positioning module, used to compare the difference between the abnormal image and the reconstructed image to obtain a residual image of the preliminary positioning defect;

[0029] The precise positioning module is used to input the abnormal image, the reconstructed image and the residual image into the discriminant network to obtain an abnormal score map for accurately positioning defects.

[0030] In addition, to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a silk screen defect detection program is stored. When the silk screen defect detection program is executed by a processor, the steps of the silk screen defect detection method based on unsupervised learning as described above are implemented.

[0031] The present invention obtains a defect-free reconstructed image by reconstructing the input image, and performs preliminary detection of the existing defective area by comparing the difference between the abnormal image and the reconstructed image. Finally, a discriminant network is used to obtain an abnormal score map for accurately locating defects, thereby achieving accurate detection of defective images. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of an embodiment of a method for detecting screen printing defects based on unsupervised learning according to the present invention;

[0033] Figure 2 It is a flow chart of an embodiment of a method for detecting screen printing defects of the present invention;

[0034] Figure 3 It is a schematic diagram of the flow chart of another embodiment of the method for detecting screen printing defects of the present invention;

[0035] Figure 4 A schematic diagram of a process for generating an abnormal image according to the present invention;

[0036] Figure 5 An example diagram of the generation of a simulated abnormal image according to the present invention;

[0037] Figure 6 The defect-free silk screen image of the present invention and the silk screen defect image obtained by using Perlin noise;

[0038] Figure 7 Another schematic diagram of a process for generating an abnormal image according to the present invention;

[0039] Figure 8The defect-free silk screen image of the present invention and the silk screen defect image obtained by Poisson fusion;

[0040] Fig. 9 A structural diagram of a network reconstructed for the present invention;

[0041] Fig.10 It is a structural diagram of ConvBlocks of the present invention;

[0042] Fig.11 Reconstruct the structure diagram of each layer of the network for the present invention;

[0043] Fig.12 The screen printing abnormal image and residual image of the present invention;

[0044] Fig.13 It is the screen printing abnormal image and abnormal score map of the present invention;

[0045] Fig.14 A schematic diagram of the structure of a silk screen defect detection device in a hardware operating environment according to an embodiment of the present invention;

[0046] Fig.15 It is a schematic diagram of the functional modules of an embodiment of a screen printing defect detection device of the present invention.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0049] Reference Figure 1 , Figure 1 The figure is a flow chart of an embodiment of a method for detecting screen printing defects based on unsupervised learning of the present invention. The method for detecting screen printing defects based on unsupervised learning disclosed by the present invention comprises:

[0050] Step S101, acquiring an abnormal image, and inputting the abnormal image into a reconstruction network to obtain a defect-free reconstructed image.

[0051] The reconstruction network is used to reconstruct the input image, that is, for an input defective abnormal image, a defect-free reconstructed image can be output through the reconstruction network. Specifically, Figure 2 As shown, the reconstruction network in this embodiment can be composed of an encoder and a decoder.

[0052] Step S102, comparing the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminarily locating defects.

[0053] See also Figure 2After image reconstruction, it is necessary to analyze the difference between the input abnormal image and the output reconstructed image to detect the location of the defect. In this embodiment, the image difference analysis method is mainly adopted, which generally includes three steps: (1) using the regional analysis method to perform a rough analysis to detect the defective area to avoid misjudging the background pixels as defects; (2) comparing the abnormal image and the reconstructed image pixel by pixel to obtain a preliminary residual image; (3) combining the results of (1) and (2), the defects are segmented more accurately to obtain a residual image with preliminary defect location.

[0054] Step S103, inputting the abnormal image, the reconstructed image and the residual image into a discriminant network to obtain an abnormal score map for accurately locating defects.

[0055] The residual image obtained based on the image difference analysis method roughly locates the location of the defect, but there is still the problem of unclear segmentation between the defect area and the background. Therefore, the present invention proposes a discriminant network for accurately locating defects, taking the abnormal image, reconstructed image and residual image as input of the discriminant network, and finally outputting an abnormal score map for accurately locating defects.

[0056] In the present invention, the discriminant network is composed of a convolutional network, the input end of the discriminant network is the channel cascade of the abnormal image, the reconstructed image and the residual image, and the output end is an abnormal score map of the same size as the abnormal image. The discriminant network used in other existing methods still includes a feature extraction network, which greatly increases the amount of calculation and the complexity of the network. For this reason, the present invention reuses the feature extractor in the reconstruction network, and uses the encoder of the reconstruction network to complete the feature extraction of the discriminant network, reducing the amount of calculation and the complexity of the model.

[0057] This method is based on the assumption that a well-trained network model reconstructs the input image, that is, for a defective sample image, the network model can reconstruct a defect-free sample image. Therefore, by reconstructing the defective image, the difference between the defect-free sample and the defective sample can be calculated, thereby achieving accurate positioning of the defective sample. The process can be expressed as:

[0058] I re =R(I src )

[0059] M o =D(I src ,I rec ,I mask )

[0060] Among them, I src represents the abnormal image generated by using the normal sample I, I rec Represents the reconstructed image output by the reconstruction network, I maskrepresents the residual image obtained by discrete cosine transform, R represents the reconstruction network, D represents the discriminant network for locating defects, and M0 represents the abnormal score map output by the discriminant network.

[0061] The silk screen defect detection method proposed in the present invention is realized by a reconstruction network and a discriminant network. After training, the reconstruction network can detect and reconstruct the abnormality of the abnormal area in the input image, reconstruct the abnormal area in the input image into a non-abnormal area, and keep the non-abnormal area in the input image unchanged. Secondly, a differential analysis method is introduced to calculate the difference between the abnormal image and the reconstructed image to obtain a residual image. The discriminant network can combine the embedded features of the abnormal image, the reconstructed image and the residual image to generate an abnormal score map that accurately locates the defects.

[0062] In this embodiment, the abnormal image is reconstructed into a defect-free reconstructed image through a reconstruction network, and an image differential analysis method is introduced to compare the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminarily locating defects. Finally, the abnormal image, the reconstructed image and the residual image are input into the discriminant network to obtain an abnormal score map for accurately locating defects, thereby achieving accurate detection of defective images.

[0063] Figure 3 FIG. 2 is a flow chart of another embodiment of the method for detecting screen printing defects of the present invention. Figure 3 As shown in the figure, the silk screen defect detection method based on unsupervised learning includes:

[0064] Step S201: Generate a simulated abnormal image.

[0065] See also Figure 2 In some embodiments, during the training phase, by performing data processing on normal images, normal sample images are simulated as defective abnormal images to obtain any number of abnormal samples, thereby solving the problem of few industrial defect samples in actual silk screen inspection and facilitating the training of the network model in the absence of real abnormal samples.

[0066] Furthermore, if Figure 4 As shown, step S201: the step of generating a simulated abnormal image may include:

[0067] Step S2011: randomly generate a noise image by a Perlin noise generator, and perform binarization processing on the noise image to obtain a simulated abnormal image mask and a reverse mask.

[0068] Figure 5 is an example of simulating abnormal image generation. Specifically, a noise image is randomly generated by a Perlin noise generator to obtain abnormal noise images of various shapes ( Figure 5, p), and binarize the abnormal noise images of various shapes through uniform random sampling thresholds to generate simulated abnormal image masks M a and reverse mask

[0069] Step S2012: sampling an abnormal texture source image from a preset source image data set, and performing data enhancement on the abnormal texture source image.

[0070] Specifically, we sample an abnormal texture source image from a source image dataset that is independent of the input image distribution ( Figure 5 , A), and then randomly select one or more methods from a preset set to enhance the sampling of the abnormal texture source image A, and the preset set at least includes brightness change, sharpness, exposure, color change and tone separation.

[0071] Step S2013, obtaining a normal sample image, and obtaining a simulated abnormal image based on the normal sample image, the simulated abnormal image mask and the reverse mask, and the abnormal texture source image after data enhancement.

[0072] Specifically, the enhanced abnormal texture source image A and the simulated abnormal image mask M a Multiply the normal image I and the simulated abnormal image mask M a Multiply, the normal image I and the inverse mask of the simulated abnormal image mask Multiply, add the three multiplication results to get the final simulated abnormal image I a , which can be expressed by the following formula:

[0073]

[0074] Among them, β is a hyperparameter, and the range of β is between [0.1, 1.0].

[0075] like Figure 6 As shown, the left side is a normal non-defective silk screen image, and the right side is a defective silk screen image simulated by Perlin noise. The above abnormal image generation method can generate different abnormal images from a single texture, and can also generate countless masks containing normal images, simulated abnormal images, and simulated abnormal images as samples for model training.

[0076] Furthermore, if Figure 7 As shown, step S201: generating a simulated abnormal image may also include:

[0077] Step S2011a: select a source image from a preset source image data set, and randomly generate a patch based on the source image.

[0078] Furthermore, a source image is selected from a source image dataset that is independent of the input image distribution, and then a patch is randomly selected from the source image. The size and shape of the patch can be adjusted without restriction.

[0079] Optionally, in this embodiment, a rectangular patch is randomly selected from the source image, and multiple rectangular patches can be added to the same target image.

[0080] Step S2012a, obtaining a normal sample image, and pasting the patch onto the normal sample image to obtain a simulated abnormal image.

[0081] Furthermore, the above-mentioned patches can be pasted to different positions on the target image, and the patches can be seamlessly integrated into the target image by using Poisson fusion to obtain an abnormal image.

[0082] Pasting an unrelated image onto a normal sample image, the common method will cause obvious discontinuity in the process of pasting part of the image, while the present invention adopts the Poisson fusion method, which can seamlessly paste part of an image onto the target image.

[0083] Specifically, for the source image g and the abnormal image f to be generated * , find an interpolation value f in the region Ω, defined as formula (1-1), to obtain the boundary condition The minimized value of f is obtained, and the abnormal image f is obtained according to the Poisson partial differential equation (Formula 1-2) * The unique solution under the Dirichlet boundary.

[0084]

[0085]

[0086] Among them, the gradient v is as shown in formula (1-3), and formula (1-4) represents the gradient of the source image g or the source image g and the abnormal image f * The gradient of the blending area.

[0087]

[0088]

[0089] like Figure 8 As shown, the left side is a normal non-defective silk-screen image, and the right side is a defective silk-screen image simulated by Poisson fusion. This embodiment uses Poisson fusion to achieve seamless fusion of the cropped rectangular patch and the target image, and simulates the normal sample image as a defective abnormal image, which is convenient for training the network model without real abnormal samples.

[0090] It can be understood that step S201 can only execute the method of steps S2011 to S2013 to generate an abnormal image by means of Perlin noise; it can also only execute the method of steps S2011a to S2012a to generate an abnormal image by means of Poisson fusion; or part of the abnormal image can be generated by means of Perlin noise, and the method of steps S2011 to S2013 can be executed, and part of the abnormal image can be generated by means of Poisson fusion, and the method of steps S2011a to S2012a can be executed.

[0091] Step S202: training the reconstruction network based on the simulated abnormal image.

[0092] The reconstruction network is used to reconstruct the input image, that is, for an input defective abnormal image, a defect-free reconstructed image can be output through the reconstruction network.

[0093] Specifically, Fig. 9 As shown in Figure 1, the encoder and decoder are mainly composed of four convolution blocks ConvBlocks and four deconvolution blocks DeconvBlocks. The structure of ConvBlocks is as follows: Fig.10 As shown in the figure, ConvBlocks contains two convolutional layers, two normalized BN layers and two LeakyReLU layers. In the reconstruction network, no pooling layer or upsampling layer is included. Specifically, in each ConvBlocks, the stride of the first convolutional layer is 2 and the stride of the second convolutional layer is 1, while in DeconvBlocks, the stride of the first convolutional layer is 1 and the stride of the second convolutional layer is 2.

[0094] Optionally, the reconstruction network also includes two convolutional layers E_pre and E_post, and two deconvolutional layers D_pre and D_post for pre-processing and post-processing. Fig. 9 As shown in the figure, the E_pre layer includes a convolution layer with a convolution kernel size of 5, a BN layer, and a LeakyReLU layer, and the E_post layer includes a convolution layer with a convolution kernel size of 8 and a BN layer. D_pre is a deconvolution layer with a convolution kernel size of 8, and D_post is a deconvolution layer with a convolution kernel size of 5. It is not difficult to see from the figure that in the reconstruction network, the size of the feature map is strictly symmetrical. As the encoder downsamples, the receptive field will expand step by step, the original image features are equivalent to being compressed, and some of the original image details will be lost. The strictly symmetrical feature map uses skip connections to save the details of the original image lost due to downsampling, so that the entire network can remember all the information of the image well.

[0095] In some embodiments, the structure of each layer of the reconstructed network is as follows Fig.11shown.

[0096] Furthermore, the reconstruction network can be trained based on simulated abnormal images.

[0097] Optionally, the reconstruction error between the abnormal image and the reconstructed image may be used as a reconstruction loss function, and the reconstruction network may be trained until convergence occurs.

[0098] In order to achieve high-precision reconstructed images and effectively detect defects on images with uneven texture distribution, a weighted loss function based on structural similarity index (SSIM) is introduced in the present invention. In the training stage of the reconstruction network, the reconstruction error between the abnormal image and the reconstructed image is used as the reconstruction loss function to further improve the quality of image reconstruction, which can be expressed as:

[0099] L(I src , I rec )=Diff(I src , I rec +λ||W N || F

[0100] Among them, W N is the hyperparameter of the reconstruction network, λ||W N || F In order to avoid overfitting, the conventional term Diff(I src ,I rec ) are two images I src and I rec The difference comparison function between .

[0101] To calculate the reconstruction error, the most widely used comparison function is the mean square error (MSE) function, which can be defined as:

[0102]

[0103] Where H and W are the width and height of the original input abnormal image, G src (i,j),G rec (i,j) are abnormal images I src and reconstructed image I rec The gray value of the pixel (i, j) in , where i, j represent the coordinates of each pixel.

[0104] However, the MSE function is more suitable for images with uniform texture distribution. There are multiple unevenly distributed textures on the abnormal image in the present invention. In the reconstruction process, the defective part is a relatively critical place, and the texture of the defective part is relatively complex. Therefore, in the training stage, the reconstruction network can quickly obtain a smaller reconstruction error in the area with relatively uniform texture, while the reconstruction error is larger in the transition area near the edge of the defect. Due to the huge difference caused by the above problems, even if the weighted loss function based on SSIM is close to convergence, the reconstructed image is still blurred, which will lead to inaccurate positioning of the detected defects.

[0105] In the present invention, in order to solve the above problem, the weight of each region can be modified when calculating the reconstruction error. A high weight value is assigned to the region where the defective part is located, while a lower weight value is assigned to the region with relatively uniform texture. In this way, the reconstruction network will focus on improving the reconstruction quality of important regions instead of being consistent for all regions.

[0106] In the reconstruction network, a window is slid across the abnormal image to extract a series of image blocks, and the variance of each image block is calculated to obtain a variance map, which can be expressed as:

[0107]

[0108] Among them, V(i,j) is the variance value of pixel (i,j), P ij is the image block centered at (i, j) extracted from the abnormal image, G p (i′,j′) represents P ij The gray value of each pixel (i′, j′) in the image block, Indicates P ij The average gray value, N rc Indicates P ij The number of pixels in the image. It can be seen that the variance can describe the complexity of the texture to a certain extent. The complex texture area has a higher variance value due to the sharp change of the gray value. Therefore, the variance can be assigned as follows:

[0109]

[0110] Where W(i,j) represents the weight value of pixel (i,j) in the weight map, and the weight value ranges from 0 to 1. b is the deviation value that balances the importance of high variance and low variance regions. C norm is a standardized parameter that ensures that the sum of W(i,j) is 1. The traditional mean square error can be written as:

[0111]

[0112] Since the MSE function is only a pixel-by-pixel comparison, it lacks regional information. Therefore, the structural similarity between the two regions in the abnormal image and the reconstructed image is used to measure the reconstruction error. The calculation process of SSIM can be expressed as:

[0113] SSIM(x,y)=l(x,y) α ×c(x,y) β ×s(x, y) γ

[0114]

[0115]

[0116]

[0117] Where x and y represent the two image blocks being compared, α, β, and γ are custom coefficients, l(x, y), c(x, y), and s(x, y) represent the comparison functions of brightness, contrast, and structure, respectively, and μ x and μ y represents the average intensity, and represents the variance, σ xy represents the correlation coefficient between x and y, c1, c2 and c3 are small constants to avoid unstable calculation when the denominator is very close to zero. In this embodiment, in order to simplify the expression, α=β=γ=1 and c3=c2 / 2 are set, so the SSIM formula is:

[0118]

[0119] L s (x, y) = 1 - SSIM (x, y)

[0120] L s (x, y) is the loss function based on SSIM. Combining SSIM with weighted average, the reconstruction loss function finally used in the reconstruction network training phase can be expressed as:

[0121]

[0122] Among them, I src and I rec are abnormal images and reconstructed images respectively, where (i, j) represents the image patch extracted from the pixel I src and I rec , and add weights to each image patch separately.

[0123] Traditional deep convolutional networks usually use the MSE loss function between the input image and the reconstructed image to train parameters, but it is not applicable to images with uneven texture distribution. The weighted loss function design method based on SSIM provided by the present invention adopts a weighted average method when calculating the loss function to adapt to the unevenness of texture distribution, thereby improving the quality of the reconstructed image.

[0124] Step S203, acquiring an abnormal image, and inputting the abnormal image into a reconstruction network to obtain a defect-free reconstructed image.

[0125] Specifically, the abnormal image may be a real abnormal sample or a simulated abnormal image.

[0126] When the abnormal image is a real abnormal sample, it can be used for silk screen defect detection; when the abnormal image is a simulated abnormal image, it can be used to verify the silk screen defect detection level.

[0127] Step S204: using a regional analysis method to compare the difference between the abnormal image and the reconstructed image, and determining the defective image block where the defect is located.

[0128] After completing the image reconstruction, it is necessary to analyze the difference between the abnormal image and the reconstructed image to preliminarily detect the location of the defect.

[0129] In some embodiments, a regional analysis method is used to roughly locate areas where defects may exist. The specific operation is: the abnormal image and the reconstructed image are respectively subdivided into image blocks of size N*N and step length S. In this embodiment, N and S are set to 32. Then, the difference between the abnormal image block and the reconstructed image block is measured by spectrum analysis to determine whether there is a defect. The measurement is achieved by discrete cosine transform, which can be expressed as:

[0130]

[0131]

[0132] Among them, f(i, j) is the grayscale value of the pixel position (i, j) in the image block, c(u) and c(v) are coefficients, F(u, v) represents the value of the amplitude spectrum image (u, v) after discrete cosine transform, and u and v represent the pixel coordinates in the amplitude spectrum image.

[0133] Step S204, comparing the abnormal image and the reconstructed image pixel by pixel, preliminarily locating defects in the defective image block, and obtaining a residual image.

[0134] Since there is a large grayscale difference between the defect and the background, there are additional high-energy components in the atlas image of the defect, so the average amplitude value of the defect area will be greater than the average amplitude value of the reconstructed image block. Due to the energy concentration effect of the discrete cosine transform, the discrete cosine transform is very sensitive to the defect area, while maintaining robustness to the transformation of the reconstructed normal area. Therefore, the discrete cosine transform can be used as a defect feature extraction method to calculate the average amplitude value of the image block as a feature to distinguish between defect blocks and normal blocks. It can be expressed as the following formula:

[0135]

[0136] Where p is the image block, f DCT (p) represents the feature extracted from p, L is the statistical range in the mean calculation, and in this embodiment, the value is 8, F(u, v) represents the amplitude value of the discrete cosine transform result corresponding to p, F(0, 0) will be affected by the average gray value of p and interfere with subsequent judgments, so F(0, 0) is subtracted here.

[0137] In the testing phase, the feature image blocks based on discrete cosine transform are calculated for the original image and the reconstructed image to distinguish the normal image blocks from the defective image blocks, which can be written as:

[0138]

[0139] Among them, r∈(1,....,R r ), c∈(1,....,Rc), R r =(HN) / S+1,R c =(WN) / S+1, ε is a threshold, I mask Analysis results in the (r,c) region.

[0140] like Fig.12 As shown, the left side is the abnormal image obtained by simulation, and the right side is the residual image after image difference analysis. In this embodiment, the region analysis method is used to roughly locate the region where defects may exist, thereby improving the performance of accurate segmentation of defects.

[0141] Step S205: input the abnormal image, the reconstructed image and the residual image into the discriminant network to obtain an abnormal score map for accurately locating defects.

[0142] The residual image obtained by the differential analysis method based on discrete cosine transform can roughly locate the position of the defect in the defect image block. However, there is still the problem of unclear segmentation between the defect area and the background at the defect location. The accurate positioning of the defect position still needs to be achieved through the discriminant network.

[0143] The present invention proposes an encoder-decoder-discriminator network architecture, wherein the reconstruction network is composed of an encoder and a decoder, and the discriminant network is composed of a discriminator, more specifically, a convolutional network. At the same time, in order to reduce the amount of calculation and the complexity of the network model, the discriminant network in the present invention shares the feature extractor of the reconstruction network, and uses the encoder of the reconstruction network to complete the feature extraction of the discriminant network. The input end of the discriminant network is the channel cascade of the abnormal image, the reconstructed image, and the residual image. Due to the normal recovery characteristics of the reconstruction network, the abnormal image and the reconstructed image have obvious abnormal features. These abnormal features are fused with the abnormal features on the residual image, and the discriminant network is used to learn the difference between the abnormal image and the normal sample image, thereby outputting an abnormal score map of the same size as the abnormal image. Fig.13 As shown in the figure, the left side is the abnormal image obtained by simulation, and the right side is the abnormal score map output by the discriminant network. The location of the defect can be clearly seen from the figure.

[0144] L2loss is often used in the reconstruction network. The reconstruction network is based on the independence of adjacent pixels. At the same time, in order to achieve high-precision reconstructed images, the present invention also introduces a weighted loss function based on SSIM. Combining the above loss functions, the total loss of the reconstruction network and the discriminant network can be expressed as:

[0145] L rec (I src ,I rec )=δL(I src ,I rec )+l2(I src ,I rec )

[0146] Among them, δ is a hyperparameter. The reconstruction network reconstructs the abnormal image into a normal image, and the discriminant network accurately locates the defect by detecting the difference between the reconstructions.

[0147] Apply Focal Loss (L) to the output of the discriminative network seg ) Enhance the robustness of defect accurate positioning and segmentation. Considering the goals of the reconstruction network and the discriminant network, the total loss function during training is:

[0148] L(I src ,I rec ,M a ,M o )=L rec (I src ,I rec )+L seg (M a ,M o )

[0149] Among them, M a is the ground truth label of the defect image, M o This is the abnormal score map output by the discriminant network. During training, when the value of the total loss function is minimized, the model training achieves the optimal effect, and the trained model can be used to detect real defects.

[0150] like Fig.14 As shown, Fig.14 The present invention is a schematic diagram of the structure of a silk screen defect detection device in the hardware operating environment involved in the embodiment of the present invention.

[0151] The screen printing defect detection device in the embodiment of the present invention may be a PC, or may be a terminal device with data processing capability such as a portable computer or a server.

[0152] like Fig.14 As shown, the silk screen defect detection device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0153] Those skilled in the art will understand that Fig.14 The structure of the silk screen defect detection device shown in the figure does not constitute a limitation on the silk screen defect detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0154] like Fig.14 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a silk screen defect detection program.

[0155] exist Fig.14In the silk screen defect detection device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the silk screen defect detection program stored in the memory 1005, and execute the operations of the various embodiments of the above silk screen defect detection method based on unsupervised learning.

[0156] Reference Fig.15 , Fig.15 It is a schematic diagram of the functional modules of an embodiment of a screen printing defect detection device of the present invention.

[0157] The screen printing defect detection device includes:

[0158] The reconstruction module 10 is used to obtain an abnormal image and input the abnormal image into a reconstruction network to obtain a defect-free reconstructed image;

[0159] A preliminary positioning module 20, used to compare the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminary positioning of the defect;

[0160] The precise positioning module 30 is used to input the abnormal image, the reconstructed image and the residual image into the discriminant network to obtain an abnormal score map for accurately positioning defects.

[0161] Furthermore, the screen printing defect detection device further includes an abnormal image generation module 40, and the abnormal image generation module 40 specifically includes:

[0162] A noise generating unit, used for randomly generating a noise image by a Perlin noise generator, and performing binarization processing on the noise image to obtain a simulated abnormal image mask and a reverse mask;

[0163] A data enhancement unit, used for sampling an abnormal texture source image from a preset source image data set, and performing data enhancement on the abnormal texture source image;

[0164] The abnormal simulation unit is used to obtain a normal sample image, and obtain a simulated abnormal image based on the normal sample image, the simulated abnormal image mask and the reverse mask, and the abnormal texture source image after data enhancement.

[0165] Furthermore, the abnormal image generating module 40 includes:

[0166] A patch generation unit, configured to select a source image from a preset source image data set and randomly generate a patch based on the source image;

[0167] The patch pasting unit is used to obtain a normal sample image and paste the patch onto the normal sample image to obtain a simulated abnormal image.

[0168] Furthermore, the screen printing defect detection device also includes:

[0169] A reconstruction training module 50, for training a reconstruction network based on simulated abnormal images;

[0170] The network convergence module 60 is used to use the reconstruction error between the abnormal image and the reconstructed image as a reconstruction loss function to train the reconstruction network until it converges.

[0171] Furthermore, the preliminary positioning module 20 includes:

[0172] An analysis unit, used for comparing the difference between the abnormal image and the reconstructed image by using a regional analysis method, and determining a defective image block where a defect is located;

[0173] The comparison unit is used to compare the abnormal image and the reconstructed image pixel by pixel, preliminarily locate the defect in the defective image block, and obtain a residual image.

[0174] The specific embodiments of the screen printing defect detection device of the present invention are basically the same as the various embodiments of the screen printing defect detection method based on unsupervised learning, and will not be described in detail here.

[0175] In addition, an embodiment of the present invention further proposes a storage medium, on which a silk screen defect detection program is stored. When the silk screen defect detection program is executed by a processor, the steps of the silk screen defect detection method based on unsupervised learning as described above are implemented.

[0176] The specific embodiments of the computer storage medium of the present invention are basically the same as the various embodiments of the above-mentioned silk screen defect detection method based on unsupervised learning, and will not be described in detail here.

[0177] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0178] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0180] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting screen printing defects based on unsupervised learning, characterized in that: The silk screen defect detection method based on unsupervised learning comprises the following steps: Acquire an abnormal image, and input the abnormal image into a reconstruction network to obtain a defect-free reconstructed image; Comparing the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminary defect location; Inputting the abnormal image, the reconstructed image and the residual image into a discriminant network to obtain an abnormal score map for accurately locating defects; The method further includes generating a simulated abnormal image, wherein the step of generating the simulated abnormal image includes: randomly generating a noise image by a Perlin noise generator, and binarizing the noise image to obtain a simulated abnormal image mask and an inverse mask; sampling an abnormal texture source image from a preset source image data set, and performing data enhancement on the abnormal texture source image; acquiring a normal sample image, and obtaining a simulated abnormal image based on the normal sample image, the simulated abnormal image mask and the inverse mask, and the abnormal texture source image after data enhancement; The method further comprises: training a reconstruction network based on a simulated abnormal image, taking a reconstruction error between the simulated abnormal image and the reconstructed image as a reconstruction loss function, and training the reconstruction network until convergence; The reconstruction network includes an encoder and a decoder, the discriminant network includes a convolutional network, and the discriminant network shares the encoder of the reconstruction network to complete feature extraction; The encoder and decoder are composed of four convolution blocks and four deconvolution blocks. The convolution block contains two convolution layers, two normalized BN layers and two LeakyReLU layers. In the reconstruction network, no pooling layer and upsampling layer are included. In each convolution block, the step size of the first convolution layer is 2 and the step size of the second convolution layer is 1, while in the deconvolution block, the step size of the first convolution layer is 1 and the step size of the second convolution layer is 2. The reconstructed network also contains two convolutional layers E_pre and E_post, and two deconvolutional layers D_pre and D_post for preprocessing and post-processing. The E_pre layer includes a convolutional layer with a convolution kernel size of 5, a BN layer, and a LeakyReLU layer. The E_post layer includes a convolutional layer with a convolution kernel size of 8 and a BN layer. D_pre is a deconvolutional layer with a convolution kernel size of 8, and D_post is a deconvolutional layer with a convolution kernel size of 5.

2. The silk screen defect detection method based on unsupervised learning as claimed in claim 1, characterized in that: The step of performing data enhancement on the abnormal texture source image comprises: One or more methods are randomly selected from a preset set to enhance sampling of the abnormal texture source image, wherein the preset set at least includes brightness change, sharpness, exposure, color change and tone separation.

3. The method for detecting screen printing defects based on unsupervised learning according to any one of claims 1 to 2, characterized in that: The step of comparing the difference between the abnormal image and the reconstructed image to obtain a residual image for preliminary defect location includes: Using a regional analysis method to compare the difference between the abnormal image and the reconstructed image, and determine the defective image block where the defect is located; The abnormal image and the reconstructed image are compared pixel by pixel, and the defect is preliminarily located in the defective image block to obtain a residual image.

4. The method for detecting screen printing defects based on unsupervised learning according to any one of claims 1 to 2, characterized in that: Also includes: The abnormal image, the reconstructed image and the residual image are used to perform initial training on the discriminant network, and the total loss function of the reconstructed network and the discriminant network is calculated until the value of the total loss function is minimized.

5. A screen printing defect detection device based on unsupervised learning, characterized in that: The screen printing defect detection device comprises: A reconstruction module, used for acquiring an abnormal image and inputting the abnormal image into a reconstruction network to obtain a defect-free reconstructed image; A preliminary positioning module, used to compare the difference between the abnormal image and the reconstructed image to obtain a residual image of the preliminary positioning defect; A precise positioning module, used for inputting the abnormal image, the reconstructed image and the residual image into the discriminant network to obtain an abnormal score map of the precisely positioned defects; The device also includes an abnormal image generation module, which is used to generate a simulated abnormal image; The abnormal image generation module is also used to execute the following instructions: randomly generate a noise image by a Perlin noise generator, and perform binarization processing on the noise image to obtain a simulated abnormal image mask and an inverse mask; sample an abnormal texture source image from a preset source image data set, and perform data enhancement on the abnormal texture source image; obtain a normal sample image, and obtain a simulated abnormal image based on the normal sample image, the simulated abnormal image mask and the inverse mask, and the abnormal texture source image after data enhancement; The device further comprises: a reconstruction training module, used for training the reconstruction network based on the simulated abnormal image; a network convergence module, used for taking the reconstruction error between the simulated abnormal image and the reconstructed image as a reconstruction loss function, and training the reconstruction network until convergence; The reconstruction network includes an encoder and a decoder, the discriminant network includes a convolutional network, and the discriminant network shares the encoder of the reconstruction network to complete feature extraction; The encoder and decoder are composed of four convolution blocks and four deconvolution blocks. The convolution block contains two convolution layers, two normalized BN layers and two LeakyReLU layers. In the reconstruction network, no pooling layer and upsampling layer are included. In each convolution block, the step size of the first convolution layer is 2 and the step size of the second convolution layer is 1, while in the deconvolution block, the step size of the first convolution layer is 1 and the step size of the second convolution layer is 2. The reconstructed network also contains two convolutional layers E_pre and E_post, and two deconvolutional layers D_pre and D_post for preprocessing and post-processing. The E_pre layer includes a convolutional layer with a convolution kernel size of 5, a BN layer, and a LeakyReLU layer. The E_post layer includes a convolutional layer with a convolution kernel size of 8 and a BN layer. D_pre is a deconvolutional layer with a convolution kernel size of 8, and D_post is a deconvolutional layer with a convolution kernel size of 5.

6. A storage medium, characterized in that: The storage medium stores a silk screen defect detection program, and when the silk screen defect detection program is executed by the processor, the steps of the silk screen defect detection method based on unsupervised learning as described in any one of claims 1 to 4 are implemented.

Images