A PCB defect detection method based on image processing

Abstract

The present invention belongs to the technical field of defect detection, and is a PCB defect detection method based on image processing, including: collecting image data of a defect-free PCB board and making it into a data set; respectively setting loss functions for the reconstruction subnet and the segmentation subnet, and using the data set for training to obtain the trained reconstruction subnet and segmentation subnet. The trained reconstruction subnet uses a diffusion model to gradually remove noise from the image data to be detected that has undergone preprocessing and data augmentation according to time steps to obtain a reconstructed image. The reconstructed image is decomposed into low-frequency components and high-frequency components through discrete wavelet transform. The low-frequency components are input into the trained segmentation subnet for comparison with the image data to be detected, and the defective parts are distinguished. The present invention uses far fewer sample numbers than other object detection models. The present invention uses discrete wavelet transform to extract high-frequency components for reverse diffusion denoising, improving the inference speed, and improving the accuracy of segmentation and defect detection through the extracted low-frequency components.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect detection, and particularly relates to a PCB defect detection method based on image processing. Background Art

[0002] With the rapid development of the electronics industry, the manufacturing technology of PCB (printed circuit board) is evolving towards high density, miniaturization and functional complexity, which poses higher requirements for the quality inspection of PCB. PCB defect detection plays a crucial role in ensuring product performance, reliability and safety. However, with the increasing complexity of PCB design, traditional detection methods are facing many challenges.

[0003] Currently, PCB defect detection technologies mainly include manual visual inspection, automatic optical inspection, automatic X-ray inspection and electrical testing. In recent years, with the booming development of deep learning technology, defect detection methods based on deep learning models such as Convolutional Neural Networks (CNNs) have gradually attracted wide attention. Deep learning models have powerful feature extraction and pattern recognition capabilities, can automatically learn complex image features without relying on traditional artificial feature design, and perform well in processing large-scale data. This makes the potential of deep learning in PCB defect detection widely recognized. However, although deep learning methods have made certain progress in the field of PCB defect detection, there are still many deficiencies. The performance of deep learning models highly depends on large-scale and high-quality labeled datasets. However, in PCB defect detection, high-quality defect data is often scarce, especially for some rare defect types, and it is difficult to collect enough samples. PCB defects usually show tiny structural changes, such as subtle solder joint damage, tiny circuit breaks, etc. Deep learning models have significant effects in dealing with larger targets, but there are still difficulties in detecting tiny defects. Since deep learning models usually perform detection on feature maps at a larger scale, they may ignore the subtle details at a smaller scale, resulting in unsatisfactory detection effects for tiny defects. In addition, the feature extraction process of convolutional neural networks usually involves multiple layers of downsampling operations, which may further lose the key information of tiny defects, making it difficult for the model to accurately locate and identify these small defects in the context of high-density and complex circuits. Especially for YOLO often used in existing PCB defect detection, its ability to detect small defects is weak and it is easy to miss detections. Deep learning models usually rely on specific defect types in the training data for learning, so when facing defect types that do not appear or appear less frequently in the training set, the generalization ability of the model is weak. The training of deep learning models usually requires a large amount of computing resources, especially when the network structure is more complex or the data volume is larger, the training time and hardware cost increase significantly. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a PCB defect detection method based on image processing, which solves the problems that when facing defect types that do not appear or appear less frequently in the training set, the generalization ability of the model is weak, and the training of deep learning models usually requires a large amount of computing resources.

[0005] The present invention is implemented as follows.

[0006] A PCB defect detection method based on image processing, the method comprising:

[0007] S1: Collect image data of a defect-free PCB board, preprocess and augment the PCB board image data, and make it into a data set.

[0008] S2: Set loss functions for the reconstruction subnet and the segmentation subnet respectively, use the data set for training to obtain the trained reconstruction subnet and segmentation subnet. The trained reconstruction subnet uses a diffusion model to gradually remove noise from the image data to be detected that has been preprocessed and augmented according to time steps to obtain a reconstructed image. The reconstructed image is decomposed into low-frequency components and high-frequency components through discrete wavelet transform. The low-frequency components are input into the trained segmentation subnet to be compared with the image data to be detected that has been preprocessed and augmented, and the defective parts are distinguished.

[0009] Further, during the process of training using the data set, in the reconstruction subnet, the defect is added as noise to the PCB board image data according to time steps for forward diffusion processing until the PCB board image data is completely converted into pure noise data; the pure noise data is processed through reverse diffusion to gradually remove noise according to time steps to obtain a reconstructed image; the loss function of the reconstruction subnet is:

[0010] , where and are two different time steps respectively, is a label indicating whether the sample is abnormal, 0 represents a normal sample, and 1 represents an abnormal sample, is the real noise added at time step , is the real noise added at time step , is the noise predicted by the reconstruction subnet at time step , is the noise predicted by the reconstruction subnet at time step .

[0011] Further, the loss function of the segmentation subnet is:

[0012] ,

[0013] where is the true anomaly mask of the input image, is the anomaly mask predicted by the segmentation subnet, is the smooth L1 loss function, is the focal loss function, is a hyperparameter used to balance the importance of the L1 loss function and the focal loss function. The true anomaly mask refers to the image area with noise.

[0014] Furthermore, the total loss function of the reconstruction subnet and the segmentation subnet is:

[0015] ,

[0016] By training, the value of the total loss function is reduced so that the output result of the segmentation subnet is close to the true anomaly mask.

[0017] Furthermore, the discrete wavelet transform includes convolving the input reconstructed image in the horizontal and vertical directions with a low-pass wavelet filter and a high-pass wavelet filter respectively to obtain the low-frequency component and the high-frequency component.

[0018] Furthermore, the high-frequency component is fed back to the reverse diffusion process, and the reverse diffusion process removes the fed-back high-frequency component as the noise to be removed.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention only needs to use the dataset of normal samples for training, avoiding the problems of difficult collection of defective data and sample imbalance. The present invention uses far fewer samples than other object detection models, can achieve small-sample learning, and at the same time maintains high accuracy. The present invention utilizes the discrete wavelet transform to extract the high-frequency component for reverse diffusion denoising, improving the inference speed, and improving the accuracy of segmentation and defect detection through the extracted low-frequency component. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is the flowchart of the method provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] See Figure 1As shown in the figure, the present invention provides a PCB defect detection method based on image processing, and the method includes:

[0023] S1: Collect image data of a defect-free PCB board, preprocess and enhance the PCB board image data, and make it into a data set;

[0024] The present invention uses a normal defect-free PCB board as a sample to collect image data.

[0025] S2: Set loss functions for the reconstruction subnet and the segmentation subnet respectively, use the data set for training to obtain the trained reconstruction subnet and segmentation subnet. The trained reconstruction subnet uses a diffusion model to gradually remove noise from the image data to be detected that has been preprocessed and enhanced according to time steps to obtain a reconstructed image. The reconstructed image is decomposed into low-frequency components and high-frequency components through discrete wavelet transform. The low-frequency components are input into the trained segmentation subnet for comparison with the image data to be detected that has been preprocessed and enhanced, and the defective parts are distinguished.

[0026] During the process of training with the data set, in the reconstruction subnet, the defect is added as noise to the PCB board image data according to time steps for forward diffusion processing until the PCB board image data is completely converted into pure noise data; the pure noise data is subjected to reverse diffusion processing to gradually remove noise according to time steps to obtain a reconstructed image; the loss function of the reconstruction subnet is:

[0027] , where and are two different time steps respectively, is a label indicating whether the sample is abnormal, 0 represents a normal sample, and 1 represents an abnormal sample, is the real noise added at time step , is the real noise added at time step , is the noise predicted by the reconstruction subnet at time step , is the noise predicted by the reconstruction subnet at time step .

[0028] During the forward diffusion processing, it is described by the following formula:

[0029] ,

[0030] where, is the noise intensity controlling each time step, represents a normal distribution, is the The image data of the time step, is the image data of the previous step, is the identity matrix, is the output of each step of the forward diffusion process.

[0031] After multiple diffusion steps, the image data will become pure noise, that is . In the reverse diffusion process, the image is reconstructed by gradually removing the noise. This process usually uses the following reverse diffusion formula:

[0032] ,

[0033] where, and are the mean and standard deviation predicted by the neural network respectively, used to generate the image at the time step.

[0034] The noise in the above process is added to the image data as a defect using Simplex noise. In the present invention, noise is added to the non-defective qualified product samples to simulate unknown defects. The area where the noise is added is used as the true anomaly mask.

[0035] The present invention trains the reconstruction subnet by minimizing the loss function of the reconstruction subnet.

[0036] The loss function of the segmentation subnet is:

[0037] ,

[0038] where is the true anomaly mask of the input image, is the anomaly mask predicted by the segmentation subnet, is the smooth L1 loss function, is the focal loss function, is a hyperparameter used to balance the importance of the L1 loss function and the focal loss function. The true anomaly mask refers to the image area with noise.

[0039] The segmentation subnet is trained by minimizing the loss function of the segmentation subnet.

[0040] During the training process, the reconstruction subnet and the segmentation subnet are not trained separately. For the entire network, the total loss function of the reconstruction subnet and the segmentation subnet is established as:

[0041] ,

[0042] Reducing the total loss function through training such that the output result of the segmentation subnet approaches the true anomaly mask.

[0043] A discrete wavelet transform layer is set between the reconstruction subnet and the segmentation subnet for performing discrete wavelet transform on the reconstructed image output by the reconstruction subnet. The purpose is to decompose the reconstructed image of the input two-dimensional data into a low-frequency subband and a high-frequency subband. Convolving the input reconstructed image with a wavelet low-pass filter and a wavelet high-pass filter can obtain the low-frequency component and the high-frequency component respectively. For each direction, including the horizontal direction and the vertical direction, first apply one filter (wavelet low-pass filter or wavelet high-pass filter), and then apply the other filter. This will generate four subbands: low-frequency-low-frequency (LL), low-frequency-high-frequency (LH), high-frequency-low-frequency (HL), and high-frequency-high-frequency (HH). The low-frequency-low-frequency (LL) is the main component of the input data and contains most of the information of the image. It is obtained by convolving the input reconstructed image with the wavelet low-pass filter in the horizontal and vertical directions. The remaining high-frequency components mainly contain some noise and edge information.

[0044] In the present invention, the low-frequency information is selected and sent into the segmentation subnet to filter out high-frequency noise, which is more conducive to the segmentation subnet comparing the enhanced image and distinguishing the defective part.

[0045] For the decomposed high-frequency information, which is mainly composed of noise, the high-frequency component is fed back to the reverse diffusion process, and the reverse diffusion process removes the fed-back high-frequency component as the noise to be removed. Through such feedback, the denoising effect of the reconstruction subnet can be increased. The reverse diffusion formula becomes the following form:

[0046] ,

[0047] is the high-frequency-high-frequency component at the t-1 time step, is the high-frequency-low-frequency component at the t-1 time step, is the low-frequency-high-frequency component at the t-1 time step.

[0048] During the training process, the input of the segmentation subnet is divided into two parts: the output of the reconstruction subnet and the PCB board image data with added defects. By comparing the differences between the two, the anomaly score of each region is obtained. The greater the difference between the two, the higher the anomaly score.

[0049] When actually used for detection, the input of the segmentation subnet includes: the output of the reconstruction subnet and the PCB board image data to be detected. By comparing the differences between the two, the anomaly score of each region is obtained. The greater the difference between the two, the higher the anomaly score, thereby distinguishing the defective part. The defect detection of the PCB board is realized.

[0050] In actual detection, for the original images collected by the industrial camera, they are divided into four sub-images each containing only one PCB board to be detected. The four divided sub-images are separately sent into the reconstruction subnet. The reconstruction subnet only performs reverse diffusion processing to remove noise from the input PCB board image data to obtain a reconstructed image, and then inputs it into the segmentation subnet to obtain the detection result.

[0051] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A PCB defect detection method based on image processing, characterized in that, The method includes: S1: Collect image data of defect-free PCB boards, preprocess and data-augment the PCB board image data, and make it into a dataset; S2: Set loss functions for the reconstruction subnet and the segmentation subnet respectively, use the dataset for training to obtain the trained reconstruction subnet and segmentation subnet. The trained reconstruction subnet uses a diffusion model to gradually remove noise from the preprocessed and data-augmented image data to be detected according to time steps to obtain a reconstructed image. The reconstructed image is decomposed into low-frequency components and high-frequency components through discrete wavelet transform. The low-frequency components are input into the trained segmentation subnet for comparison with the image data to be detected, and the defective parts are distinguished; During the process of training using the dataset, in the reconstruction subnet, defects are added as noise to the PCB board image data according to time steps for forward diffusion processing until the PCB board image data is completely converted into pure noise data; the pure noise data undergoes reverse diffusion processing, and noise is gradually removed according to time steps to obtain a reconstructed image; The high-frequency components are fed back to the reverse diffusion processing, and the reverse diffusion processing removes the fed-back high-frequency components as noise to be removed.

2. The PCB defect detection method based on image processing according to claim 1, wherein Loss function for reconstructing the subnet is as follows: , where and are two different time steps respectively, is a label indicating whether the sample is abnormal. 0 represents a normal sample and 1 represents an abnormal sample. is the real noise added at time step . is the real noise added at time step . is the noise predicted by the reconstruction subnet at time step . is the noise predicted by the reconstruction subnet at time step .

3. The PCB defect detection method based on image processing according to claim 2, wherein, The loss function of the segmented subnet is as follows: ， wherein is the true anomaly mask of the input image, is the anomaly mask predicted by the segmentation subnet, is the smooth L1 loss function, is the focal loss function, is a hyperparameter used to balance the importance of the L1 loss function and the focal loss function, and the true anomaly mask refers to the image region with noise.

4. The method for detecting PCB defects based on image processing according to claim 3, wherein Total loss function for reconstructing and splitting subnets is as follows: ， Reducing the total loss function through training such that the output result of the segmentation subnet is close to the true anomaly mask.

5. A PCB defect detection method based on image processing according to claim 2, characterized in that, The discrete wavelet transform includes convolving the input reconstructed image in the horizontal and vertical directions with a low-pass wavelet filter and a high-pass wavelet filter respectively to obtain low-frequency components and high-frequency components.

Images