PCB defect detection system and method based on improved YOLO algorithm

Abstract

The invention relates to a PCB (Printed Circuit Board) defect detection system and method based on an improved YOLO algorithm, the detection system adopts the improved YOLOX algorithm, and by optimizing data enhancement, instability caused by single Mosaic data enhancement is reduced, the precision of a model is improved, and the accuracy of the model is improved. Comprising an image acquisition and processing module, a neural network model improvement module, a model training module, a model verification module and a defect detection module. Meanwhile, the detection method comprises the steps of building a PCB defect data set image acquisition system, acquiring PCB surface defect images as an original data set through an industrial CCD camera, preprocessing the data set images, improving a YOLOX network model, setting parameters of a training model, generating a PCB defect detection model, testing a test set and the like. The defect detection system can realize automatic detection of PCB defects, and has the advantages of low cost, high speed and high precision.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision defect detection, and in particular to a PCB defect detection system and method based on an improved YOLO algorithm. Background Art

[0002] PCB (Printed Circuit Board) is a vital component of modern electronic devices, used to connect and support electronic components. With the development of electronic manufacturing, the development of PCB boards towards miniaturization and complexity is the general trend, which leads to various defects more likely to occur in the PCB manufacturing process, such as: leaks, rat bites, open circuits, short circuits, burrs, fake copper, etc. These defects may cause circuit failures or performance degradation, and in severe cases may cause life-threatening situations for users. However, most of the defects of PCB boards are very small in size, and the color is similar to the background color of the PCB circuit. It is very difficult to detect them by human eyes, and the detection accuracy is not high. Therefore, the development of efficient PCB defect detection technology is of great significance in the modern electronic manufacturing industry, which can effectively improve product quality, reduce production costs, and ensure the stability and reliability of electronic equipment.

[0003] At present, the mainstream PCB defect detection methods include machine vision detection, manual sampling, x-ray detection, ultrasonic detection, etc. Among them, manual sampling is too inefficient and prone to missed detection and false detection; x-ray detection has high accuracy but high cost and slow speed, and is only suitable for products with high precision requirements; ultrasonic detection can only detect internal defects of PCB and cannot find surface problems. For PCB surface defect detection, visual inspection is low cost and high efficiency, and is currently the most mainstream detection method.

[0004] At present, target detection algorithms are mainly divided into two categories: single target detection algorithms headed by YOLO and dual target detection algorithms headed by FasterRCNN. Among them, the dual target detection algorithm has problems such as poor model accuracy and slow recognition speed when the defect types are unbalanced and the defect size is relatively small. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a PCB defect detection system based on an improved YOLO algorithm. The detection system adopts an improved YOLO algorithm, optimizes data enhancement, reduces the instability caused by single Mosaic data enhancement, and improves the accuracy of the model. The system includes: an image acquisition processing module, a neural network model improvement module, a model training module, a model verification module, and a defect detection module; The image acquisition and processing module acquires images of PCB surface defects by building an image acquisition system, and processes the acquired images to obtain a data set; the image acquisition and processing module includes a camera, a lens, a clamping device, and a base, and a ring light source is also provided on the lens side. After the camera and the lens are assembled, they are arranged together on the clamping device, and the output interface of the camera is connected to the computer hardware; The neural network model improvement module, model training module, model verification module and defect detection module are all run on computer hardware as data modules. The neural network model improvement module is improved based on the YOLOX neural network to build a defect detection model. The improvement includes using the Focus structure to stack the feature layers and expand the input channels; ResBlock is used to reconstruct CSPLayer in the feature fusion stage. The reconstructed CSPRB module can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the gradient duplication problem; then spatial pyramid pooling is used to process the maximum pooling of four different scales, which greatly improves the receptive field; finally, Decoupled Head is used as YOLO Head; The model training module is used to set the network parameters of the defect detection model, input the prepared data set into the defect detection model for training, and use Adam as the optimizer in the training. The learning rate of the Adam optimizer is set to 0.001, the weight decay is set to 0.0005, the number of training epochs is set to 500, and the batch size is set to 4; The model verification module is used to use the verification set to test the defect detection model that has been initially trained, adjust the learning rate and epoch parameters according to the verification results, improve the verification accuracy, and obtain the final defect detection model; The defect detection module is used to input the test set data into the trained defect detection model, detect and mark the type, location and confidence of the PCB surface defects in the test set.

[0006] In one embodiment of the present invention, the lighting lens ring on the lens and the PCB sample to be tested on the base are on the same vertical straight line, and a rectangular light source and a light source controller are also provided on the base, and a control line is provided between the input end of the light source controller and the computer hardware; wherein the camera is an industrial CCD camera, and a ring light source is provided, which is coaxially and vertically fixed above the PCB board to be tested, and a rectangular light source is placed under the PCB board, and the defect image of the PCB surface is obtained by adjusting the distance between the camera, the light source and the PCB board and the exposure time.

[0007] The present invention also provides a PCB defect detection method based on an improved YOLO algorithm, which is built based on the PCB defect detection system and includes the following steps: Step S1: Building an image acquisition system to acquire images of defective samples; Step S2: Editing the acquired defect image to obtain an image of a uniform size; Step S3: Use data enhancement method to expand the sample, and then perform noise reduction and contrast enhancement on the image; Step S4: selecting image samples from the image, marking the locations and types of defects in the image samples, and creating a data set; Step S5: input the processed RGB three-channel image into the improved YOLOX neural network for training to obtain a target detection model for PCB panel defects; Step S6: Read the obtained target detection model and test the test set prepared in step S4 to achieve recognition, classification and location of PCB defects; Specifically, the target detection model in step S5 includes: a CSPDarknet53 backbone feature extraction network using a Focus structure, a spatial pyramid pooling structure SPP, a path aggregation network PAFPN, and a YOLO Head with a Decoupled Head.

[0008] In one embodiment of the present invention, the size of the image after being resized in step S2 is 640*640.

[0009] In one embodiment of the present invention, the data enhancement method described in step S3 is specifically as follows: first, the acquired defect image is translated, flipped, and rotated for data expansion, and then the expanded image is subjected to image denoising and enhancement. The image is first subjected to denoising filtering processing by combining average filtering and smooth filtering, wherein moving is selected as the smoothing processing method of smooth filtering, and span is set to 5; the number of windows of average filtering is set to 4. This denoising method can effectively remove flicker noise in the image without reducing the resolution of the image, and then contrast-limited adaptive image enhancement is used to improve the contrast between the defect and the background in the image.

[0010] In one embodiment of the present invention, the types of defects in step S4 include: leak holes, mouse bites, open circuits, short circuits, burrs, and fake copper. The image annotation tool LabelImg is used to annotate the standardized data set. Each annotated PCB image will generate a corresponding txt file. The content contained in the txt file includes: category information, center point position coordinates, width, and height of the target bounding box. The target bounding box is a labeled rectangular box that surrounds the defect.

[0011] In one embodiment of the present invention, in order to ensure the independence of the data set in step S4, the data set is first divided into a training validation set and a test set in a ratio of 9:1, and then the training set and the validation set are divided in a ratio of 9:1 in the training validation set.

[0012] In one embodiment of the present invention, the backbone feature network CSPDarknet53 uses a residual network Residual, in which the residual convolution is divided into two parts. The backbone part is a 1*1 convolution plus a 3*3 convolution; the residual side part directly combines the input and output of the backbone, which makes the network easy to optimize and can improve the accuracy by increasing the depth; in order to further improve the feature extraction capability of the backbone network, a Focus network structure is added to the backbone network, and a value is taken for every pixel of the input image to obtain four independent feature layers, and then the four independent feature layers are stacked. At this time, the width and height information is concentrated in the channel information, thereby expanding the input channel four times; the spliced ​​feature layers become twelve channels compared to the original three channels; In the feature fusion stage, CSPLayer is reconstructed using ResBlock. The reconstructed CSPRB module can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the gradient duplication problem; The Decoupled Head predicts pixel categories by adding one or more additional branches to the feature map of the backbone network.

[0013] The above technical scheme of the present invention has the following advantages over the prior art: the PCB defect detection system of the present invention truly establishes a machine vision system for automatically collecting and detecting defective images, and the effectiveness of the detection system has been verified on an actual PCB board; the present invention improves the YOLOX algorithm, and reduces the instability caused by single Mosaic data enhancement by optimizing data enhancement, thereby improving the accuracy of the model; the Focus structure is added to the backbone feature network of the present invention, and the channel information is effectively expanded; the present invention also introduces a Decoupled Head as one of the YOLO Heads, which improves the detection capability of tiny targets and can better process the image features of subtle defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0015] Figure 1 It is a structural schematic diagram of the image acquisition and processing module in the PCB defect detection system based on the improved YOLO algorithm of the present invention.

[0016] Figure 2 It is a flow chart of the PCB defect detection method based on the improved YOLO algorithm of the present invention.

[0017] Figure 3 It is the original sample image of the PCB data set described in the present invention.

[0018] Figure 4 The improved YOLOX algorithm structure described in the present invention Figure Ⅰ ; Figure 4-1 The improved YOLOX algorithm structure described in the present invention Figure II .

[0019] Figure 5 This is a defect detection result diagram described in the present invention.

[0020] As shown in the figure, 1. camera; 2. lens; 3. clamping device; 4. ring light source; 5. computer hardware; 6. base; 7. rectangular light source; 8. light source controller; 9. PCB sample. DETAILED DESCRIPTION

[0021] like Figure 1 As shown, this embodiment provides a PCB defect detection system based on an improved YOLO algorithm. The detection system adopts an improved YOLO algorithm, optimizes data enhancement, reduces the instability caused by single Mosaic data enhancement, and improves the accuracy of the model. It is characterized by comprising: an image acquisition processing module, a neural network model improvement module, a model training module, a model verification module, and a defect detection module; The image acquisition and processing module acquires images of PCB surface defects by building an image acquisition system, and processes the acquired images to obtain a data set; the image acquisition and processing module includes a camera, a lens, a clamping device, and a base, and a ring light source is also provided on the lens side. After the camera and the lens are assembled, they are arranged together on the clamping device, and the output interface of the camera is connected to the computer hardware; The neural network model improvement module, model training module, model verification module and defect detection module are all run on computer hardware as data modules. The neural network model improvement module is improved based on the YOLOX neural network to build a defect detection model. The improvement includes using the Focus structure to stack the feature layers and expand the input channels; ResBlock is used to reconstruct CSPLayer in the feature fusion stage. The reconstructed CSPRB module can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the gradient duplication problem; then spatial pyramid pooling is used to process the maximum pooling of four different scales, which greatly improves the receptive field; finally, Decoupled Head is used as YOLO Head; The model training module is used to set the network parameters of the defect detection model, input the prepared data set into the defect detection model for training, and use Adam as the optimizer in the training. The learning rate of the Adam optimizer is set to 0.001, the weight decay is set to 0.0005, the number of training epochs is set to 500, and the batch size is set to 4; The model verification module is used to use the verification set to test the defect detection model that has been initially trained, adjust the learning rate and epoch parameters according to the verification results, improve the verification accuracy, and obtain the final defect detection model; The defect detection module is used to input the test set data into the trained defect detection model, detect and mark the type, location and confidence of the PCB surface defects in the test set.

[0022] The lighting lens ring on the lens and the PCB sample to be tested on the base are on the same vertical straight line, and a rectangular light source and a light source controller are also provided on the base, and a control line is provided between the input end of the light source controller and the computer hardware; the camera is an industrial CCD camera, and a ring light source is provided, which is coaxially and vertically fixed above the PCB board to be tested, and a rectangular light source is placed below the PCB board. The defect image of the PCB surface is obtained by adjusting the distance between the camera, the light source and the PCB board and the exposure time.

[0023] like Figure 2 As shown in the flow chart of the PCB defect detection method, this embodiment provides a PCB defect detection method based on the improved YOLO algorithm, including building a PCB defect image acquisition system, acquiring PCB defect images and performing image processing and enhancement, data set preparation, training model construction, training model and generating weight files, reading weight files and testing test sets and the like.

[0024] Step 1: Build a PCB defect image acquisition system, such as Figure 1 As shown, the image acquisition processing module acquires images of PCB surface defects by building an image acquisition system, wherein the camera is perpendicular to the PCB sample, and the image is clear by adjusting the distance between the camera and the sample.

[0025] Step 2: Obtain PCB defect images and perform image processing and enhancement, and standardize the images collected in the first step. In the embodiment, there are many types of defects in the PCB production process. The defect types are mainly divided into six types: leaks, mouse bites, open circuits, short circuits, burrs, and fake copper. The size of the collected defective images is unified to 640*640. Image enhancement operation is performed on the image after the unified size. The collected defective image is first translated, flipped, and rotated for data expansion, and then the expanded image is subjected to image denoising and enhancement. The image is first subjected to denoising filtering by combining average filtering and smooth filtering, wherein moving is selected as the smoothing method of smooth filtering, and span is set to 5; the number of windows of average filtering is set to 4. This denoising method can effectively remove the flicker noise in the image without reducing the resolution of the image, and then the contrast-limited adaptive image enhancement is used to improve the contrast between the defect and the background in the image.

[0026] Step 3: Dataset production, the processed image dataset is first divided into a training validation set and a test set in a ratio of 9:1, and then the training set and the validation set are divided in a ratio of 9:1 in the training validation set. In the embodiment, there are 500 images of each defect in the dataset, and the entire dataset has a total of 3000 images, namely 2430 training sets, 270 validation sets, and 300 test sets. The image annotation software Labelimg is used to annotate the images in the dataset for defects, where the leak is named missing_hole, the mouse bite is named mouse_bite, the open circuit is named open_circuit, the short circuit is named short, the burr is named Spur, and the fake copper is named spurious_copper. For each annotated image Labelimg will generate a corresponding txt file, which contains the category information of the target bounding box and the position coordinates (x, y), width and height (w, h) of the center point of the target bounding box.

[0027] Step 4: Training model construction, constructing the YOLO neural network model. YOLOX uses a combination of Mosaic and Mixup to enhance the data. Mosaic first flips and scales the input image, and then arranges and splices the image so that the new image contains the previous annotation information. Mixup performs mixed enhancement on the image, mixing images of different types, thereby expanding the data set. In the original YOLOX model, the last 15 epochs will turn off data enhancement so that the model converges. However, in the embodiment, it is found that turning off data enhancement in the last 50 epochs will increase the accuracy of the model and improve the detection effect. Figure 4The improved YOLOX network structure diagram, the improved Y·OLOX network structure mainly includes the CSPDarknet53 backbone feature extraction network using the Focus structure, the spatial pyramid pooling structure SPP, the path aggregation network PAFPN, and the YOLO Head with the Decoupled Head. The Focus structure is added to the improved backbone feature extraction network CSPDarknet53. The Focus structure was first proposed in YOLOv5. The focus structure cuts the original input image and then splices the cut images. After splicing, a convolution is performed to change the size of the original input image from 640*640*3 to a feature image of 320*320*64. This structure can reduce the loss of the image when downsampling. In the original YOLOX model, CSPDarknet53 is mainly composed of CBS (convolution layer + normalization layer + activation function) and CSPLayer. The CSPLayer layer is also composed of CBS and residual blocks (composed of CBS). This approach can further deepen the depth of the neural network, thereby improving the feature extraction and feature fusion capabilities of the neural network. However, in actual use, it is found that the learning and expression ability of this model is weak when targeting small target defects. Therefore, the present invention reconstructs CSPLayer with Res2Block. Res2Block can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the gradient duplication problem. Figure 4 CSPRB_X is the reconstructed CSPLayer structure diagram. The SPP structure is to perform convolution on the features obtained in Dark4 and then perform maximum pooling processing. The feature extraction is performed through maximum pooling of different pooling kernel sizes, which greatly improves the receptive field; the Head detection head in the present invention adopts the Decoupled Head decoupling head structure, and the decoupling head can retain more detailed features of the target. The above operations can obtain more detailed features for small defects in PCB, thereby effectively enhancing the small target feature extraction and detection capabilities of the feature extraction network.

[0028] Step 5: Train the model and generate a weight file, and input the training set into the constructed training network structure model for training. First, set the initial network parameters of the defect detection model. Adam is used as the optimizer in training. The learning rate of the Adam optimizer is set to 0.001, the weight decay is set to 0.0005, the number of epochs for training is set to 500, and the batch size is set to 4. In the specific training process, the network first divides the input image into grids of different sizes, and each grid is independently responsible for different areas. If the center of the target to be detected is in a certain grid, the grid is responsible for detecting the target.

[0029] Step 6: Use the validation set images to verify the initially trained defect detection model, and adjust the network parameter settings according to the verification results, including learning rate, batch size, and weight decay. After modification, retrain until the verification results meet expectations.

[0030] Step 7: Read the weight file and test the test set. Use Python to write a script to read the weight file and input the prepared test set into the trained model for testing. The test result graph generated by the test will be saved in a separate folder. When all images in the test set are tested, the prompt "finish" will be displayed. The average detection time for each image is about 0.7 seconds. The test results are as follows Figure 5 shown.

[0031] When the PCB defect detection method proposed in this embodiment is applied to industrial production, the PCB image collected according to the method in step 1 is input into the trained improved YOLOX training network structure model, so that real-time surface defect recognition, classification and positioning can be achieved, which plays a huge role in the technicians' later troubleshooting of machine failures and improving production quality. The single target detection algorithm based on the YOLO series has achieved a higher detection speed at the expense of a small amount of accuracy and is widely used in industrial scenarios. Compared with other algorithms in the YOLO series, the improved YOLOX algorithm used in this patent adopts an anchor-free strategy. It no longer relies on pre-defined anchor boxes, but adaptively adjusts by predicting the center point and size of the object, thereby improving the flexibility of detection. Mosaic and Mixup are used to enhance the input data, which effectively improves the data diversity and the generalization ability of the model. In the feature fusion stage, CSPLayer is reconstructed with ResBlock. The reconstructed CSPRB module can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the problem of gradient duplication. SPP is added to the backbone network to increase the network receptive field. PANet is used to better fuse multi-layer features and improve the detection ability of the model in the case of small targets and complex backgrounds. And the decoupling head is used to improve the prediction accuracy.

[0032] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A PCB defect detection system based on an improved YOLO algorithm, wherein the detection system adopts an improved YOLO algorithm, optimizes data enhancement, reduces the instability caused by single Mosaic data enhancement, and improves the accuracy of the model, and is characterized in that: include: Image acquisition and processing module, neural network model improvement module, model training module, model verification module, defect detection module; The image acquisition and processing module acquires images of PCB surface defects by building an image acquisition system, and processes the acquired images to obtain a data set; the image acquisition and processing module comprises a camera (1), a lens (2), a clamping device (3), and a base (6); a ring light source (4) is also provided on the side of the lens (2); after the camera (1) and the lens (2) are assembled, they are arranged together on the clamping device (3); and the output interface of the camera (1) is connected to the computer hardware (5); The neural network model improvement module, model training module, model verification module, and defect detection module are all run on computer hardware (5) as data modules. The neural network model improvement module is improved based on the YOLOX neural network to construct a defect detection model. The improvement includes stacking feature layers using a Focus structure and expanding input channels. In the feature fusion stage, CSPLayer was reconstructed using ResBlock. The reconstructed CSPRB module can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the gradient duplication problem. Then, spatial pyramid pooling was used to process the maximum pooling of four different scales, which greatly improved the receptive field. Finally, Decoupled Head was used as YOLO Head. The model training module is used to set the network parameters of the defect detection model, input the prepared data set into the defect detection model for training, and use Adam as the optimizer in the training. The learning rate of the Adam optimizer is set to 0.001, the weight decay is set to 0.0005, the number of training epochs is set to 500, and the batch size is set to 4; The model verification module is used to use the verification set to test the defect detection model that has been initially trained, adjust the learning rate and epoch parameters according to the verification results, improve the verification accuracy, and obtain the final defect detection model; The defect detection module is used to input the test set data into the trained defect detection model, detect and mark the type, location and confidence of the PCB surface defects in the test set.

2. The PCB defect detection system according to claim 1, characterized in that: The light collecting lens ring on the lens (2) and the PCB sample (9) to be tested on the base (6) are on the same vertical straight line, and a rectangular light source (7) and a light source controller (8) are also provided on the base (6), and a control line is provided between the input end of the light source controller (8) and the computer hardware (5) to connect.

3. A PCB defect detection method based on an improved YOLO algorithm, based on the PCB defect detection system of claim 1, characterized in that: The steps include: Step S1: Building an image acquisition system to acquire images of defective samples; Step S2: Editing the acquired defect image to obtain an image of a uniform size; Step S3: Use data enhancement method to expand the sample, and then perform noise reduction and contrast enhancement on the image; Step S4: selecting image samples from the image, marking the locations and types of defects in the image samples, and creating a data set; Step S5: input the processed RGB three-channel image into the improved YOLOX neural network for training to obtain a target detection model for PCB panel defects; Step S6: Read the obtained target detection model and test the test set prepared in step S4 to achieve recognition, classification and location of PCB defects; Specifically, the target detection model in step S5 includes: a CSPDarknet53 backbone feature extraction network using a Focus structure, a spatial pyramid pooling structure SPP, a path aggregation network PAFPN, and a YOLOHead with a Decoupled Head.

4. The PCB defect detection method according to claim 3, characterized in that: The size of the image after resizing in step S2 is 640*640.

5. The PCB defect detection method according to claim 3, characterized in that: The data enhancement method described in step S3 is specifically to first translate, flip, and rotate the collected defect image for data expansion, and then perform image noise reduction and enhancement on the expanded image. First, the image is subjected to noise reduction filtering by combining average filtering and smooth filtering, wherein moving is selected as the smoothing processing method of smooth filtering, and span is set to 5; the number of windows of average filtering is set to 4.

6. The PCB defect detection method according to claim 3, characterized in that: The types of defects in step S4 include: leaks, rat bites, open circuits, short circuits, burrs, and fake copper. The image annotation tool LabelImg is used to annotate the standardized data set. Each annotated PCB image will generate a corresponding txt file. The content contained in the txt file includes: category information, center point position coordinates, width, and height of the target bounding box. The target bounding box is a labeled rectangular box that surrounds the defect.

7. The method for manufacturing the simplified trolley bearing improving device according to claim 3, characterized in that: In step S4, in order to ensure the independence of the data set, the data set is first divided into a training validation set and a test set in a ratio of 9:1, and then the training set and the validation set are divided in a ratio of 9:1 in the training validation set.

8. The method for manufacturing the simplified trolley bearing improving device according to claim 1, characterized in that: The main feature network CSPDarknet53 uses a residual network Residual, in which the residual convolution is divided into two parts. The main part is a 1*1 convolution plus a 3*3 convolution; the residual side part directly combines the input and output of the main part, which makes the network easy to optimize and can improve the accuracy by increasing the depth; in order to further improve the feature extraction ability of the main network, a Focus network structure is added to the main network, and a value is taken for every pixel of the input image to obtain four independent feature layers, and then the four independent feature layers are stacked. At this time, the width and height information is concentrated in the channel information, thereby expanding the input channel four times; the spliced ​​feature layers become twelve channels compared to the original three channels; In the feature fusion stage, CSPLayer is reconstructed using ResBlock. The reconstructed CSPRB module can better handle multi-scale feature fusion, obtain more semantic information, and alleviate the gradient duplication problem; The Decoupled Head predicts pixel categories by adding one or more additional branches to the feature map of the backbone network.

Images