PCB tiny defect detection method based on deep learning
By improving the YOLOv8 network model, combining data enhancement, improved FPN and PPA modules, using the EIOU_loss loss function, the accuracy and efficiency problems of PCB board micro defect detection in traditional methods are solved, and higher detection accuracy and regression rate are achieved.
Patent Information
- Application Number
- CN202510465175.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-15
- Publication Date
- 2025-07-18
AI Technical Summary
The traditional PCB board micro defect detection methods have problems such as low detection accuracy, low efficiency and poor robustness, especially in complex scenarios, which are difficult to effectively identify micro defects.
The improved YOLOv8 network model is adopted to enhance data, introduce improved FPN networks suitable for small target features, parallel patch aware attention module (PPA), and use the EIOU_loss loss function to improve feature information extraction and fusion capabilities, improve detection accuracy and regression rate.
It improves the accuracy and regression rate of micro defect detection of PCB boards, improves the accuracy and efficiency of detection, and is suitable for micro defect recognition in complex scenarios.
Smart Images
Figure CN120339242A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of PCB board defect detection, and particularly relates to a method for detecting minute defects on a PCB board based on deep learning. Background Art
[0002] In the electronics manufacturing industry, minute defects on printed circuit boards (PCBs), such as common defects like missing_hole, mouse_bite, open_circuit, short, spur on the board surface, spurious_copper, etc., are difficult to be detected with high accuracy and efficiency by traditional detection methods. To improve production efficiency and reliability, the industry needs a fast and accurate detection method. Identifying and locating minute defects on printed circuit boards (PCBs) through computer vision and deep learning technologies can better meet the detection requirements. This technology generally involves steps such as image processing, feature extraction, model training, and object detection. Nowadays, electronic products are developing rapidly, and the complexity and integration level of PCBs are also constantly increasing. Traditional manual detection methods are no longer able to meet the requirements of efficient and accurate detection.
[0003] Regarding the defect detection on the surface of PCB boards, currently commonly used methods mainly include traditional detection methods based on digital image processing and object detection methods based on deep learning. The detection accuracy based on digital image processing depends on the professional knowledge and experience accumulation of technicians, with poor versatility, low efficiency, and weak robustness. While object detection algorithms can enable detection with good accuracy, speed, and higher efficiency, there is still room for improvement in PCB defect detection based on deep learning, such as insufficient learnable data required for deep learning, differences between training data and actual application scenarios, difficulty in obtaining information in complex scenarios, and difficulty in detecting minute defects, resulting in relatively low accuracy of defect detection. Summary of the Invention
[0004] (1) Technical Problems to be Solved
[0005] The purpose of the present invention is to provide a method for detecting minute defects on a PCB board based on deep learning. After improving the traditional YOLOv8 network model, the method enhances the accuracy and regression rate of detecting minute defects on the PCB board, and is more suitable for the detection and classification of minute defects on the PCB board.
[0006] (2) Technical Solutions
[0007] To achieve the above purpose, the present invention adopts the following technical solutions:
[0008] A method for detecting tiny defects on PCB boards based on deep learning, comprising the following steps:
[0009] Step 1: Obtain the original dataset, perform data augmentation on the dataset to obtain the augmented dataset; the pictures in the augmented dataset include the original pictures and the augmented pictures;
[0010] Step 2: Annotate the augmented dataset; divide the augmented dataset into a training set and a test set;
[0011] Step 3: Improve the original YOLOv8 network model;
[0012] Step 4: Apply the augmented training dataset to train the improved YOLOv8 network model;
[0013] Step 5: Apply the augmented test dataset to verify the detection performance of the improved YOLOv8 network model;
[0014] Among them, the improved YOLOv8 network model includes a backbone network, a neck network, and a detection head network; the backbone network includes a first convolutional layer, a second convolutional layer, a first C2F module, a third convolutional layer, a second C2F module, a fourth convolutional layer, a third C2F module, a fifth convolutional layer, a fourth C2F module, and a focus modulation module connected in sequence; a first PPA module, a second PPA module, and a third PPA module are respectively introduced after the first C2F module, the second C2F module, and the third C2F module, and are spliced in the neck network;
[0015] The neck network includes a first upsampling module, a first splicing module, a fifth C2F module, a second upsampling module, a second splicing module, a sixth C2F module, a third upsampling module, a third splicing module, a seventh C2F module, a sixth convolutional layer, a fourth splicing module, an eighth C2F module, a seventh convolutional layer, a fifth splicing module, and a ninth C2F module; the focus modulation module is connected to the first upsampling module, and the first splicing module splices the features output by the third PPA module and the features output by the first upsampling module and inputs them into the fifth C2F module. After being output by the fifth C2F module, it is input into the second upsampling module. The features output by the second PPA module and the second upsampling module are spliced by the second splicing module and then input into the sixth C2F module. After being output by the sixth C2F module, it is input into the third upsampling module. The features output by the first PPA module and the third upsampling module are spliced by the third splicing module and then input into the seventh C2F module. After being output by the seventh C2F module, it is input into the sixth convolutional layer; the features output by the sixth convolutional layer and the features output by the sixth C2F module are spliced by the fourth splicing module and then input into the eighth C2F module. After being output by the eighth C2F module, it is input into the seventh convolutional layer; the features output by the seventh convolutional layer and the features output by the fifth C2F module are spliced by the fifth splicing module and then input into the ninth C2F module; the features output by the seventh C2F module, the eighth C2F module, and the ninth C2F module are input into the detection head network;
[0016] The improved FPN includes introducing the output features of the first C2F module in the backbone network into the neck network and splicing them with the output features of the third upsampling in the neck network through the third splicing module; and introducing a detection head after the seventh C2F module to replace the deepest detection head of YOLOv8;
[0017] The PPA module includes an input feature map, a PW convolutional module, a first convolutional module, a second convolutional module, a third convolutional module, a splicing module, an attention module, and an output feature map connected in sequence; in addition, the output of the PW convolutional module is respectively input into a local feature module and a global feature module. The output features of the local feature module, the output features of the global feature module, the output features of the first convolutional module, the output features of the second convolutional module, and the output features of the third convolutional module are spliced by the splicing module and then input into the attention module;
[0018] The local feature module and the global feature module include obtaining a set of spatially continuous patches by unfolding and reshaping the output features of PW convolution, followed by channel averaging, then performing linear calculations through FFN and applying an activation function to obtain the probability distribution of the spatial dimension of the linearly calculated features, adjusting their weights, and finally outputting the final features after reshaping and interpolation of the features by the feature selection module. The local and global feature modules are obtained by changing the aggregation and displacement of non-overlapping patches in the spatial dimension to change the parameter p.
[0019] The loss function of the detection head network uses the EIOU_loss loss function. The EIOU_loss loss function model separates the influence factors of the aspect ratios of the predicted box and the ground truth box on the basis of the penalty term of CIOU, calculates the length and width of the predicted box and the ground truth box respectively, and adds Focal to focus on high-quality anchor boxes. The EIOU loss function is defined as follows:
[0020]
[0021] where w c and h c are the width and height of the smallest bounding box covering the two boxes. The loss function is divided into three parts: IOU loss L IOU , distance loss L dis and orientation loss L asp .
[0022] Furthermore, the data augmentation in step 1 refers to performing operations such as rotation, translation, scaling, mirroring, cropping, and splicing on the original images in the PCB dataset to achieve data augmentation; and finally converting them into a YOLO format dataset.
[0023] Furthermore, the data annotation in step 2 is to annotate the augmented dataset through Labelimg software, and make the annotated dataset into a training dataset and a test dataset for training and improving the YOLOv8 network model.
[0024] (III) Beneficial Effects
[0025] The present invention replaces the original FPN network with an improved FPN network suitable for small target feature information extraction, strengthens the ability of the network model to extract small target information features, introduces a PPA parallel patch perception attention module between the backbone network and the neck network to strengthen the feature information fusion ability of the network model, and uses the EIOU_loss loss function to replace the CIOU_loss loss function of the original network model, which can more accurately locate the targets in the image, improve the robustness of the network model, and at the same time improve the regression performance of the network model. The present invention can improve the regression rate of micro defect detection while ensuring the improvement of the detection accuracy of micro defects on the PCB board. Brief Description of Drawings
[0026] Figure 1 It is a schematic flowchart of the present invention;
[0027] Figure 2 It is a schematic diagram of the network structure of the present invention;
[0028] Figure 3 It is a schematic diagram of the improved FPN structure of the present invention;
[0029] Figure 4 It is a schematic diagram of the PPA module structure of the present invention. Detailed Description of the Invention
[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.
[0031] Embodiment
[0032] As Figure 1 shown, a method for detecting minute defects on PCB boards based on deep learning provided in this embodiment includes the following steps:
[0033] Step 1: Obtain the original data set, perform data augmentation on the data set to obtain the augmented data set; the pictures in the augmented data set include the original pictures and the augmented pictures;
[0034] Step 2: Annotate the augmented data set; divide the augmented data set into a training set and a test set;
[0035] Step 3: Improve on the basis of the original YOLOv8 network model;
[0036] Step 4: Use the augmented training data set to train the improved YOLOv8 network model;
[0037] Step 5: Use the augmented test data set to verify the detection performance of the improved YOLOv8 network model;
[0038] The structure of the improved YOLOv8 network model in this embodiment is as Figure 2 shown, and it includes a backbone network, a neck network, and a detection head network.
[0039] In this embodiment, the FPN network in the backbone network is replaced with an improved FPN network, which enhances the ability of the network model to extract the feature information of small targets. The backbone network includes a first convolutional layer, a second convolutional layer, a first C2F module, a third convolutional layer, a second C2F module, a fourth convolutional layer, a third C2F module, a fifth convolutional layer, a fourth C2F module, and a focus modulation module connected in sequence;
[0040] In this embodiment, the PPA parallel patch-aware attention module is introduced between the backbone network and the neck network to enhance the feature information fusion ability of the network model. The first PPA module, the second PPA module, and the third PPA module are respectively introduced after the first C2F module, the second C2F module, and the third C2F module and are spliced in the neck network. The neck network includes a first upsampling module, a first splicing module, a fifth C2F module, a second upsampling module, a second splicing module, a sixth C2F module, a third upsampling module, a third splicing module, a seventh C2F module, a sixth convolutional layer, a fourth splicing module, an eighth C2F module, a seventh convolutional layer, a fifth splicing module, and a ninth C2F module; The focus modulation module is connected to the first upsampling module. The first splicing module splices the features output by the third PPA module and the features output by the first upsampling module and inputs them into the fifth C2F module. After being output by the fifth C2F module, they are input into the second upsampling module. The features output by the second PPA module and the second upsampling module are spliced by the second splicing module and then input into the sixth C2F module. After being output by the sixth C2F module, they are input into the third upsampling module. The features output by the first PPA module and the third upsampling module are spliced by the third splicing module and then input into the seventh C2F module. After being output by the seventh C2F module, they are input into the sixth convolutional layer; The features output by the sixth convolutional layer and the features output by the sixth C2F module are spliced by the fourth splicing module and then input into the eighth C2F module. After being output by the eighth C2F module, they are input into the seventh convolutional layer; The features output by the seventh convolutional layer and the features output by the fifth C2F module are spliced by the fifth splicing module and then input into the ninth C2F module; The features output by the seventh C2F module, the eighth C2F module, and the ninth C2F module are input into the detection head network;
[0041] The improved FPN of this embodiment is as Figure 3 shown, replacing the FPN network of the original network model. The improved FPN includes introducing the output features of the first C2F module in the backbone network into the neck network and splicing them with the output features of the third upsampling in the neck network through the third splicing module; And a detection head is introduced after the seventh C2F module to replace the deepest detection head of YOLOv8;
[0042] The PPA parallel patch-aware attention module of this embodiment is as Figure 3As shown, the PPA module includes an input feature map, a PW convolution module, a first convolution module, a second convolution module, a third convolution module, a splicing module, an attention module, and an output feature map, which are connected in sequence. In addition, the output of the PW convolution module is respectively input into a local feature module and a global feature module. The output features of the local feature module, the output features of the global feature module, the output features of the first convolution module, the output features of the second convolution module, and the output features of the third convolution module are spliced by the splicing module and then input into the attention module. The local feature module and the global feature module include obtaining a set of spatially continuous patches by unfolding and reshaping the output features of the PW convolution, then performing channel averaging, and then performing linear calculations through FFN and applying an activation function to obtain the probability distribution of the spatial dimension of the linear calculation features and adjusting their weights. Finally, the features are reshaped and interpolated by the feature selection module to output the final features. By changing the aggregation and displacement of non-overlapping patches in the spatial dimension and changing the parameter p, the local and global feature modules are respectively obtained.
[0043] In this embodiment, the network structure of the detection head network remains unchanged, and only the loss function in the network structure is changed. In this embodiment, the EIOU_loss loss function is used to replace the CIOU_loss loss function of the source network model. The loss function of the detection head network uses the EIOU_loss loss function. The EIOU_loss loss function model separates the influence factors of the aspect ratios of the predicted box and the ground truth box on the basis of the penalty term of CIOU, calculates the length and width of the predicted box and the ground truth box respectively, and adds Focal to focus on high-quality anchor boxes. The EIOU loss function is defined as follows:
[0044]
[0045] where w c and h c are the width and height of the smallest bounding box covering the two boxes. The loss function is divided into three parts: IOU loss L IOU , distance loss L dis and orientation loss L asp .
[0046] Data augmentation in this embodiment refers to performing operations such as rotation, translation, scaling, mirroring, cropping, and splicing on the original images in the PCB dataset to achieve data augmentation; finally, it is converted into a YOLO format dataset.
[0047] Data annotation in this embodiment is to annotate the augmented dataset through Labelimg software, and make the annotated dataset into a training dataset and a test dataset for training and improving the YOLOv8 network model.
[0048] The method of this embodiment is compared with the traditional YOLOv8 network model for detecting PCB micro-defects. The size of the pictures collected in the dataset is size = 640 pixels × 640 pixels; 6 types of defects are labeled, namely solder joint notch, mouse bite, open circuit, short circuit, burr on the board surface, and redundant copper, and the corresponding category labels are: 0, 1, 2, 3, 4, 5, and the corresponding names are: missing_hole, mouse_bite, open_circuit, short, spur, spurious_copper. The division ratio of the training set to the validation set is 9:1; and the data is saved in the YOLO format; the monitoring network includes the original network, YOLOv8 + improved FPN network, YOLOv8 + PPA network, and the network model of this embodiment; this embodiment is YOLOv8 + improved FPN + PPA; among them, the network setting parameters are shown in Table 1.
[0049] Table 1 Unified parameter settings
[0050]
[0051] The device parameters used in this embodiment are as follows: Processor: Intel Core i5-12400F 4.40GHz; GPU graphics card information: NVIDIA GeForce RTX 2080Ti; Python interpreter information: Python3.8, Torch1.12.1; The accuracy rate, regression rate, map50, and map50-90 are used to evaluate the detection results of the network model, and the comparison results of the ablation experiments are shown in Table 2.
[0052] Table 2 Evaluation table of the detection results of the network model
[0053]
[0054] The results show that the accuracy rate of the network model of this embodiment has increased by 4.8% compared with the original YOLOv8 network model, the regression rate has increased by 1%, map50 has increased by 3%, and map50-95 has increased by 4.9%, which proves that the improved YOLOv8 network model of this application has improved in both the detection accuracy and regression rate of PCB board micro-defects.
[0055] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A method for detecting tiny defects on a PCB board based on deep learning, characterized in that, Including: Obtain PCB board data in real time, transfer the obtained PCB board data to the trained improved YOLOv8 network model, and obtain the micro defect detection results; The process of training the YOLOv8 network model includes: Step 1: Obtain the original dataset, perform data augmentation on the dataset to obtain the augmented dataset; the pictures in the augmented dataset include the original pictures and the augmented pictures; Step 2: Annotate the augmented dataset; divide the augmented dataset into a training set and a test set; Step 3: Improve the original YOLOv8 network model; Step 4: Use the augmented training dataset to train the improved YOLOv8 network model; Step 5: Use the augmented test dataset to verify the detection performance of the improved YOLOv8 network model; Among them, the improved YOLOv8 network model includes a backbone network, a neck network, and a detection head network; The backbone network includes a first convolutional layer, a second convolutional layer, a first C2F module, a third convolutional layer, a second C2F module, a fourth convolutional layer, a third C2F module, a fifth convolutional layer, a fourth C2F module, and a focus modulation module connected in sequence; a first PPA module, a second PPA module, and a third PPA module are respectively introduced after the first C2F module, the second C2F module, and the third C2F module, and are spliced in the neck network; The neck network includes a first upsampling module, a first splicing module, a fifth C2F module, a second upsampling module, a second splicing module, a sixth C2F module, a third upsampling module, a third splicing module, a seventh C2F module, a sixth convolutional layer, a fourth splicing module, an eighth C2F module, a seventh convolutional layer, a fifth splicing module, and a ninth C2F module; the focus modulation module is connected to the first upsampling module, and the first splicing module splices the features output by the third PPA module and the features output by the first upsampling module and inputs them into the fifth C2F module. After being output by the fifth C2F module, it is input into the second upsampling module. The features output by the second PPA module and the second upsampling module are spliced by the second splicing module and then input into the sixth C2F module. After being output by the sixth C2F module, it is input into the third upsampling module. The features output by the first PPA module and the third upsampling module are spliced by the third splicing module and then input into the seventh C2F module. After being output by the seventh C2F module, it is input into the sixth convolutional layer; the features output by the sixth convolutional layer and the features output by the sixth C2F module are spliced by the fourth splicing module and then input into the eighth C2F module. After being output by the eighth C2F module, it is input into the seventh convolutional layer; the features output by the seventh convolutional layer and the features output by the fifth C2F module are spliced by the fifth splicing module and then input into the ninth C2F module; the features output by the seventh C2F module, the eighth C2F module, and the ninth C2F module are input into the detection head network; The improved FPN includes introducing the output features of the first C2F module in the backbone network into the neck network, and splicing the output features of the third upsampling through the third splicing module in the neck network; and introducing a detection head after the seventh C2F module to replace the YOLOv8 deep detection head; The YOLOv8 deep detection head is the detection head network derived from the last layer of C2F module in the original YOLOv8 network; The PPA module includes an input feature map, a PW convolution module, a first convolution module, a second convolution module, a third convolution module, a splicing module, an attention module, and an output feature map connected in sequence; in addition, the output of the PW convolution module is respectively input into the local feature module and the global feature module, and the output features of the local feature module, the output features of the global feature module, the output features of the first convolution module, the features output by the second convolution module, and the features output by the third convolution module are spliced through the splicing module and then input into the attention module; The local feature module and the global feature module include: Unfolding and reshaping the output features of the PW convolution to obtain a set of spatial continuous patches, then performing channel averaging, and then applying an activation function to obtain the probability distribution of the spatial dimension of the linear calculation features after linear calculation through FFN, and adjusting its weight, and finally reshaping and interpolating the features through the feature selection module to output the final features, and respectively obtaining the local and global feature modules by changing the aggregation and displacement parameter p of non-overlapping patches in the spatial dimension; The loss function of the detection head network adopts the EIOU_loss loss function. The EIOU_loss loss function model separates the influencing factors of the aspect ratio of the predicted box and the real box based on the penalty term of CIOU, calculates the length and width of the predicted box and the real box respectively, and adds the Focal focus on the high-quality anchor box; the EIOU loss function is defined as follows: where w c and h c are the width and height of the smallest bounding box covering the two boxes. The loss function is divided into three parts: the IOU loss L IOU , the distance loss L dis and the orientation loss L asp .
2. The method for detecting minute defects on a PCB board based on deep learning according to claim 1, wherein, The data enhancement described in step 1 refers to performing operations such as rotation, translation, scaling, mirroring, cropping, and splicing on the original images in the PCB dataset to achieve data enhancement; finally, the data is converted into a YOLO format dataset.
3. A method for detecting minute defects on a PCB board based on deep learning according to claim 1, characterized in that, The data annotation described in step 2 is to annotate the enhanced data set through Labelimg software, and use the annotated data set to make a training data set and a test data set for training the improved YOLOv8 network model.