PCB surface tiny defect detection method based on image feature enhancement

Abstract

The invention discloses a PCB surface tiny defect detection method based on image feature enhancement, and the method specifically comprises the steps: obtaining a PCB defect image, and marking the type of the defect image; optimizing a PCB defect detection algorithm based on an RT-DETR model, and improving the PCB defect detection algorithm into Lite-DETR; a PCB defect image training set is adopted to train the improved algorithm model; and transmitting a defect image test set into the trained model, recording a detection result and evaluating the performance of the model. According to the method, the accuracy and the model complexity can be effectively balanced, and a reliable solution is provided for PCB surface tiny defect detection in a complex industrial environment.

Description

Technical Field

[0001] The present invention belongs to the field of quality inspection in the electronic information industry, and particularly relates to a method for detecting minute defects on the surface of a PCB based on image feature enhancement. Background Art

[0002] Printed circuit boards (PCBs) are an indispensable core component in modern electronic information products, and are widely used in fields such as smart phones, household appliances, medical diagnosis, transportation, and the Internet of Things. High-quality PCBs are crucial for the normal operation of electronic devices, so the detection of defects on the surface of PCBs is particularly critical. With the miniaturization of modern electronic products, PCB designs tend to have more compact and dense wiring. However, the precision of PCBs has made defect features extremely minute, with most defects accounting for only 0.005%-0.07% of the entire image, and the defects are highly similar in color to the background, and it is also difficult to distinguish between different defect categories. Traditional manual visual inspection methods are no longer applicable. In recent years, machine vision inspection methods represented by Automatic Optical Inspection (AOI) have gradually replaced manual visual inspection. Although AOI technology has improved the accuracy of defect detection, it still has limitations such as being vulnerable to environmental interference and only being able to detect simple defects. The rapid development of Deep Learning (DL) has opened up broad application prospects for PCB defect detection.

[0003] Currently, deep learning methods are mainly divided into two categories: those based on convolutional neural networks and those based on Transformers. The former mainly includes RCNN, Fast-RCNN, Faster-RCNN, and the YOLO series. These detectors have become relatively mature and have achieved good results in industrial defect detection. However, they rely on post-processing operations such as Non-maximum Suppression (NMS), which affects the detection speed and cannot achieve end-to-end detection. In addition, due to limited multi-scale feature processing capabilities, these methods cannot effectively capture the multi-scale features of PCB defects. Detectors based on Transformers effectively solve these problems.

[0004] Transformer-based detectors no longer rely on post-processing operations such as NMS, thus significantly simplifying the object detection process. The latest Real-time Object Detection (RT-DETR) model has outperformed the mature YOLO series in terms of performance, achieving true "end-to-end" real-time detection. In the task of PCB surface defect detection in industrial scenarios, RT-DETR has shown significant potential. However, RT-DETR still has some limitations, including insufficient ability to capture the detailed features of tiny defects, high model complexity, and weak generalization ability. Therefore, how to effectively balance accuracy and model complexity has become the key to the practical feasibility of RT-DETR in industrial applications. Summary of the Invention

[0005] In order to achieve high detection accuracy, the present invention provides a method for detecting tiny defects on the PCB surface based on image feature enhancement.

[0006] A method for detecting tiny defects on the PCB surface based on image feature enhancement according to the present invention is characterized by comprising the following steps:

[0007] Step 1: Obtain a PCB defect image and annotate the defect image.

[0008] Step 2: Optimize the PCB defect detection algorithm based on the Real-time Object Detection (RT-DETR) model and improve it to Lite-DETR.

[0009] Step 3: Use the PCB defect image training set to train the improved algorithm model.

[0010] Step 4: Input the defect image test set into the trained model, record the detection results, and evaluate the model performance.

[0011] Further, Step 1 is specifically to obtain a public dataset, use the Labelimg tool to annotate the dataset images, and generate label files in TXT format.

[0012] Further, Lite-DETR is divided into three stages: feature extraction, feature fusion, and prediction:

[0013] In the feature extraction stage, the image dataset is input into the feature extraction network mainly based on ResNet18. By optimizing the residual structure of ResNet18, more local detailed information of PCB defects is retained while reducing the computational cost. And a Context Guided block is introduced, and a lightweight Basic-CG block is constructed to enhance the model's attention to the context information of defect features and improve the model's ability to extract tiny defect features.

[0014] In the feature fusion stage, the model introduces a Selective Boundary Aggregation (SBA) block to construct a refined cross-scale feature fusion structure, and performs bidirectional fusion on the feature layers (P2, P3, P4) in the feature extraction backbone with the help of SBA. SBA can selectively aggregate the boundary information of shallow features and the semantic information of deep features, effectively avoiding information redundancy and inconsistency problems in the feature fusion process. Through the feature fusion process, the model can extract rich features from information at different scales and levels, thereby improving the detection effect and efficiency, and is applicable to PCB defect detection in complex background environments.

[0015] In the prediction stage, the model converts the image features in the feature extraction and fusion stages into the final object detection results, and completes the prediction of defect categories and the regression of bounding boxes.

[0016] Further, step 3 is specifically as follows: After improving the RT-DETR model, train the Lite-DETR model: Set the size of the input image to 640×640 pixels, select AdamW as the optimizer, set the initial learning rate to 0.0002, the number of iterations is 200 epochs, the batch size is set to 8, and use WMPDIoU as the loss function to reduce the sensitivity to feature scale changes, make the prediction box fit the true box more accurately, and accelerate the convergence process of the model in the feature extraction stage.

[0017] Further, in step 4, the model performance metrics are recall, precision, mean average precision (mAP), and model size (measured by the number of model parameters). They are respectively expressed as:

[0018] The definition of precision is the ratio of the number of samples predicted as positive by the model to the number of samples that are actually positive among them.

[0019]

[0020] Among them, TP (True Positive) represents the number of instances correctly predicted as the target, and FP (False Positive) is the number of instances wrongly predicted as the target.

[0021] The definition of recall is the ratio of the number of samples that are actually positive to the number of samples accurately identified as positive by the model, as shown in formula (2):

[0022]

[0023] Among them, FN (False Negative) is the number of instances that are actually the target but not correctly predicted.

[0024] AP represents the area enclosed by the Precision-Recall curve (i.e., the P-R curve) in the interval [0, 1]. The P-R curve is drawn by calculating the values of multiple groups of Precision and Recall respectively based on the prediction results of the model. The formula is as follows:

[0025]

[0026] Among them, AP is an indicator for evaluating the detection performance of a single category. The higher its value, the better the performance of the classifier.

[0027] The mean average precision mAP is an evaluation indicator for the classification detection accuracy of multi-class problems. The mean average precision will be calculated under different Intersection over Union (IoU) thresholds, where n is the number of all categories in the dataset.

[0028]

[0029] The model size is directly related to the hardware configuration on the production line site. Considering that there are generally only some ordinary computing resources on the production line site, this indicator is very crucial.

[0030] The beneficial technical effects of the present invention compared with the prior art.

[0031] The present invention analyzes the characteristics of tiny PCB surface defects that are difficult to distinguish from the background. By improving the RT-DETR model, an algorithm model Lite-DETR for detecting tiny defects on the PCB surface based on image feature enhancement is proposed. A lightweight and efficient backbone network (LEBN) is designed based on the optimized ResNet-18, and the Context Guided block is combined to effectively retain more detailed features of tiny defects and strengthen the attention to the context information of defects. Secondly, an Image Feature Augmentation Module (IFAM) is proposed to improve the generalization of the model to data. Finally, the improved Refined Cross-Scale Feature Fusion Module (RCCFM) is used to selectively transmit information between features, enabling the model to effectively capture the multi-scale features of defects. At the same time, the WMPDIoU loss function that combines the normalized Wasserstein distance and MPDIoU is used to optimize the position matching accuracy between the predicted box and the ground truth box. The experimental results on the relevant public dataset show that the accuracy of Lite-DETR is better than that of various advanced defect detection models, and the number of parameters is 5.22M, which is reduced by 74% compared with the original model, effectively balancing accuracy and model complexity, and providing a reliable solution for tiny defect detection in complex industrial environments. Description of the Drawings

[0032] Figure 1 It is the framework diagram of the Lite-DETR model proposed by the present invention.

[0033] Figure 2 It is the structural diagram of the Context Guided block.

[0034] Figure 3 It is the structural diagram of the Image Feature Augmentation Module (IFAM).

[0035] Figure 4 It is the network structural diagram of the Selective Boundary Aggregation (SBA) block.

[0036] Figure 5 It is the detection effect diagram of the experimental HRIPCB dataset ((a) short circuit; (b) open circuit; (c) burr; (d) hole; (e) rat bite; (f) residual copper).

[0037] Figure 6For the experimental detection effect diagram of the Deep PCB dataset. Specific implementation method

[0038] The following further elaborates on the present invention in conjunction with the attached drawings and specific implementation methods.

[0039] A method for detecting tiny defects on the surface of a PCB based on image feature enhancement according to the present invention is characterized by including the following steps:

[0040] Step 1: Obtain a PCB defect image and label the defect image.

[0041] Obtain a public dataset, and use the Labelimg tool to label the dataset images to generate label files in TXT format.

[0042] Step 2: Optimize the PCB defect detection algorithm based on the Real-time Object Detection (RT-DETR) model and improve it to Lite-DETR.

[0043] The framework diagram of Lite-DETR is as Figure 1 shown, and it is divided into three stages: feature extraction, feature fusion, and prediction:

[0044] In the feature extraction stage, the image dataset is input into a lightweight and efficient backbone network (LEBN), which retains more local detail information of PCB defects while reducing the computational cost. And introduce a context-guided block (Context Guided), and construct a lightweight Basic-CG block to enhance the model's attention to the context information of defect features and improve the model's ability to extract tiny defect features.

[0045] In the feature fusion stage, the model selectively transmits information between features through a refined cross-scale feature fusion module (RCCFM), enabling the model to effectively capture multi-scale features of defects. Through the feature fusion process, the model can extract rich features from information at different scales and different levels, thereby improving the detection effect and efficiency, and being applicable to the detection of PCB defects in complex background environments.

[0046] In the prediction stage, the model converts the image features that have gone through the feature extraction and fusion stages into the final object detection results, completing the prediction of defect categories and the regression of bounding boxes.

[0047] The optimization is specifically as follows:

[0048] (1) A lightweight and efficient backbone network (LEBN) was designed, combined with a Context Guided block. The Context Guided block contains a standard 3×3 convolutional layer and a dilated 3×3 convolutional layer. Finally, a global average pooling layer is used to aggregate the global background, as Figure 2 shown. More detailed features of tiny defects are retained during the feature extraction process, and the attention to the context information of defect features is strengthened, thereby enhancing the model's ability to capture tiny defect features.

[0049] (2) An Image Feature Augmentation Module (IFAM) was designed. The structure of image feature augmentation includes horizontal flipping by 180 degrees, vertical flipping by 180 degrees, and ROI cropping, as Figure 3 shown. This method performs simple and efficient augmentation operations (horizontal flipping, vertical flipping, ROI cropping) on the feature maps extracted by LEBN, effectively retaining the important spatial structure information of the features, enhancing the model's adaptability to diverse features, thereby improving the model's generalization ability, and at the same time avoiding an increase in data processing complexity.

[0050] (3) A Selective Boundary Aggregation (SBA) block was introduced to construct a refined cross-scale feature fusion structure. The Selective Boundary Aggregation block mainly contains two Recalibrated Attention Units (RAU). The Recalibrated Attention Unit solves the feature redundancy problem through multiple dot product, negation, and splicing operations. Bidirectional fusion of the feature layers (P2, P3, P4) in LEBN is performed with the help of SBA. As Figure 4 shown, SBA can selectively aggregate the boundary information of shallow features and the semantic information of deep features, effectively avoiding information redundancy and inconsistency problems during the feature fusion process.

[0051] (4) By using a loss function of WMPDIoU that combines the normalized Wasserstein distance and MPDIoU, the model can reduce the sensitivity to feature scale changes, make the predicted bounding box fit the ground truth box more precisely, and accelerate the model's convergence process.

[0052] Step 3: The improved algorithm model was trained using the PCB defect image training set.

[0053] After improving the RT-DETR model, the Lite-DETR model is trained: the size of the input image is set to 640×640 pixels, AdamW is selected as the optimizer, the initial learning rate is set to 0.0002, the number of iterations is 200 epochs, and the batch size is set to 8.

[0054] Step 4: Input the defect image test set into the trained model, record the detection results, and evaluate the model performance.

[0055] The model performance metrics are recall, precision, mean average precision (mAP), and model size (measured by the number of model parameters). They are respectively expressed as:

[0056] The definition of precision is the ratio of the number of samples predicted as positive by the model to the number of samples that are actually positive among them.

[0057]

[0058] Among them, TP (True Positive) represents the number of instances correctly predicted as the target, and FP (False Positive) is the number of instances incorrectly predicted as the target.

[0059] The definition of recall is the ratio of the number of samples that are actually positive to the number of samples accurately identified as positive by the model, as shown in formula (2):

[0060]

[0061] Among them, FN (False Negative) is the number of instances that are actually the target but not correctly predicted.

[0062] AP represents the area enclosed by the precision-recall curve (i.e., the P-R curve) in the interval [0,1]. The P-R curve is drawn by calculating multiple sets of precision and recall values through the prediction results of the model. The formula is as follows:

[0063]

[0064] Among them, AP is an indicator for evaluating the detection performance of a single category. The higher its value, the better the performance of the classifier;

[0065] The mean average precision mAP is an evaluation indicator for the classification detection accuracy of multi-class problems. The mean average precision will be calculated under different intersection over union (IoU) thresholds, where n is the number of all categories in the dataset;

[0066]

[0067] The model size is directly related to the hardware configuration on the production line site. Considering that there are only some ordinary computing resources on the general production line site, this indicator is very crucial.

[0068] By testing on the HRIPCB dataset and the Deep PCB dataset, the method proposed in the present invention is compared with a variety of advanced defect detection models. The comparison results on the HRIPCB dataset are shown in Table 1, and the comparison results on the Deep PCB dataset are shown in Table 2. The detection effect of the method of the present invention is as Figure 5 and Figure 6 shown.

[0069] Table 1 Comparative experiments on the HRIPCB dataset

[0070]

[0071] Table 2 Comparative experiments on the Deep PCB dataset

[0072]

[0073]

[0074] It can be seen that the present method shows high detection accuracy on both the HRIPCB dataset and the Deep PCB dataset.

Claims

1. A method for detecting minute defects on the surface of a PCB based on image feature enhancement, characterized in that It includes the following steps: Step 1: Obtain PCB defect images and label the defect image categories; Step 2: Optimize the PCB defect detection algorithm based on the RT-DETR model and improve it to Lite-DETR; Step 3: Use the PCB defect image training set to train the improved algorithm model; Step 4: Input the defect image test set into the trained model, record the detection results and evaluate the model performance.

2. The method for detecting minute defects on the surface of a PCB based on image feature enhancement according to claim 1, wherein, Specifically, in Step 1, obtain a public dataset, use the Labelimg tool to label the dataset images, and generate label files in TXT format.

3. A method for detecting minute defects on the surface of a PCB based on image feature enhancement according to claim 1, characterized in that, The Lite-DETR is divided into three stages: feature extraction, feature fusion, and prediction: In the feature extraction stage, the image dataset is input into the feature extraction network mainly based on ResNet18. By optimizing the residual structure of ResNet18, more local detail information of PCB defects is retained; and a Context Guided block is introduced, and a lightweight Basic-CG block is constructed; In the feature fusion stage, the model introduces a Selective Boundary Aggregation block (SBA) to construct a refined cross-scale feature fusion structure, and uses the SBA to perform bidirectional fusion on the feature layers in the feature extraction backbone; In the prediction stage, the model converts the image features in the feature extraction and fusion stages into the final object detection results, and completes the prediction of defect categories and the regression of bounding boxes.

4. A method for detecting minute defects on the surface of a PCB based on image feature enhancement according to claim 1, characterized in that Specifically, in Step 3: After improving the RT-DETR model, train the Lite-DETR model: set the size of the input image to 640×640 pixels, select AdamW as the optimizer, set the initial learning rate to 0.0002, the number of iterations to 200 epochs, the batch size to 8, and use WMPDIoU as the loss function to reduce the sensitivity to feature scale changes, make the prediction box fit the real box more accurately, and accelerate the convergence process of the model in the feature extraction stage.

5. A method for detecting minute defects on the surface of a PCB based on image feature enhancement according to claim 1, characterized in that, In Step 4, the model performance metrics are Recall, Precision, mean average precision (mAP), and model size, which are respectively expressed as: The definition of Precision is the ratio of the number of samples predicted as positive by the model to the number of samples that are actually positive among them; Among them, TP represents the number of instances correctly predicted as the target, and FP is the number of instances wrongly predicted as the target; The definition of Recall is the ratio of the number of samples that are actually positive to the number of samples accurately identified as positive by the model, as shown in formula (2): Among them, FN is the number of instances that are actually the target but not correctly predicted; AP represents the accuracy - curve, that is, the area enclosed by the P-R curve in the interval [0,1]. The P-R curve is drawn by calculating multiple sets of Precision and Recall values respectively based on the prediction results of the model, and the formula is as follows: Among them, AP is an index to evaluate the detection performance of a single category, and the higher its value, the better the performance of the classifier; The mean average precision (mAP) is an evaluation metric for the classification detection accuracy of multi-class problems. The mean average precision will be calculated under different intersection over union (IoU) thresholds, where n is the number of all classes in the dataset; The model size directly affects the associated hardware configuration in the production line site.

Images