A PCB defect detection method based on RT-DETR

Abstract

This invention discloses a PCB board defect detection method based on RT-DETR, belonging to the fields of computer vision and industrial inspection technology. The method includes: constructing a PCB board defect detection dataset; designing a CSP-MES module based on the dataset and embedding this module into the RT-DETR backbone network to obtain a PCB board defect detection model; and obtaining PCB board defect detection results based on the PCB board defect detection model. This invention improves the model's ability to perceive and detect multi-scale defects by introducing a multi-scale edge information selection module combined with a dual-domain selection mechanism to adaptively filter key features related to defects.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial inspection technology, and in particular to a PCB board defect detection method based on RT-DETR. Background Technology

[0002] Traditional PCB board defect detection methods have the following problems: (1) High cost and limited application scenarios: Existing detection methods mostly rely on manual visual inspection or high-precision AOI equipment. Although the detection accuracy is high, the equipment cost is expensive, maintenance is complicated, and it is difficult to adapt to the rapidly changing production line requirements.

[0003] (2) High model complexity and slow inference speed: Existing target detection models (such as RT-DETR-R18) have a large number of parameters and high computational complexity, making it difficult to achieve real-time detection on embedded devices, which limits their application in industrial fields.

[0004] (3) Insufficient multi-scale feature representation capability: The size difference of PCB board defects is large (such as micro-open circuits, burrs, etc.), and traditional convolutional modules have difficulty capturing global semantic information and local detail features at the same time, resulting in a high rate of missed detection of small targets.

[0005] (4) Loss of edge information: Edge details are easily lost during downsampling, affecting the positioning accuracy of defects, especially edge defects (such as rat bites and burrs).

[0006] (5) Feature redundancy and insufficient selection: Existing methods often simply splice or add features when fusing multi-scale features without effectively filtering redundant information, resulting in low model computation efficiency and susceptibility to background interference. Summary of the Invention

[0007] To address the aforementioned shortcomings in existing technologies, this invention provides a PCB board defect detection method based on RT-DETR. By introducing a multi-scale edge information selection module and combining it with a dual-domain selection mechanism, the method adaptively filters key features related to defects, thereby improving the model's ability to perceive and detect multi-scale defects.

[0008] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: a PCB board defect detection method based on RT-DETR, comprising the following steps: S1: Construct a PCB board defect detection dataset; S2: Based on the dataset, design the CSP-MES module and embed the module into the RT-DETR backbone network to obtain a PCB board defect detection model; S3: Based on the PCB board defect detection model, obtain the PCB board defect detection results.

[0009] Furthermore, the PCB board defect detection dataset includes six types of defects: vias, rodent bites, open circuits, short circuits, burrs, and fake copper.

[0010] Furthermore, the CSP-MES module includes a multi-scale feature extraction branch, an edge enhancement branch, a local convolution branch, and a dual-domain selection fusion layer.

[0011] Furthermore, the multi-scale feature extraction branch pools the input feature map to multiple preset scales of 3×3, 6×6, 9×9, and 12×12 through adaptive average pooling. Each scale is then subjected to 1×1 convolution for dimensionality reduction and 3×3 depthwise separable convolution to extract local details, forming a multi-scale feature representation.

[0012] Furthermore, the edge enhancement branch employs the EdgeEnhancer module, which extracts the low-frequency smoothing component of the features through 3×3 average pooling. The smoothing feature is then subtracted from the original feature to obtain high-frequency edge information. This high-frequency edge information is generated through 1×1 convolution and sigmoid activation. The original feature and the enhanced edge feature are then added together to strengthen the edge information. The formula is as follows:

[0013]

[0014]

[0015] in, Indicates low-frequency smoothing characteristics. This represents an average pooling operation with a kernel size of 3×3, a stride of 1, and padding of 1. Indicates the input feature map, Represents the edge attention map. This represents the Sigmoid activation function. This indicates a convolutional layer with a kernel size of 1×1. This represents the final output feature map.

[0016] Furthermore, the local convolution branch processes the original input features through a 3×3 convolution, keeping the number of channels unchanged and preserving the original local detail information.

[0017] Furthermore, the dual-domain selection fusion layer concatenates the output of the local convolutional branch with the feature maps at each scale that have undergone edge enhancement and upsampling to the original size in the channel dimension, and the concatenated features are then fed into the dual-domain selection mechanism (DSM) module. The dual-domain selection mechanism (DSM) module performs attention weighting from the spatial domain and the channel domain respectively. In the spatial domain, spatial descriptors are extracted through global average pooling and global max pooling, and a spatial attention map is generated through a 7×7 convolution. In the channel domain, channel descriptors are generated through global average pooling, and a channel attention map is generated through two fully connected layers. The spatial attention map and the channel attention map are multiplied element-wise with the original features, and then the number of channels is restored through a 1×1 convolution, and the final enhanced features are output.

[0018] The beneficial effects of this invention are: by using the self-developed CSP-MES module, after extracting multi-scale edge features, this invention introduces the DSM attention mechanism to focus on important areas from both spatial and channel dimensions, effectively suppressing background noise and significantly improving the detection accuracy of PCB micro-defects, while maintaining low computational overhead, making it suitable for real-time deployment on industrial production lines. Attached Figure Description

[0019] Figure 1 This is a flowchart of the PCB board defect detection method based on RT-DETR of the present invention.

[0020] Figure 2 This is a schematic diagram of the CSP-MES module structure. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 1 As shown, a PCB board defect detection method based on RT-DETR includes the following steps: S1: Construct a PCB board defect detection dataset; The PCB board defect detection dataset is the Peking University Open PCB Defect Dataset, containing 1386 images covering six types of defects: vias, rodent bites, open circuits, short circuits, burrs, and fake copper. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio.

[0023] By using the aforementioned publicly available synthetic dataset, it is easier to verify and compare the algorithm, while also covering a variety of typical PCB defect types, thus enhancing the model's generalization ability.

[0024] S2: Based on the dataset, design the CSP-MES module and embed the module into the RT-DETR backbone network to obtain a PCB board defect detection model; like Figure 2 As shown, the CSP-MES module includes a multi-scale feature extraction branch, an edge enhancement branch, a local convolution branch, and a dual-domain selection fusion layer.

[0025] The multi-scale feature extraction branch pools the input feature map to multiple preset scales of 3×3, 6×6, 9×9, and 12×12 through adaptive average pooling, capturing global contextual information under different receptive fields respectively; each scale is sequentially subjected to 1×1 convolution dimensionality reduction and 3×3 depthwise separable convolution (the number of groups is equal to the number of channels after dimensionality reduction) to extract local details, forming a multi-scale feature representation.

[0026] In the specific implementation, the number of channels in the input feature map is inc, and the number of preset scales is k (i.e. the length of the bins list). Each scale branch first obtains a feature map of size bin×bin through adaptive average pooling, then compresses the number of channels to inc / k through 1×1 convolution, and finally extracts local features through 3×3 depthwise separable convolution with inc / k grouping.

[0027] In this embodiment, the input feature map x has a size of B×C×H×W. The preset scale bins = [3,6,9,12], i.e., k=4. For each bin, the feature map is pooled to B×C×bin×bin using nn.AdaptiveAvgPool2d(bin); then the number of channels is compressed to C / 4 using Conv(C, C / / 4, 1); and then local features are extracted using a 3×3 depthwise separable convolution Conv(C / 4, C / 4, 3, g=C / 4) with C / 4 groups. Four feature maps of different scales are obtained, with sizes of B×(C / 4)×3×3, B×(C / 4)×6×6, B×(C / 4)×9×9, and B×(C / 4)×12×12, respectively.

[0028] The edge enhancement branch employs the EdgeEnhancer module to enhance edge information in the feature map at each scale. This module first extracts the low-frequency smoothing component of the features using 3×3 average pooling (padding=1), subtracting the smoothing feature from the original feature to obtain the high-frequency edge information, i.e., edge = x - avgpool(x). Subsequently, an edge attention map is generated through 1×1 convolution and sigmoid activation. The original feature is then added to the enhanced edge feature to achieve edge information enhancement. The output feature map maintains its size and number of channels unchanged, as shown in the formula:

[0029]

[0030]

[0031] in, This represents low-frequency smooth features, indicating local mean information of the input feature map, preserving the overall structure but weakening details and edges. This represents an average pooling operation with a kernel size of 3×3, a stride of 1, and padding of 1. This operation extracts the low-frequency smoothing components of the feature map, resulting in a smoothed feature map edge with the same size as the input x. This represents the input feature map, with a shape of B×C×H×W, where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map. This represents the edge attention map, with the same shape as x. It serves as a weighting coefficient for adaptive enhancement of edge information. The Sigmoid activation function maps the convolution output to the (0,1) interval, generating an edge attention map. The value of each element in this map represents the importance of the edge information at the corresponding location. This indicates a convolutional layer with a kernel size of 1×1, C input channels, and C output channels. This convolution performs a linear transformation on high-frequency edge information, learning the importance weights of edge features in each channel. The final output feature map is composed of the original input x and the weighted edge information. The element-wise addition is used to obtain the enhanced edge features, which are then fused back into the original features. This makes the model pay more attention to the edge regions of defects and improves the detection capability of boundary-type defects.

[0032] In this embodiment, EdgeEnhancer first obtains smooth features through 3×3 average pooling (stride=1, padding=1) and calculates the edge difference: edge = x - avgpool(x); then, it generates an edge attention map through 1×1 convolution (Sigmoid activation); finally, it adds the original features to the edge attention map, and the output feature map size remains unchanged (still B×(C / 4)×bin×bin). The edge-enhanced features are upsampled to the original size H×W through bilinear interpolation to obtain a B×(C / 4)×H×W feature map.

[0033] The local convolution branch processes the original input features through a 3×3 convolution, keeping the number of channels unchanged and preserving the original local detail information, serving as a branch for subsequent fusion.

[0034] In this embodiment, the original input x is processed by Conv(C, C, 3) to obtain a local feature map of B×C×H×W.

[0035] The dual-domain selection fusion layer concatenates the output of the local convolutional branch with the feature maps at each scale that have been edge-enhanced and upsampled to the original size in the channel dimension. The number of channels in the concatenated feature map becomes inc×2. The concatenated features are then fed into the dual-domain selection mechanism DSM module. The dual-domain selection mechanism (DSM) module performs attention weighting from the spatial domain and the channel domain respectively. In the spatial domain, spatial descriptors are extracted through global average pooling and global max pooling, and a spatial attention map is generated through a 7×7 convolution. In the channel domain, channel descriptors are generated through global average pooling, and a channel attention map is generated through two fully connected layers. The spatial attention map and the channel attention map are multiplied element-wise with the original features, and then the number of channels is restored to inc through a 1×1 convolution, and the final enhanced features are output.

[0036] In this embodiment, the local feature map is concatenated with four upsampled edge enhancement feature maps along the channel dimension to obtain a concatenated feature of B×(C+4*(C / 4))×H×W = B×(2C)×H×W. This feature is then fed into the DSM module. The DSM module first calculates spatial attention and channel attention separately: spatial attention is obtained by performing global average pooling and global max pooling on the feature map along the channel dimension to obtain two descriptors of B×1×H×W, which are then concatenated and processed by a 7×7 convolution to generate a spatial attention map (B×1×H×W); channel attention is obtained by performing global average pooling on the feature map along the spatial dimension to obtain a descriptor of B×2C×1×1, which is then processed by two fully connected layers (dimensionality reduction and then dimensionality increase) to generate a channel attention map (B×2C×1×1). Then, the spatial attention map and the channel attention map are multiplied element-wise with the concatenated feature (broadcast mechanism) to obtain the weighted feature. Finally, the number of channels is restored to C by a 1×1 convolution Conv(2C, C), and the final enhanced feature map is output.

[0037] The aforementioned multi-scale pooling and edge enhancement ensure the sensitivity of features to defect edges. The DSM module focuses on important regions through spatial and channel dual attention, effectively eliminating redundant information and improving feature discrimination and model efficiency.

[0038] S3: Based on the PCB board defect detection model, obtain the PCB board defect detection results.

[0039] This invention embeds a CSP-MES module on the basis of RT-DETR-R18. Experimental results show that on the PCB defect dataset, mAP50 is improved to 95.8%, the number of parameters increases by only 3%, and the inference speed reaches 60 FPS, meeting the needs of industrial real-time detection.

[0040] In one embodiment of the present invention, the PKU-Market-PCB dataset, open from Peking University, is used for model training and validation. This dataset contains 1,386 high-resolution PCB images, covering six typical defects: missing holes, mouse bites, open circuits, short circuits, spurs, and spurious copper. The dataset is divided into a training set (1,109 images), a validation set (138 images), and a test set (139 images) in an 8:1:1 ratio.

[0041] The experimental hardware environment consisted of an NVIDIA RTX 3090 GPU, and the software environment was based on the PyTorch 2.0 deep learning framework with CUDA version 11.8. The input image size was uniformly adjusted to 640×640 pixels, the batch size was set to 16, the training period was 300 epochs, the initial learning rate was 0.01, and a cosine annealing strategy was used for decay.

[0042] To verify the effectiveness of the CSP-MES module described in this invention, the following comparison scheme was set up: Comparative Example 1 (Baseline Model): Original RT-DETR-R18 model, without any embedded improvement modules; Comparative Example 2 (YOLOv8m): A medium-scale YOLOv8 model widely used in current industrial inspection. Comparative Example 3 (Improved YOLO): SWS-YOLOv8n model based on YOLOv8 with Shuffle Attention (published in ACM in 2024). The present invention proposes embedding a CSP-MES module in the RT-DETR-R18 backbone network.

[0043] Standard evaluation indicators in the field of target detection are adopted: mAP@50: Average precision when the IoU threshold is 0.5; mAP@50:95: Mean precision of IoU threshold from 0.5 to 0.95 (step size 0.05); Parameters: The total number of model parameters, expressed as ×10 6 count; FLOPs: Number of floating-point operations, in G(10^6) 9 )count; FPS: Frames per second, a measure of real-time performance.

[0044] The comparative experiments are shown in Table 1: Table 1 Comparative Experiments

[0045] As can be seen, the proposed solution achieves 95.8% mAP@50, a 4.1 percentage point improvement over the baseline RT-DETR-R18, a 1.6 percentage point improvement over YOLOv8m, and a 5.9 percentage point improvement over SWS-YOLOv8n. At the more stringent mAP@50:95 metric, the proposed solution achieves 56.5%, a 3.5 percentage point improvement over the baseline, a 0.1 percentage point decrease over YOLOv8m, and a 5.2 percentage point improvement over SWS-YOLOv8n. This significant performance improvement stems from the synergistic effect of three core designs: the multi-scale edge information extraction branch captures global contextual information under different receptive fields through adaptive pooling, effectively addressing the challenge of large differences in PCB defect sizes; the EdgeEnhancer module strengthens defect edge feature expression through a high-frequency edge enhancement mechanism, significantly improving edge details easily lost during downsampling; and the DSM dual-domain selection mechanism adaptively filters key features from spatial and channel dimensions, effectively suppressing interference from complex PCB background noise.

[0046] The proposed solution demonstrates superior competitiveness in computational efficiency. Regarding parameter control, the proposed solution uses 20.48M parameters, an increase of only 3.0% (+0.6M) compared to the baseline, significantly lower than YOLOv8m's 25.9M (a relative reduction of 21.0%). In terms of computational complexity, GFLOPs are 59.3, an increase of 4.0% (+2.3) compared to the baseline, significantly lower than YOLOv8m's 78.9 (a relative reduction of 24.8%). Regarding real-time performance, FPS reaches 60, an improvement of 5.3% (+3 fps) compared to the baseline, meeting the real-time detection requirements of industrial production lines (typically requiring ≥30 FPS). It is worth noting that although YOLOv8m and SWS-YOLOv8n have higher inference speeds (120 fps and 85 fps respectively), their accuracy-efficiency trade-off is significantly inferior to that of the proposed solution. Specifically, YOLOv8m achieves only a 2.5% accuracy improvement with a 30.3% increase in parameters and a 38.4% increase in computation, while this invention achieves a 4.1% accuracy improvement with a 3.0% increase in parameters and a 4.0% increase in computation, demonstrating a significant cost-performance advantage. This indicates that the CSP-MES module achieves a substantial enhancement in feature representation capabilities with minimal computational overhead.

[0047] SWS-YOLOv8n achieves 89.9% mAP@50 with only 3.0M parameters through the Shuffle Attention mechanism, but its mAP@50:95 is only 51.3%, widening the gap with the proposed solution to 5.2 percentage points. This difference reveals the limitations of lightweight attention mechanisms in defect localization accuracy: while Shuffle Attention can effectively enhance feature discrimination, it lacks the ability to explicitly model edge details, leading to significant performance degradation under multi-scale IoU evaluation. In contrast, the EdgeEnhancer module of this invention achieves precise defect boundary localization through explicit high-frequency edge extraction and enhancement, combined with spatial-channel collaborative screening using the DSM mechanism. This advantage is crucial for industrial applications of PCB defect detection, as high-precision defect localization not only improves detection confidence but also provides reliable pixel-level location information for subsequent defect segmentation, classification, and automated repair. Therefore, the proposed solution achieves a more suitable balance between parameter quantity and localization accuracy for the needs of precision industrial inspection.

[0048] The ablation experiments are shown in Table 2: Table 2 Ablation Experiment

[0049] Ablation experiments further validated the effectiveness of each module design. Local convolution and multi-scale pooling branches jointly contributed a 1.2% improvement in mAP@50, demonstrating that adaptive pooling, by capturing global contextual information under different receptive fields, can effectively adapt to the diverse distribution of PCB defect sizes. The EdgeEnhancer module further contributed an additional 1.5% improvement in mAP@50, validating the crucial role of explicit enhancement of high-frequency edge information in detecting minute defects and significantly improving the problem of easily lost edge details during traditional downsampling. The DSM dual-domain selection mechanism further contributed a 1.4% improvement in mAP@50 and a 1.3% improvement in mAP@50:95, demonstrating that the spatial and channel attention collaborative selection mechanism can effectively remove redundant information in complex PCB backgrounds, significantly improving defect localization accuracy while enhancing feature discrimination power. The progressive optimization of the three modules ultimately achieved a cumulative 4.1% improvement in mAP@50, reflecting the synergistic gain effect of multi-scale edge information selection and dual-domain attention selection.

[0050] This invention provides a PCB board defect detection method based on RT-DETR, which improves the model's ability to detect minute defects by using a multi-scale edge information enhancement module, balancing accuracy and efficiency, and is suitable for online inspection scenarios of industrial PCB boards.

[0051] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the invention.

Claims

1. A PCB board defect detection method based on RT-DETR, characterized in that, Includes the following steps: S1: Construct a PCB board defect detection dataset; S2: Based on the dataset, design the CSP-MES module and embed the module into the RT-DETR backbone network to obtain a PCB board defect detection model; S3: Based on the PCB board defect detection model, obtain the PCB board defect detection results.

2. The PCB board defect detection method based on RT-DETR according to claim 1, characterized in that, The PCB board defect detection dataset includes six types of defects: holes, rodent bites, open circuits, short circuits, burrs, and fake copper.

3. The PCB board defect detection method based on RT-DETR according to claim 1, characterized in that, The CSP-MES module includes a multi-scale feature extraction branch, an edge enhancement branch, a local convolution branch, and a dual-domain selection fusion layer.

4. The PCB board defect detection method based on RT-DETR according to claim 3, characterized in that, The multi-scale feature extraction branch pools the input feature map to multiple preset scales of 3×3, 6×6, 9×9, and 12×12 through adaptive average pooling. Each scale is then subjected to 1×1 convolution for dimensionality reduction and 3×3 depthwise separable convolution to extract local details, forming a multi-scale feature representation.

5. The PCB board defect detection method based on RT-DETR according to claim 3, characterized in that, The edge enhancement branch employs the EdgeEnhancer module, which extracts the low-frequency smoothing component of features through 3×3 average pooling. High-frequency edge information is obtained by subtracting the smoothing feature from the original feature. This high-frequency edge information is then generated through 1×1 convolution and sigmoid activation. Finally, the original feature and the enhanced edge feature are added together to strengthen the edge information. The formula is as follows: in, Indicates low-frequency smoothing characteristics. This represents an average pooling operation with a kernel size of 3×3, a stride of 1, and padding of 1. Indicates the input feature map, Represents the edge attention map. This represents the Sigmoid activation function. This indicates a convolutional layer with a kernel size of 1×1. This represents the final output feature map.

6. The PCB board defect detection method based on RT-DETR according to claim 3, characterized in that, The local convolution branch processes the original input features through a 3×3 convolution, keeping the number of channels unchanged and preserving the original local detail information.

7. The PCB board defect detection method based on RT-DETR according to claim 3, characterized in that, The dual-domain selection fusion layer concatenates the output of the local convolutional branch with the feature maps at each scale that have been edge-enhanced and upsampled to the original size in the channel dimension. The concatenated features are then fed into the dual-domain selection mechanism (DSM) module. The dual-domain selection mechanism (DSM) module performs attention weighting from the spatial domain and the channel domain respectively. In the spatial domain, spatial descriptors are extracted through global average pooling and global max pooling, and a spatial attention map is generated through 7×7 convolution. In the channel domain, channel descriptors are generated through global average pooling, and a channel attention map is generated through two fully connected layers. After multiplying the spatial attention map and channel attention map element-wise with the original features, the number of channels is restored by a 1×1 convolution, and the final enhanced features are output.

Images