RT-DETR improvement-based board-to-board connector surface defect detection method

Abstract

The invention provides a method for detecting surface defects of a board-to-board connector based on RT-DETR improvement. The method comprises the following steps: acquiring a surface defect image of the board-to-board connector, preprocessing the image, and constructing a defect data set; an RT-DETR model is improved, in a backbone network, an FBConv module and an MlpBlock module are designed to replace a first convolutional layer and a fourth Basic Block module respectively, in a high-efficiency hybrid encoder, a RepNCSPELAN4 feature extraction module is used to replace three original RepC3 modules, a lightweight RepFB module is designed to replace the last RepC3 module, a feature pyramid network BiFPN is introduced to replace a Concat module, and a high-efficiency hybrid encoder is obtained. Replacing a part of the convolutional layer with an SCDown module to carry out space down-sampling; finally, an original GIoU loss function is replaced with an Inner-MPDIOU loss function, and the improved model is named as BTB-RTDETR (Basic Target Bridge-Real Time Development Target Ratio); and training the BTB-RTDETR model for detecting the surface defects of the board-to-board connector. According to the invention, the detection accuracy and the detection efficiency of the surface defects of the board-to-board connector workpiece are effectively improved, and the labor cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a method for detecting surface defects in board-to-board connectors based on an improved RT-DETR method. Background Technology

[0002] With the rapid development of the electronics manufacturing industry, the demand for board-to-board connectors in devices such as mobile phone motherboards is constantly increasing, and the requirements for connector manufacturing process quality are becoming increasingly stringent. Affected by factors such as manufacturing processes, production environment, and equipment precision, various defects are prone to appear on the surface of board-to-board connectors during actual production, such as terminal patches, metallic flakes, edge depressions, and edge protrusions. These defects can lead to abnormal product function or reduced quality. Therefore, how to efficiently and accurately detect surface defects in board-to-board connectors has become an urgent problem to be solved.

[0003] Currently, most factories still rely on manual visual inspection or traditional machine vision methods to detect surface defects in connectors. However, manual inspection is inefficient, costly, and susceptible to subjective factors. Traditional vision algorithms also struggle to guarantee accuracy in complex backgrounds and with minute defects, failing to meet the quality inspection needs of modern high-speed industrial production. With the rapid development of computer technology and industrial intelligence, deep learning-based target detection technology has made significant progress in recent years. Leveraging its advantages of automatically learning image features and efficient defect recognition, it has gradually become the mainstream solution for industrial surface defect detection. Existing target detection algorithms are mainly divided into two categories: one is detection algorithms based on convolutional neural networks (CNNs), typically including two-stage detection methods (such as R-CNN and Faster R-CNN) and single-stage detection methods (such as SSD and the YOLO series); the other is detection algorithms based on the Transformer architecture (such as DETR). Two-stage detection algorithms offer higher accuracy but slower detection speeds, while single-stage algorithms do not require candidate region extraction, resulting in faster detection speeds but relatively lower accuracy. Furthermore, some algorithms (such as the YOLO series) require non-maximum suppression (NMS) to remove redundant detection boxes, increasing post-processing overhead and potentially affecting model robustness. The introduction of the Transformer architecture into the field of object detection has improved detection performance through global feature modeling. For example, the DETR model proposed by Carion et al. implements an end-to-end object detection process, but it has a large number of parameters and slow training convergence. The Deformable DETR proposed by Zhu et al. reduces computational complexity and improves detection performance for targets of different scales through a deformable attention mechanism. The RT-DETR (Real-Time Detection Transformer) model launched by the Baidu team has achieved a better balance between speed and accuracy: RT-DETR eliminates the anchor frame design, adopts an efficient Transformer module and an optimized detection head, and achieves a balance between real-time performance and accuracy, surpassing the YOLO algorithm of the same period in both detection accuracy and inference speed. However, the RT-DETR model still has shortcomings in balancing detection accuracy and efficiency when facing surface defect detection tasks of board-to-board connectors, which have large variations in defect size, limited sample numbers, and unclear defect contours. Therefore, by improving the model, the aim is to further improve the model's detection accuracy while reducing model complexity and improving model detection efficiency, so as to meet the requirements of high accuracy and deployability for workpiece defect detection in actual industrial environments. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention provides a method for detecting surface defects in board-to-board connectors based on an improved RT-DETR approach. By improving and optimizing the network model, high-precision and efficient detection of surface defects in board-to-board connectors is achieved.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A method for detecting surface defects in board-to-board connectors based on RT-DETR improvement, comprising the following steps:

[0006] Step 1: Obtain defect surface image samples of board-to-board connectors through front-end image acquisition equipment to provide a data foundation for subsequent model training.

[0007] Step 2: Preprocess the acquired connector images, including: defect annotation, image augmentation, grayscale conversion, and dataset partitioning.

[0008] Step 3: Based on the RT-DETR model, improve its backbone network, efficient hybrid encoder and loss function to obtain the board-to-board connector surface defect detection model BTB-RTDETR.

[0009] Step 4: Input the board connector surface defect dataset into the BTB-RTDETR improved model for training.

[0010] Step 5: Input the surface image of the board-to-board connector to be inspected into the optimized defect detection model, perform inference calculations, and output the defect detection results. The model will directly locate and identify the defect locations and their categories in the image.

[0011] Step 6: Transmit the detected defect results to the back-end processing module or display device to complete the quality inspection of surface defects of the board-to-board connector. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the design of a board-to-board connector surface defect detection method based on RT-DETR, as proposed in this invention.

[0013] Figure 2 This is a schematic diagram of the overall network structure of RT-DETR-r18.

[0014] Figure 3 This is a schematic diagram of the overall network structure of the BTB-RTDETR defect detection model proposed in this invention.

[0015] Figure 4 This is a schematic diagram of the FBConv module structure designed in this invention.

[0016] Figure 5 This is a schematic diagram of the MlpBlock module structure designed in this invention.

[0017] Figure 6 This is a schematic diagram of the MlpNet module structure designed in this invention.

[0018] Figure 7This is a schematic diagram of the RepNCSPELAN4 feature extraction and fusion module structure.

[0019] Figure 8 This is a schematic diagram of the RepFB module designed in this invention.

[0020] Figure 9 This is a schematic diagram of the BiFPN network structure.

[0021] Figure 10 This is a schematic diagram of the standard convolutional module and the SCDown module.

[0022] Figure 11 This is a schematic diagram illustrating the principle of the Inner-IoU loss function.

[0023] Figure 12 This is a schematic diagram illustrating the principle of the MPDIoU loss function.

[0024] Figure 13 This is a comparison chart of the visualization detection results of the model proposed in this paper and different models. Detailed Implementation

[0025] The following detailed description, in conjunction with the accompanying drawings and embodiments, provides a method for detecting surface defects in board-to-board connectors based on an improved RT-DETR. It should be noted that, unless otherwise specified, the features in the embodiments can be combined with each other.

[0026] This invention provides a method for detecting surface defects in board-to-board connectors based on an improved RT-DETR method. The overall process of the method includes six steps: image acquisition, image preprocessing, model construction, model training, defect detection, and result output.

[0027] Step 1: Image Acquisition. An industrial camera is used to acquire surface images of the board-to-board connectors. The camera captures complete surface information of the board-to-board connectors and various types of defects. The acquired raw image data is stored in the data acquisition module.

[0028] Step 2: Image Preprocessing. Preprocessing includes manually annotating the acquired defect images, using the LabelImg image annotation tool to mark the bounding box of the defect location in each image, and assigning a corresponding defect category label. Defect categories include terminal patches (oxidation spots or stains on the connector terminal surface), metallic flakes (metal fragments or burrs remaining on the connector surface), edge dents (dents or defects at the connector edge), and edge protrusions (excess protrusions at the connector edge). The annotated images are then subjected to appropriate data augmentation (such as random rotation or adding noise) to increase the number of defect sample images and improve the model's robustness. Subsequently, the augmented images undergo uniform grayscale conversion. After image augmentation, a board-to-board connector defect detection dataset is constructed, and the dataset is proportionally divided into training, validation, and test sets (70% of images for training, 10% for validation, and 20% for final testing and evaluation) to ensure the reliability of model training and evaluation.

[0029] Step 3: Model Construction. Using the data prepared in Step 2, construct the surface defect detection model for the board-to-board connector of this invention. For example... Figure 2 As shown, the RT-DETR-r18 model mainly consists of three parts: a backbone network, an efficient hybrid encoder, and a Transformer decoder. This invention makes several improvements to the RT-DETR-r18 model structure to obtain the board-to-board connector surface defect detection model BTB-RTDETR, whose network structure is shown below. Figure 2 As shown, the specific improvements include:

[0030] (1) Improved backbone network structure: The original RT-DETR backbone uses a ResNet-18 network, which includes 3 convolutional layers, max pooling layers, and 4 basic residual modules (BasicBlock). This invention first improves the input layer of the backbone network: the original first-layer convolutional (Conv) is replaced with an FBConv module (e.g., Figure 4 (As shown). Inspired by SPDConv, the FBConv module consists of two parallel paths: a spatial slice parallel convolution branch and a regular module convolution branch. A channel attention mechanism is introduced before the output. In the FBConv module, the input feature map is processed along... Figure 4 The left and right paths are processed in parallel. Figure 4 On the left path, the input feature map first undergoes parallel slicing, dividing it into multiple sub-feature maps by channel. This not only maximizes utilization but also learns complementary features, avoiding information redundancy. Then, rich feature representations are obtained through concatenation and 2D convolution. Figure 4 On the right path, the input features are processed through 2D convolution to extract local features, then enter a two-layer convolutional MLP submodule to enhance non-linear feature fusion capabilities. The outputs of the two paths are then fused through channel-wise attention-weighted fusion (global information is extracted through adaptive average pooling, followed by two 2D convolutions and an intermediate ReLU activation function, and finally attention weights are obtained through Sigmoid, and the fused features are recalibrated by channel), and finally output after ReLU activation. Compared to the single-path convolution of SPDConv, the FBConv module expands the receptive field and enhances the ability to extract local details and global semantics through multi-path parallel processing and the addition of attention mechanisms. FBConv significantly improves the model's feature extraction performance in cases of small target defects and blurred edges, making the model more sensitive and robust to complex defects on the surface of board-to-board connectors.

[0031] Furthermore, this invention replaces the last layer BasicBlock module of the backbone network with an MlpBlock module (e.g., Figure 5 (As shown). BasicBlock efficiently extracts basic features through two layers of convolution and residual connections, but its fixed kernel size limits the utilization of features at different scales. MlpBlock structurally integrates residual connections and optional shortcuts, as well as channel attention and the MlpNet submodule. In the MlpBlock module, when shortcuts are enabled, the input features are processed by convolution and the MlpNet module to extract multi-level features, then important channel information is highlighted through channel attention, and the output features are fused with the original input through residual shorting, and finally output through the ReLU activation function. When shortcuts are disabled, the main branch follows the same process as when shortcuts are enabled, but a short stride branch is added. If short stride=1, it will pass through a convolutional layer; if short stride=2, it will pass through a two-dimensional adaptive average pooling and then convolution. Finally, the short branch is fused with the main branch output and output through the ReLU activation function. (As shown) Figure 5 (The diagram shows different modes). The MlpNet module is a key submodule in MlpBlock, inspired and improved by MANet. Its network structure is as follows: Figure 6As shown, firstly, the input features undergo preliminary convolution. In the left path of the MlpNet module, a 1×1 convolution is used to preserve the feature size, while the right path uses two 1×1 convolutions. The middle path is split into three paths by a split, and then passes through the Bottleneck module. In the Bottleneck module, the input features undergo two 3×3 convolutions, and are then fused with the original input features through residual connections, enhancing non-linear representation. Finally, the output features from the three paths are concatenated, and the concatenated features undergo a final 1×1 convolution for output. This design improves the model's multi-scale feature fusion and ensures training stability.

[0032] In summary, the MlpBlock module enables the backbone network to integrate features from different levels while maintaining efficient gradient propagation, thereby improving the model's adaptability to various defect patterns in board-to-board connectors and its detection accuracy.

[0033] (2) Improved Structure of High-Efficiency Hybrid Encoder: The high-efficiency hybrid encoder of RT-DETR-r18 contains multiple parallel fusion convolutional modules RepC3. RepC3 adjusts and fuses two 1×1 convolutional channels, and then extracts features through multiple 3×3 convolutional blocks. Although efficient, it does not make sufficient use of multi-scale information. Therefore, this invention introduces the RepNCSPELAN4 module (e.g., Figure 7 The RepNCSPELAN4 module replaces the original three RepC3 modules (as shown). Based on the Generalized Efficient Layer Aggregation Network (GELAN), RepNCSPELAN4 divides the input features into two parts, which are then processed by two separate RepNCSP sub-modules and finally fused together. The RepNCSPELAN4 module combines the strengths of CSPNet and ELAN networks: leveraging CSPNet's cross-stage grouping strategy to reduce computational bottlenecks and promote information flow between features at different stages; and utilizing ELAN's efficient hierarchical aggregation strategy to extract and fuse features at different levels. The RepNCSP sub-modules combine multi-scale feature extraction and attention mechanisms to maintain semantic content while ensuring necessary spatial information, enabling the output features to possess both detailed texture information and global contextual relationships. Through this design, the RepNCSPELAN4 module significantly improves the model's ability to extract and fuse defect features at different scales, reduces computational complexity, and improves detection accuracy. The convolution process of the RepNCSPELAN4 module is shown in the following formula:

[0034]

[0035] In the formula: This indicates that the input feature map is at the 1st... On each channel, the location The value; Indicates that the convolution kernel is in On the passageway, location The value; Indicates the output feature map at the th On each channel, the location The value; It is the first The bias of each channel; It is the size of the convolution kernel.

[0036] Furthermore, to further reduce model computation and improve real-time detection performance, this invention designs a RepFB module in the high-efficiency hybrid encoder to replace the last RepC3 module (e.g., Figure 8 The diagram shows the structure of the RepFB module and its sub-modules. The RepFB module adopts a two-branch structure: the left branch first passes through a convolutional layer, and then sequentially connects three FasterBlock sub-modules for deep feature extraction; the right branch only adjusts the number of channels of the input features through a single convolution, preserving the original information. The outputs of the two branches are then concatenated and fused along the channel dimension, and the output features are obtained through a final convolutional layer. The FasterBlock sub-module introduces mechanisms such as partial convolution (Partial_conv3) and DropPath random path discarding: Partial_conv3 groups the input channels into two groups, performing a two-dimensional convolution on only one group, significantly reducing the computational cost of a single convolution, while concatenating the outputs of the two groups before outputting; DropPath randomly discards some feature propagation paths within FasterBlock during training, equivalent to integrating multiple sub-networks, improving the model's generalization and robustness to anomalies. Through the multi-branch parallelism and regularization design of the RepFB module, the model's computational cost is significantly reduced, training is more stable, and convergence is accelerated.

[0037] In summary, the RepNCSPELAN4 and RepFB modules are used in tandem in the high-efficiency hybrid encoder section, enabling the model to efficiently fuse multi-scale defect features while also meeting the requirements of lightweight design and high efficiency.

[0038] (3) Feature Fusion and Downsampling Improvement: Since board-to-board connector defects may appear at different scales, fully fusing features from each layer is crucial for improving the detection rate of small defects. This invention introduces a Bidirectional Feature Pyramid Network (BiFPN) structure into the model's efficient hybrid encoder to replace the original simple feature cascade connection. BiFPN achieves bidirectional feature transfer from top to bottom and bottom to top through adaptive weighted fusion between multi-level features: high-level semantic features can be transferred to the lower level to enhance the localization of large-scale defects; low-level detailed features can also be transferred to the higher level to improve the identification of small defects. The BiFPN module network structure is as follows: Figure 9As shown in the diagram, the arrow path from layer P7 to layer P6 is a top-down path used to convey semantic information from higher-level features; the arrow path from layer P3 to layer P7 is a bottom-up path used to convey lower-level feature information; the curved paths of layers P6, P5, and P4 are newly added edges between the input and output nodes within the same layer. In the BiFPN module, features at each scale are weighted according to their importance and participate in the fusion, making the fusion result more effective for defect targets. By introducing BiFPN, the model's ability to detect defects at different scales in complex backgrounds is further improved.

[0039] The BiFPN module performs bidirectional cross-scale connectivity while also weighting each input feature, and the calculation formula is as follows:

[0040]

[0041] In the formula: and Represents different learnable weight parameters; Indicate input features; set The value is set to 0.0001 to avoid numerical instability.

[0042] After obtaining the weighted features, weighted feature fusion is performed, and the calculation formula is as follows:

[0043]

[0044]

[0045] In the formula: Indicates the first Intermediate features of the layer; Represents input features; Indicates output features; This indicates upsampling or downsampling of the feature map.

[0046] Regarding downsampling, this invention replaces some of the original network's 3×3 convolutional downsampling layers with SCDown modules (such as...). Figure 10 (As shown). The SCDown module employs an optimized spatial channel downsampling strategy to reduce the size of the feature map while preserving key information as much as possible: for example, by reducing resolution through grouped average pooling or selective convolution, thereby reducing parameters and computational cost. After SCDown processing, the number of model parameters is reduced, while subsequent layers still obtain clear feature inputs, ensuring real-time detection.

[0047] The mathematical expressions for the computational complexity and number of parameters of the SCDown module are as follows:

[0048]

[0049]

[0050] In Formula 5: An asymptotic representation of computational complexity describes the growth trend of the computational cost or number of parameters of an algorithm as the input size changes. and These are the height and width of the feature map, respectively; Number of channels; Corresponding point convolution, Corresponding depthwise convolution. In Equation 6: The number of parameters for point convolution; The number of parameters for depthwise convolution.

[0051] (4) Loss Function Optimization: This invention employs an improved target loss function during model training. As mentioned earlier, the bounding box regression loss is replaced with the Inner-MPDIoU loss function. The Inner-MPDIoU loss function combines the advantages of the auxiliary bounding box-based IoU loss function (Inner-IoU) and the minimum point distance bounding box regression loss function (MPDIoU): Inner-IoU focuses on the core part of the bounding box rather than the whole, thus providing a more accurate evaluation of the overlapping area inside the bounding box. Simultaneously, it incorporates a scale factor ratio to control the scale of the auxiliary bounding box, allowing for the use of auxiliary bounding boxes of different scales for different tasks and detectors, accelerating bounding box regression and enhancing the generalization ability of the loss function. The principle of the Inner-IoU loss function is as follows: Figure 11 As shown, the ground truth (GT) and anchor boxes represent... and The center point of the real bounding box is ( , ) represents the center point of the anchor frame, while () represents the center point of the anchor frame. , The actual bounding box is represented by ). The height and width of the actual bounding box are represented by . and Indicate that the height and width of the anchor frame are respectively represented by... and This indicates that the variable " "" represents the scaling factor, which is typically in the range of [0.5, 1.5]. The specific formula for the Inner-IoU loss function is defined as follows:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063] MPDIoU directly minimizes the distance between the top-left and bottom-right points of the predicted bounding box and the true bounding box, enabling board-to-board connector surface defect detection to achieve higher robustness. Its parameter diagram is shown below. Figure 12 As shown. The formula for calculating the MPDIoU loss function is as follows:

[0064]

[0065]

[0066]

[0067] In the formula: ( , )and( , These are the coordinates of the top-left and bottom-right corners of the prediction box, respectively; , )and( , ( ) are the coordinates of the top left and bottom right corners of the actual bounding box, respectively; , These are the distances between the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box, respectively. and These represent the height and width of the input image, respectively.

[0068] This invention combines Inner-IoU and MPDIoU loss functions, adding an internal feature point metric from Inner-IoU to MPDIoU to form the Inner-MPDIoU loss function. This combination allows the loss function to more precisely measure the difference between the predicted and ground truth bounding boxes, ensuring that defects of different sizes receive appropriate loss feedback signals during regression. Compared to the traditional GIoU loss function, Inner-MPDIoU exhibits faster convergence speed and higher localization accuracy in the defect detection task of this invention.

[0069] Following the model construction steps outlined above, the final BTB-RTDETR defect detection model is shown in the schematic diagram of its overall network structure. Figure 3 As shown, the improved model achieves high-precision positioning and classification of defects on the surface of board-to-board connectors, achieving a balance between accuracy and efficiency.

[0070] Step 4: Model Training. After building the improved model architecture, the BTB-RTDETR model is trained using the training set prepared in Step 2. After training, the optimal weights are obtained and loaded into the BTB-RTDETR model for evaluation on the test set. The evaluation results show that the model of this invention can accurately identify various types of defects, improving detection precision and recall compared to the original model, thus proving the effectiveness of the improved scheme.

[0071] The hyperparameters used in this invention experiment included the AdamW optimizer, 300 training epochs, an initial learning rate of 0.0001, a batch size of 16, an input image size of 640×640, and other parameters kept at their default values.

[0072] The evaluation metrics for this invention's experiments include: Precision (P), Recall (R), Average Precision (AP), Mean Average Precision (mAP), GFLOPs (gigaflops per second), and Params. The formulas for calculating these evaluation metrics are as follows:

[0073]

[0074]

[0075]

[0076]

[0077] In the formula: The number of samples that are correctly classified as positive samples; The number of samples that were misclassified as negative samples; The number of errors in the prediction of positive samples; This represents the total number of defect categories.

[0078] Step 5: Defect Detection. In practical applications, the trained BTB-RTDETR model is deployed to the detection system for online defect detection. After acquiring new board-to-board connector images to be tested, the system uses the model for inference. Specifically, the image is preprocessed in the same way as during training and then input into the model's backbone network. The neural network automatically extracts multi-level features, which are then processed by a hybrid encoder and decoder to directly output the predicted defect information. The model output includes the category, location coordinates (boundary box of the defect region), and confidence score for each detected defect. If no defect is found in an image, the model outputs a result indicating no detection.

[0079] The detection accuracy and computational complexity of the board-to-board connector surface defect detection model BTB-RTDETR, which is based on the RT-DETR improvement proposed in this invention, are compared with the current mainstream target detection models for the surface defect dataset of board-to-board connectors. The results of different models are shown in Table 1.

[0080] Table 1. Comparison of experimental results between the model of this invention and other mainstream models.

[0081] Model Params / M GFLOPs Precision / % Recall / % mAP@0.5 / % mAP@0.5:0.95 / % Faster R-CNN 41.364 192.0 — — 45.5 32.1 YOLOv5m 23.887 64.0 76.3 42.0 50.2 31.4 YOLOv8m 24.645 78.7 74.1 44.0 51.6 32.6 YOLOv10m 19.606 58.9 78.4 42.0 52.5 33.1 YOLOv12m 19.176 67.1 80.2 44.2 55.7 37.5 DETR 41.556 87.783 — — 57.8 29.9 Deformable DETR 40.099 176.0 — — 78.1 43.5 DINO 47.546 250.0 — — 82.0 52.5 RT-DETR-r18 18.956 57.0 84.5 77.4 84.6 54.5 BTB-RTDETR 18.100 57.0 86.7 78.7 86.5 56.3

[0082] Comparing the experimental results in the table, it can be seen that the BTB-RTDETR improved model proposed in this invention is superior to other mainstream models in terms of parameter quantity and floating-point operation quantity. The average accuracy mAP@0.5 / % and mAP@0.5:0.95 / % also show a leading performance, demonstrating good detection efficiency and detection accuracy.

[0083] Step 6: Result Output. Finally, the defect detection results obtained in Step 5 are sent to the production line's quality control system. On one hand, the detected defect locations can be displayed in real-time on the industrial control interface, with defect types marked with boxes and labels on the board-to-board connector image for intuitive viewing by on-site operators. On the other hand, the detection results can also be transmitted as signals to the automatic sorting device. When a workpiece is determined to have a defect, the defective product is mechanically removed from the production line to prevent it from flowing into the next stage. The detection results can also be saved to a database for subsequent statistical analysis.

[0084] The above description of the embodiments demonstrates that the defect detection method provided by the present invention can accurately and efficiently detect surface defects in board-to-board connectors. Practical applications show that the method of the present invention has high detection accuracy and strong robustness, outperforming traditional manual inspection and unimproved algorithms. Those skilled in the art should understand that adjustments and modifications can be made to the specific processes and module configurations of the above embodiments without departing from the principles of the present invention, and these changes should all be considered within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in board-to-board connectors based on an improved RT-DETR method, characterized in that, Includes the following steps: Step 1: Image acquisition. The surface defect images generated during the manufacturing process of the board-to-board connector are acquired by using the board-to-board connector image acquisition platform. The original image dataset of surface defects of the board-to-board connector is obtained. Step 2: Image preprocessing. The raw dataset acquired in Step 1 is labeled with defect locations and categories using the LabelImg image annotation tool. Subsequently, the labeled dataset is augmented by adding background noise and random rotation, and the augmented dataset images are converted to grayscale. Finally, the dataset is randomly divided proportionally to construct a board-to-board connector surface defect dataset. Step 3: Improvement and optimization of the target detection model. Based on the RT-DETR model, its backbone network, efficient hybrid encoder and loss function are improved and optimized to obtain the board-to-board connector surface defect detection model BTB-RTDETR. Step 4: Model training. Input the board connector surface defect dataset obtained in Step 2 into the improved BTB-RTDETR model obtained in Step 3 for training. Step 5: Defect detection. Input the surface image of the board-to-board connector to be inspected into the improved defect detection model to obtain the defect detection results, including the defect category and confidence level. Step 6: Output the results. Provide the defect detection results to the backend equipment for display and processing to complete the defect production quality inspection of the board-to-board connectors.

2. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 1, characterized in that, Step 2, image preprocessing includes: using the LabelImg image annotation tool to annotate the bounding box positions and defect categories of defect targets in the image. The defect categories include four types: terminal patches (bk), metallic flakes (lp), edge depressions (ax), and edge protrusions (tq). Subsequently, the annotated board-to-board connector images are augmented by adding background noise and random rotation. In the board connector surface defect detection task, the focus is mainly on the shape features of the defects, rather than relying on the color information of the defects. Therefore, the augmented dataset images are converted into grayscale images. Finally, the dataset is randomly divided into training set, validation set, and test set in a ratio of 7:1:

2.

3. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 1, characterized in that, Step 3, Improvement and optimization of the target detection model, the specific methods are as follows: (1) Replace the first convolutional layer in the RT-DETR backbone network with the FBConv module; (2) Replace the last BasicBlock residual module in the RT-DETR backbone network with the MlpBlock module; (3) Replace the three RepC3 modules in the RT-DETR high-efficiency hybrid encoder with the RepNCSPELAN4 module; (4) Replace the last RepC3 module in the RT-DETR high-efficiency hybrid encoder with the lightweight RepFB module; (5) In the RT-DETR high-efficiency hybrid encoder, replace the feature concatenation Concat module with the BiFPN module; (6) In the RT-DETR high-efficiency hybrid encoder, use SCDown to replace some convolutional layers for spatial downsampling; (7) Replace the GIoU loss function in the original RT-DETR model with the Inner-MPDIoU loss function.

4. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: In the backbone network of RT-DETR-r18, the input image information passes through three standard convolutional layers. However, the sampling shape of standard convolutions is fixed, and the convolutional kernel operation is limited to a local window, failing to capture other positional information and thus unable to effectively handle diverse defect targets on the surface of board-to-board connectors. Therefore, an FBConv module was designed to replace the first convolutional layer in the backbone network. The FBConv module is a further improvement on SPDConv (space-to-depth convolution), which consists of a space-to-depth (SPD) module and a non-strided convolutional layer. First, compared to the single path of SPDConv, the FBConv module adds parallel convolutional branches. These two paths capture spatially rearranged features and traditional convolutional features, respectively, and then, by adding and fusing them along the channel dimension, the richness of feature representation can be greatly enhanced. Furthermore, by adding the MLP submodule, additional non-linear transformations can be introduced to the model, helping it learn more flexible feature representations. FBConv integrates a channel attention module to perform weighted modulation on fused features. This allows it to adaptively amplify useful features, suppress redundant noise, and enhance the discriminability of small targets in the feature map. It effectively improves the model's sensitivity to small objects, thereby increasing detection accuracy. In summary, the FBConv module, through its parallel branch parallel processing, multi-path feature fusion, and attention mechanism design, makes the model more sensitive and robust when processing images of unclear contours and small-sized board-to-board connector surface defects.

5. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: The RT-DETR-r18 backbone network adopts the ResNet network architecture, explicitly constructing a high-dimensional feature space by increasing the feature dimension. To balance the computational burden, the backbone network uses only a four-layer block structure. The BasicBlock serves as the basic building block, designed to efficiently extract and pass information from the input feature map to support subsequent task processing. The BasicBlock module mainly alleviates the gradient vanishing problem in deep networks through residual connections. When using fixed-size convolutional kernels to extract features, it fails to fully utilize global contextual information, which may not effectively handle features at different scales and may not fully utilize features from all layers. Therefore, the MlpBlock module was designed to replace the BasicBlock module located in the last layer of the RT-DETR-r18 backbone network. This enhances the model's feature extraction capability and gradient flow, thereby improving the model's feature representation and semantic depth at different levels. Replacing the original BasicBlock module with the MlpBlock module enables the model to learn more complex and diverse features, further improving the detection accuracy of surface defects in board-to-board connectors. The MlpBlock module controls residual skip connections by adding a shortcut parameter, which maintains direct gradient propagation and ensures feature alignment and fusion between different layers. The added Channel Attention module dynamically adjusts channels to enhance the weights of important channels, strengthens key features, and improves feature discrimination capabilities. The shortstride option allows for the selection of whether to perform downsampling, adapting to multi-scale feature extraction. By adding feature branch processing to the MlpNet module, feature representation becomes more comprehensive. Simultaneously, by adjusting channels using 1×1 convolutions, computational resources are allocated rationally, reducing the number of parameters, and stacked convolutional layers enhance feature reuse through dense connections. In summary, MlpBlock achieves enhanced information flow and multi-scale feature fusion through the combination of residual connections, parallel multi-convolutional paths, and channel attention.

6. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: The efficient hybrid encoder RT-DETR-r18 utilizes the RepC3 module, a reparameterized convolutional module. This module processes the input feature map along two parallel paths, each using a 1×1 convolutional layer to adjust the channel dimensions, followed by element-wise addition to achieve feature fusion. The fused feature map undergoes three reparameterized convolutional blocks (ReConV), followed by further convolutional layer optimization before final output. While RepC3 achieves efficient feature extraction and fusion, it still suffers from high computational complexity and underutilization of multi-scale information when dealing with the complex environment of board-to-board connector surfaces, affecting the accuracy of surface defect detection. To further enhance the model's feature extraction and detection performance, the RepNCSPELAN4 module is used to replace the RepC3 module in the high-efficiency hybrid encoder.

7. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: The RepC3 module suffers from high computational complexity and insufficient feature extraction capabilities. Therefore, the RepFb module was designed to replace the last RepC3 module in the RT-DETR-r18 cross-scale feature fusion module. Replacing the original RepC3 module with the RepFb module improves the model's computational efficiency and generalization ability, meeting the requirements for surface defect detection of board-to-board connectors in real-world industrial environments. The RepC3 module relies solely on consecutive 1×1 convolutions and reparameterized convolutional blocks (ReConV), resulting in a limited receptive field. In contrast, the RepFb module improves computational efficiency by using Partial_conv3 in its internal FasterBlock module for block-parallel multi-branch convolutions, while capturing both local details and global information for more comprehensive multi-scale feature extraction. The Partial_conv3 module in the RepFb module splits the feature map into multiple sub-blocks for parallel processing, significantly reducing the computational cost per branch. Furthermore, it performs 2D convolutions only on a subset of channels, compared to the RepC3 module where all channels participate in 3×3 convolutions. ReConv, which stacks convolutions and leads to redundant computation, is addressed in the RepFb module by significantly reducing convolution computation while maintaining the receptive field. The RepC3 module lacks a random dropout mechanism, while the RepFb module's FasterBlock incorporates DropPath, allowing the network to randomly drop some feature paths during training. This effectively integrates multiple sub-networks, improving model generalization and robustness to anomalous samples. Furthermore, the residual connections in the FasterBlock module ensure more efficient gradient and information flow, resulting in faster convergence and more stable training. In summary, the RepFb module effectively reduces computation and enhances training stability through a combination of parallel convolutions, block-based parallel processing, residual connections, and random path dropout.

8. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: In RT-DETR-r18, the Concat module is used to stitch feature maps from different sources and scales to achieve feature fusion. However, it only performs a single feature stitch and lacks a bidirectional interaction mechanism between features, resulting in insufficient representation ability for cross-scale targets. Furthermore, the Concat module has a large number of channels, requiring additional dimensionality-reducing convolutions to reduce the number of parameters, which increases computational cost. Therefore, to further reduce the computational cost of the model, achieve model lightweighting, and improve multi-scale feature fusion, the BiFPN module is used to replace the original Concat module in the neck network. The BiFPN module enhances the effective transmission of information between features at different levels by introducing bidirectional and weighted feature fusion mechanisms, significantly improving information fusion between feature maps. Simultaneously, cross-scale connections are optimized by removing nodes with only one input edge, adding extra edges between input and output nodes at the same level, and treating each bidirectional path as a feature network layer and repeating it multiple times. Finally, by using the BiFPN module structure design, not only is the feature representation ability and detection accuracy of the board-to-board connector surface defect detection model improved, but the computational complexity of the model is also reduced.

9. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: To address the high computational complexity introduced by the 3×3 convolutional layers used for spatial downsampling in the RT-DETR-r18 model, this paper introduces SCDown as a downsampling method to replace the original 3×3 convolutional modules. The SCDown module first adjusts the channel dimensions using 1×1 convolutions to maintain the original size of the input feature map, effectively reducing the number of parameters compared to standard convolutional modules. Then, spatial downsampling is performed through depthwise convolutions. This design not only reduces the computational cost of spatial downsampling and channel adjustment in traditional convolutions but also allows the model to more effectively acquire key local features, improving the detection of small terminal patch defects on the surface of board-to-board connectors. By replacing the 3×3 convolutional layers with the SCDown module, the fixed-scale processing of traditional standard convolutions is transformed into dynamic adaptive detection at various target scales, effectively improving the model's detection performance of board-to-board connector surface defects at different scales while significantly reducing the number of parameters required for computation.

10. The method for detecting surface defects in board-to-board connectors based on RT-DETR improvement according to claim 3, characterized in that: Bounding box loss functions play a crucial role in object detection and localization, measuring the difference between predicted and actual bounding boxes. Current Bounding Box Regression (BBR) based on the original IoU (Intersection over Union) method focuses on accelerating convergence by adding new loss terms, neglecting the inherent limitations of the IoU loss term and failing to reflect the true distance between predicted and actual bounding boxes. Therefore, the RT-DETR-r18 network uses the GIoU loss function, introducing a minimum closed box to locate defects and assess the similarity between two bounding boxes to calculate the regression loss. However, the GIoU loss function cannot adaptively adjust to different detectors and detection tasks to use auxiliary bounding boxes of different scales to accelerate bounding box regression. By combining the Inner-IoU and MPDIoU loss functions, and improving the MPDIoU loss function by introducing internal feature points from the Inner-IoU loss function, we obtain the Inner-MPDIoU loss function. Using it to replace the original GIoU loss function can better capture the detailed changes inside the target, thereby improving the model's detection performance and generalization ability.

Images