PCB defect detection method based on enhanced RTDETR

Abstract

The invention discloses a PCB (Printed Circuit Board) defect detection method based on an enhanced RTDETR. Comprising the following steps: collecting various PCB defect image data, classifying and marking to form a data set; preprocessing the data set to improve the quality of the input image data; an enhanced RTDETR network model is constructed, wherein the enhanced RTDETR network model comprises an innovation module comprehensive cross-stage local attention CCA module and a self-adaptive fast spatial pyramid pooling ASPPF module; and the constructed enhanced RTDETR model is used for learning features and categories of defects. According to the invention, through the comprehensive cross-stage local attention module, the fusion capability and the feature focusing capability of multi-scale feature expression are enhanced, and the target identification accuracy and the positioning precision are improved; through adaptive fast spatial pyramid pooling, in combination with a parameter-free attention module and an adaptive weight fusion mechanism, the capability of capturing detail information on feature maps with different scales is enhanced, and the target object sensing capability and the global semantic understanding capability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of PCB defect detection and machine vision, and particularly relates to a PCB defect detection method based on enhanced RTDETR. Technical Background

[0002] Industrial defect detection is an important link to ensure product quality and production efficiency, aiming to reduce losses and potential safety risks caused by product defects. In the industrial production process, it is a common practice to classify different types of defects and take corresponding treatment measures. By accurately identifying different types of defects (such as missing holes, mouse bite marks, open circuits, short circuits, protrusions or excess copper impurities on the PCB board), the qualified rate of products can be effectively improved and production costs can be reduced.

[0003] In traditional PCB surface defect detection methods, methods based on image processing and machine learning are usually used to detect and locate defect information in images. These methods require manual design and extraction of features, and their performance and robustness are not strong enough for changes in different scenarios and perspectives. Summary of the Invention

[0004] Object of the Invention: Aiming at the problems in the background technology, the present invention proposes a PCB defect detection method based on enhanced RTDETR, which can better learn the features in the image, enable the model to meet the real-time requirements and performance requirements of industrial detection, and improve the accuracy of detecting and locating PCB surface defects.

[0005] Technical Solution: The present invention proposes a PCB defect detection method based on enhanced RTDETR, including the following steps:

[0006] Step 1: Collect PCB defect pictures, classify and label them to form a PCB defect detection data set, and divide the data set into a training set, a validation set and a test set according to a certain ratio;

[0007] Step 2: Perform image preprocessing on the PCB defect detection data set;

[0008] Step 3: Build an enhanced RTDETR model, and the enhanced RTDETR model includes three parts: a feature extraction network Backbone, a feature fusion network Neck and a detection head Head;

[0009] The feature extraction network Backbone is composed of multiple residual modules connected in series to form a ResNet network; the feature fusion network Neck is composed of four CCA modules, one ASPPF module, one AIFI module, six CBS modules, five fusion modules, and two upsampling modules, which perform feature fusion on multi-scale information and primary semantic information to obtain a higher-order feature representation.

[0010] Step 4: Input the preprocessed image into the constructed enhanced DETR model for feature extraction and feature fusion, and finally input the extracted features into the detection module for classification and localization.

[0011] Further, the image preprocessing in Step 2 includes: enhancing the image data by any one or several of flipping, translation, and cropping, and using a Gaussian filter to denoise the image to make it clearer and more readable. Finally, convert the image after the above operations into tensor data readable by the model.

[0012] Further, the feature extraction network Backbone includes 2 CBS modules and 5 residual modules, and the 2 CBS modules and 5 residual modules are connected in series in sequence.

[0013] Further, the specific architecture of the feature fusion network Neck is as follows:

[0014] The output of the fifth residual module is used as the input of the ASPPF module. After the ASPPF module is connected to the AIFI module and passes through the first CBS module, the output feature after sampling by the first upsampling module is fused with the feature after the fourth residual module passes through the second CBS module through the first fusion module;

[0015] After the output of the first fusion module passes through the first CCA module, the third CBS module, and the second upsampling module in sequence, the feature is fused with the feature after the third residual module passes through the fourth CBS module through the second fusion module;

[0016] The feature after the second fusion module is fused passes through the second CCA module and the fifth CBS module, and then is fused with the feature output by the third CBS module through the third fusion module and then input to the third CCA module and the sixth CBS module, and then is fused with the feature output by the first CBS module through the fourth fusion module;

[0017] The feature after the fourth fusion module is fused is input to the fourth CCA module, and then is respectively fused with the feature output by the third CCA module and the feature output by the second CCA module through the fifth fusion module and then input to the detection head Head part.

[0018] Furthermore, the detection head Head part consists of an IoU-aware Query Selection module and a decoder Decoder. The IoU-aware Query Selection module does not exist during the inference stage. During training, it constrains the detector to generate high classification scores for features with high IoU and low classification scores for features with low IoU, so that the predicted bounding boxes corresponding to the Top-K features selected by the model according to the classification scores simultaneously have high classification scores and high IoU scores. The decoder Decoder divides the input features into three parts: Q, K, and V. The Q and K parts are added with learned position encodings and then used as the input of the multi-head self-attention module. The output result is added with a residual connection and then enters the feed-forward neural network (FFN). After combining with the residual connection again, the decoding work is completed. The FFN consists of two linear layers.

[0019] Furthermore, the CCA module includes a spatial attention mechanism SA branch, a Bottleneck branch, and a channel attention mechanism CA branch;

[0020] The input primary semantic features and multi-scale feature representations are weighted by the spatial positions of the spatial attention mechanism SA. The input primary semantic features and multi-scale feature representations automatically learn the weights of each channel under the action of the channel attention mechanism CA. The high-level semantic information of the two branches is weighted and fused with the semantic information of the Bottleneck branch.

[0021] Furthermore, the residual module includes a CBS layer with a 3×3 convolutional kernel and three CBS layers for channel adjustment. The input feature matrix first passes through a 1×1 CBS layer to adjust the input channels, then passes through a 3×3 CBS layer for feature extraction, and finally passes through a 1×1 CBS layer to adjust the channels again to select a more suitable semantic expression for the PCB detection task. Then, a skip connection is used to make the gradient flow smoother. For the case of dimension mismatch, there is an additional 1×1 CBS layer for dimension adjustment to achieve the addition operation.

[0022] Furthermore, the convolutional kernel size of the first residual module is 7×7, and a larger kernel convolution is used to obtain a larger receptive field; the convolutional kernels of the subsequent residual modules are all 3×3.

[0023] Furthermore, the ASPPF module integrates the SimAM mechanism into the SPPF module.

[0024] Beneficial effects:

[0025] 1. The CCA module adopted in the image feature fusion network of the present invention provides different degrees of spatial and channel information through weighted fusion without affecting the main body's gradient propagation, ultimately achieving the purpose of improving the overall performance of the model. The application of this module enables the algorithm to exhibit higher accuracy and recall rate in PCB defect detection, while maintaining low computational resource requirements.

[0026] 2. The ASPPF module used in the present invention further enhances the model's feature expression ability, adjusts the size of the feature matrix, and the ability of feature fusion by introducing the SimAM mechanism and adaptive weight fusion technology on the basis of maintaining the advantages of the original SPPF module. The overall model shows higher precision, faster speed, and better recall rate in the PCB defect detection task, and is particularly suitable for application scenarios that require efficient and accurate detection.

[0027] 3. The algorithm proposed in the present invention not only effectively optimizes the number of parameters and operation efficiency, but also significantly improves the detection accuracy and recall rate, while maintaining strong robustness and generalization ability. These advantages make it very suitable for PCB defect detection tasks with strict requirements for high precision, high speed, and high recall rate in industrial environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is the flow schematic diagram of the present invention.

[0029] Figure 2 is the overall framework diagram of the method of the present invention.

[0030] Figure 3 is the schematic diagram of the CCA module in the method of the present invention.

[0031] Figure 4 is the schematic diagram of the ASPPF module in the method of the present invention.

[0032] Figure 5 is the schematic diagram of the Decoder module in the method of the present invention.

[0033] Figure 6 is the training process diagram of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The present invention will be described in detail below with reference to the accompanying drawings and by way of examples.

[0035] The present invention discloses a PCB defect detection method based on enhanced RTDETR. The overall process is as Figure 1 shown, and specifically includes the following steps:

[0036] Step 1: Collect various PCB defect images, classify and label them to form a PCB defect detection dataset. Divide the classified image dataset into a training set, a validation set, and a test set according to the ratio of 7:2:1. The purpose of the division is to evaluate the performance and generalization ability of the model and prevent the model from overfitting on the training set. It includes: missing_hole, mouse-bite, open-circuit, short, spur, spurious-copper.

[0037] Step 2: Preprocess the collected images: such as using operations like image rotation, flipping, and translation to increase the diversity and richness of the data. And use a Gaussian filter to denoise the images to make them clearer and more readable, which helps improve the generalization ability of the model so that it can process images under different angles, scales, and lighting conditions. Finally, scale the images to a unified size and convert them into tensor data.

[0038] Step 3: Build an enhanced RTDETR network, as Figure 2 shown, which consists of three parts: one is the backbone network responsible for extracting semantic features at different scales, the second is the Neck part responsible for fusing feature matrices at different scales and obtaining semantic characteristics more suitable for the PCB defect task, and the third is the detection head module responsible for completing feature decoding from the fused semantics to perform class matching and target localization.

[0039] The backbone network includes 2 CBS modules and 5 residual modules, which are sequentially connected in series.

[0040] The residual module (ResNet Block) includes a CBS module with a 3×3 convolutional kernel and three CBS modules for channel adjustment. The residual module includes a CBS layer with a 3×3 convolutional kernel and three CBS layers for channel adjustment; the input feature matrix first passes through a 1×1 CBS layer to adjust the input channels, then passes through a 3×3 CBS layer for feature extraction, and finally passes through a 1×1 CBS layer to adjust the channels again to select a semantic expression more suitable for the PCB detection task. Then, a skip connection is used to make the gradient flow smoother. Then, for the case of dimension mismatch, there is an additional 1×1 CBS layer for dimension adjustment to achieve the addition operation.

[0041] The Neck part consists of four CCA modules, one ASPPF module, one AIFI module, six CBS modules, five Concat modules, and two Upsample modules.

[0042] The output of the fifth residual module serves as the input to the ASPPF module. After the ASPPF module is connected to the AIFI module, the output features after sampling through the first upsampling module after passing through the first CBS module are fused with the features after the fourth residual module passing through the second CBS module through the first fusion module.

[0043] After the output of the first fusion module, the features passing through the first CCA module, the third CBS module, and the second upsampling module in sequence are fused with the features after the third residual module passing through the fourth CBS module through the second fusion module.

[0044] The features after the second fusion module pass through the second CCA module and the fifth CBS module, and then are fused with the features output by the third CBS module through the third fusion module and input to the third CCA module and the sixth CBS module. After that, they are fused with the features output by the first CBS module through the fourth fusion module.

[0045] The features after the fourth fusion module are input to the fourth CCA module, and then are respectively fused with the features output by the third CCA module and the second CCA module through the fifth fusion module and then input to the detection head Head part.

[0046] The detection head part consists of an IoU-aware Query Selection module and a decoder Decoder. Depending on the algorithm mode, the IoU-aware Query Selection module does not exist in the inference stage. During training, it constrains the detector to generate high classification scores for features with high IoU and low classification scores for features with low IoU. Thus, the predicted bounding boxes corresponding to the Top-K features selected by the model according to the classification scores simultaneously have high classification scores and high IoU scores. Its role is to select a fixed number of features from the feature sequence output by the Encoder as object queries, which are mapped to confidence and bounding boxes by the prediction head after passing through the Decoder. The number of object queries is determined by the specific task scenario as a hyperparameter, and in the PCB defect detection task, the number of object queries is set to 100. The structure of the Decoder is as Figure 5 shown. The input features are divided into three parts: Q, K, and V. The Q and K parts are added with learnable position encoding (consisting of a convolutional module) and then used as the input to the multi-head self-attention module. After the output result is added with a residual connection and enters the feed-forward neural network (FFN), it is combined with the residual connection again to complete the decoding work, where the FFN consists of two linear layers.

[0047] The processed data is fed into the constructed deep convolutional model, and the preprocessed image data is input into the Backbone network. At this time, the input size of the image is 640×640. The input image data passes through five residual modules in sequence, and the results of each residual block are retained starting from the second residual module to provide a multi-scale feature matrix for the feature fusion module. The convolutional kernel size of the first residual block is 7×7, and a larger kernel convolution is used to obtain a larger receptive field. The convolutional kernels of the subsequent residual blocks are all 3×3. By using convolutional kernels of different sizes or strides, the receptive field of the convolutional operation can be effectively expanded, enabling the network to perceive a wider range of context information. The combined use of large and small convolutional kernels significantly reduces the number of parameters while keeping the receptive field unchanged. As the network depth increases, the size of the feature map gradually decreases with convolutional operations, forming semantic expressions at different scales. The feature expressions in the shallower layers of the model have less semantic information because they have undergone fewer convolutional operations and thus represent primary features such as contours and textures.

[0048] The primary features extracted by the Backbone network are fed into the feature fusion network (Neck). The input feature expressions first pass through a convolutional layer that uses 1×1 convolutional kernels to align the channels of the features. The three different-sized feature expressions (AIFI, the third and fourth residual blocks) are all input into the feature fusion network in the Neck part. The feature fusion network is designed similar to a feature pyramid to ensure sufficient interaction and fusion of the information flow and gradients between features of different sizes. Finally, the three-layer feature expressions are fused and input into the detection part. Among them, the deep semantic information enters the AIFI module through feature focusing of the ASPPF module to provide a high-quality deep feature expression. The decoder uniformly decodes the high-level feature representations output by the feature fusion network, and finally the decoded information is fed into the detection head module for final defect localization and classification.

[0049] The CCA module has three branches. After different processes, they are fused, the channels are adjusted, and then output. Among them, branch one is weighted by the spatial position of the spatial attention mechanism (SA), which helps the model better understand the object and scene structure in the semantic information, so as to more accurately learn higher-level feature expressions. Branch two automatically learns the weights of each channel under the action of the channel attention mechanism (CA) for the input semantic information, so that the network can emphasize the channels that are most helpful for the current task and reduce the attention to noise and irrelevant features. Branch three can better fuse multi-scale features under the action of Bottleneck, and weightedly fuse the high-level semantic information of the two branches with the semantic information of the Bottleneck branch.

[0050] The ASPPF integrates the SimAM mechanism into the SPPF module, performs the SimAM operation after each pooling operation to assign corresponding attention weights to each neuron, thereby improving the representation ability of the feature map, and inferring the three-dimensional attention weights of the feature map of this layer through the energy function without adding parameters to the original network. After each SimAM operation, information interaction will be carried out to pass the expression of the previous level downward. Finally, the semantic information is fused and output. This not only enables the feature matrix at this stage to retain more useful information, but also makes the model more generalizable.

[0051] Step 4: Obtain the trained model weights and test the PCB defect images.

[0052] To better adapt to the current PCB defect detection scenario, the loss function used in the present invention is the weighted sum of the recognition loss and the center loss.

[0053] The recognition loss is calculated by the fully supervised cross-entropy loss for classification. Given an input image x, the output of the deep convolutional neural network for K classes is defined as O 1 、O 2 、O 3 、...、O K , and the "Softmax" function is used to normalize each output value to a probability within the range [0,1], as shown in the following formula:

[0054]

[0055] where p i is the probability of the i-th class of the input image. If the true class of x is i, then y i =1, and the recognition loss is defined as the following equation:

[0056] L R (x,y) = -y i ·log p x

[0057] The center loss is given m input images x 1 、x 2 、···、x m in a training mini-batch, and the center of the deep features is defined for each of the K classes. c yi is the deep global local feature center of x i . The calculation formula of the center loss is as follows:

[0058]

[0059] With the above two loss terms, namely the recognition loss and the center loss, the overall loss function of the method can be written as the following formula:

[0060] L = (1 - α)L R + αL C

[0061] where α is a weight parameter used to balance each loss term in the overall loss function.

[0062] The performance evaluation metrics of the present invention adopt Precision, Recall, mean Average Precision (mAP), FPS (frames per second), and Parameters. The formulas for each evaluation metric are as follows:

[0063]

[0064] Among them, TP (True Positive): indicates that the classifier correctly classifies a positive sample (positive class) as a positive sample; FP (False Positive): indicates that the classifier incorrectly classifies a negative sample (negative class) as a positive sample; TN (True Negative): indicates that the classifier correctly classifies a negative sample as a negative sample; FN (False Negative): indicates that the classifier incorrectly classifies a positive sample as a negative sample.

[0065]

[0066] where C represents the number of classes, and AP i represents the Average Precision of the i-th class.

[0067] FPS represents the number of images detected within one second; Parameters represents the total number of parameters of the entire model, and the size and memory occupancy of the model can be calculated using this parameter.

[0068] Table 1

[0069]

[0070] A PCB defect detection method based on enhanced RTDETR proposed by the present invention. Table 1 is a comparison table of the speed and performance between the method of the present invention and mainstream algorithms. The recognition accuracy of the model of the present invention reaches 97%, and the recall rate reaches 93%. Compared with traditional models, this model has higher recognition accuracy and a wider range of application scenarios. The PCB defect detection method based on this model determines whether the sample is a qualified product by identifying and locating the surface defects of the PCB. This model provides an efficient, accurate, and real-time classification method for industrial defect detection, and can improve the speed of PCB product detection to a certain extent.

[0071] Compared with traditional defect detection methods, the use of the enhanced RDETR algorithm significantly reduces the model complexity, while improving the detection accuracy and comprehensiveness. The model proposed in the present invention, by designing the CCA module and the ASPPF module, not only improves the recognition accuracy of various defects, but also is applicable to various complex PCB defect detections, providing an efficient and accurate technical solution for industrial PCB defect detection. This improved algorithm can quickly and accurately detect subtle defects, thus ensuring the quality control level of the production line.

[0072] The above embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable those who are familiar with this technology to understand the content of the present invention and implement it accordingly, and it should not be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit and essence of the present invention should be covered within the protection scope of the present invention.

Claims

1. A PCB defect detection method based on enhanced RTDETR, characterized in that: The steps include: Step 1: Collect PCB defect images, classify and annotate them to form a PCB defect detection dataset, and divide the dataset into training set, validation set and test set according to a certain ratio; Step 2: Perform image preprocessing on the PCB defect detection dataset; Step 3: Build an enhanced RTDETR model, which includes three parts: feature extraction network Backbone, feature fusion network Neck and detection head Head; The feature extraction network Backbone uses multiple residual modules in series to form a ResNet network; the feature fusion network Neck is composed of four CCA modules, one ASPPF module, one AIFI module, six CBS modules, five fusion modules and two upsampling modules, which fuses multi-scale information and primary semantic information to obtain higher-order feature expressions. Step 4: Input the preprocessed image into the built enhanced DETR model for feature extraction and feature fusion, and finally input the extracted features into the detection module for classification and positioning.

2. A PCB defect detection method based on enhanced RTDETR according to claim 1, characterized in that: The image preprocessing in step 2 includes: enhancing the image data by any one or more methods of flipping, translating, and cropping, and denoising the image using a Gaussian filter to make the image clearer and more readable, and finally converting the image after the above operations into tensor data readable by the model.

3. The PCB defect detection method based on enhanced RTDETR according to claim 1, characterized in that: The feature extraction network Backbone includes 2 CBS modules and 5 residual modules, and the 2 CBS modules and the 5 residual modules are connected in series in sequence.

4. The PCB defect detection method based on enhanced RTDETR according to claim 3 is characterized in that: The specific architecture of the feature fusion network Neck is as follows: The output of the fifth residual module is used as the input of the ASPPF module. After the ASPPF module is connected with the AIFI module, the output features sampled by the first upsampling module after passing through the first CBS module and the output features of the fourth residual module after passing through the second CBS module are fused through the first fusion module; The features of the output of the first fusion module after passing through the first CCA module, the third CBS module, and the second upsampling module in sequence are fused with the features of the output of the third residual module after passing through the fourth CBS module through the second fusion module; The features fused by the second fusion module are passed through the second CCA module and the fifth CBS module, and then fused with the features output by the third CBS module through the third fusion module, and then input into the third CCA module and the sixth CBS module, and then fused with the features output by the first CBS module through the fourth fusion module; The features fused by the fourth fusion module are input to the fourth CCA module and then respectively combined with the features output by the third CCA module and the features output by the second CCA module through the fifth fusion module and then input to the detection head.

5. The PCB defect detection method based on enhanced RTDETR according to claim 1, characterized in that: The detection head part is composed of an IoU-aware Query Selection module and a decoder. The IoU-aware Query Selection module does not exist in the reasoning stage. During training, the detector is constrained to generate high classification scores for features with high IoU and low classification scores for features with low IoU, so that the prediction boxes corresponding to the Top-K features selected by the model according to the classification scores have both high classification scores and high IoU scores; the decoder divides the input features into three parts, Q, K, and V, adds the Q and K parts to the learning position encoding, and then uses them as the input of the multi-head self-attention module. The output result is added with the residual connection and enters the feedforward neural network (FFN), and then combined with the residual connection again to complete the decoding work, where the FFN consists of two linear layers.

6. A PCB defect detection method based on enhanced RTDETR according to claim 4, characterized in that: The CCA module includes a spatial attention mechanism SA branch, a Bottleneck branch, and a channel attention mechanism CA branch; The input primary semantic features and multi-scale feature expressions are weighted by the spatial position of the spatial attention mechanism SA, and the weight of each channel is automatically learned under the action of the channel attention mechanism CA, and the high-level semantic information of the two branches is weightedly fused with the semantic information of the Bottleneck branch.

7. A PCB defect detection method based on enhanced RTDETR according to claim 1, characterized in that: The residual module includes a CBS layer with a convolution kernel of 3×3 and three CBS layers for channel adjustment; the input feature matrix first passes through a 1×1 CBS layer to adjust the input channel, then passes through a 3×3 CBS layer for feature extraction, and finally passes through a 1×1 CBS layer to adjust the channel selection again to better match the semantic expression of the PCB inspection task, and then uses jump connections to make the gradient flow smoother. Then, for the case of dimensional mismatch, there is an additional 1×1 CBS layer for adjusting the dimension to achieve the addition operation.

8. A PCB defect detection method based on enhanced RTDETR according to claim 3, characterized in that: The convolution kernel size of the first residual module is 7×7, and a larger receptive field is obtained through large-kernel convolution; the convolution kernels of subsequent residual modules are all 3×3.

9. A PCB defect detection method based on enhanced RTDETR according to claim 1, characterized in that: The ASPPF module integrates the SimAM mechanism into the SPPF module.

Images