PCBA tiny defect detection method based on improved DETR

Abstract

The invention provides a PCBA (printed circuit board assembly) tiny defect detection method based on an improved DETR (digital electronic tag). Firstly, a PCBA image data set is obtained and divided. And then, an RT-DETR backbone network is improved, and a GhostConv module and a MambaOut module are introduced to construct a lightweight backbone GMONet so as to improve feature extraction and calculation efficiency. The AIFI in the coding stage is optimized, and a TAIFI module for counting perceptual attention is designed to enhance tiny feature expression. And in the decoding stage, the improved CAFF module is used for replacing the original RepC3 module. And adding GSConv at each convolution position of the neck of the model to form a consistent lightweight convolution system so as to further reduce the parameters and improve the reasoning speed. Training and testing are carried out based on the improved model, the parameter quantity and the calculated quantity can be remarkably compressed while the detection precision is kept, rapid and accurate recognition of PCBA tiny defects is achieved, and the method is suitable for a real-time industrial visual detection environment.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of detection, and in particular to a PCBA micro defect detection method based on an improved DETR. BACKGROUND

[0002] PCBA is widely used in electronic product manufacturing, and its quality directly affects the performance and reliability of the terminal product. With the development of electronic products towards miniaturization and high precision, micro defects on the surface of PCBA have increasingly become an important factor affecting product reliability. Micro defects usually have the characteristics of small size, weak texture, blurred edges, uneven distribution, and are more difficult to be accurately recognized by traditional visual detection methods under complex lighting, reflection or multi-angle shooting conditions.

[0003] The existing PCBA defect detection methods mainly include traditional computer vision methods based on manual feature extraction and target detection methods based on deep learning. The traditional method relies on manually designed features, and has limited detection performance when facing diversified defect types and complex backgrounds, and is prone to missed detection or false detection. In recent years, deep learning methods have made significant progress in target detection, and RT-DETR has realized end-to-end detection by converting the target detection task into a set prediction problem, eliminating the cumbersome candidate frame generation and post-processing steps. However, RT-DETR still has deficiencies in micro target detection and low contrast target recognition: the feature response to micro defects is weak and is easily covered by background interference; the multi-scale feature fusion is insufficient, and it is difficult to balance global semantic information and local detail information; the calculation amount and model parameters are large, which is not conducive to real-time detection on edge devices. SUMMARY

[0004] The technical problem to be solved by the present application is that the existing PCBA micro defect detection method has the problems of low detection accuracy, difficulty in recognizing micro targets, poor adaptability to low contrast and complex background, and limited real-time detection capability on edge devices.

[0005] Therefore, the present application provides a PCBA micro defect detection method based on an improved DETR, which improves the recognition accuracy of micro, low contrast and edge blur defects on the one hand, and realizes model lightweight and real-time inference while maintaining high detection accuracy on the other hand, and is suitable for automatic detection in industrial production environment.

[0006] The technical scheme adopted by the present application to solve its technical problem is:

[0007] A DETR-based PCBA micro defect detection method, comprising the following steps:

[0008] S1, obtaining image data set of PCBA to be detected, and dividing the image into data set to obtain training set, test set and validation set;

[0009] S2. Improve the backbone network of the RT-DETR architecture and build a backbone network GMONet that integrates GhostConv and MambaOut mechanisms to enhance feature extraction capabilities and network computation efficiency.

[0010] S3. Improve the AIFI module in the RT-DETR encoding stage by designing a TAIFI module with a statistically perceptual attention mechanism to enhance the model's ability to express subtle features.

[0011] S4. In the RT-DETR decoding stage, the original feature fusion module RepC3 is improved by adding a context-aware feature fusion module CAFF. The CAFF module introduces a context anchor attention mechanism on the basis of inheriting the RepNCSPELAN4 backbone structure to improve the feature fusion effect.

[0012] S5. Introduce GSConv convolutions at all convolution positions in the neck of the RT-DETR model to build a consistent lightweight convolution system, thereby further reducing the number of model parameters and improving inference speed.

[0013] S6. Train the RT-DETR model improved by steps S2 to S5, and input the test set images into the trained model to obtain the detection results of PCBA micro-defects.

[0014] Furthermore, step S1 specifically includes:

[0015] S11. Use circuit board defect image samples from the publicly available PCBA-DET dataset. The samples include circuit board images from multiple angles and under different lighting conditions, and label the defect types. The defect types include: loose fan screws, missing fan screws, loose motherboard screws, missing motherboard screws, loose fan wiring, missing fan wiring, fan scratches, and motherboard scratches.

[0016] S12. Divide the image samples according to the ratio of training set, validation set and test set, wherein the ratio is 7:1:2;

[0017] S13. Perform data augmentation processing on the training set and validation set samples. The data augmentation includes rotation, translation, brightness adjustment and noise perturbation to improve the generalization ability of the detection model.

[0018] Through the above steps, the method can construct a diverse and balanced training dataset and generate high-quality samples for model training and validation, thereby improving the model's accuracy and robustness in identifying minor PCBA defects under different angles and lighting conditions.

[0019] Furthermore, the improvement to the backbone network of the RT-DETR model aims to enhance the model's feature representation ability and computational efficiency in identifying minute defects.

[0020] In step S2, the improvements to the backbone network specifically include:

[0021] S21. The input PCBA image first undergoes low-level feature extraction via the GhostConv module. The GhostConv module utilizes a cheap operation to reconstruct feature channels, reducing redundant convolution calculations and thus effectively reducing computational complexity while preserving the original spatial structure information, enabling rapid encoding of high-resolution images.

[0022] S22. Multiple dynamic aggregation modules (GMO-Block 1 to GMO-Block 4) are sequentially set in the backbone network. Each GMO-Block consists of a standard convolutional layer and a DMambaOut module. The DMambaOut module adopts a dual-path feature modeling mechanism, including a shortcut path and a deep modeling path: the shortcut path is used to preserve shallow detail information and directly skips subsequent convolutional operations using identity mapping to ensure stable gradient propagation; the deep modeling path inputs into multiple GatedCNN Blocks and achieves dynamic adjustment and context fusion of features through gated convolution.

[0023] S23, GatedCNNBlock, consists of a normalization layer, two parallel linear paths, and a gating fusion mechanism. Its computation process includes:

[0024] First, the input feature X is normalized.

[0025] Then, feature responses and gating weights are extracted through convolutional paths and gating paths, respectively.

[0026] The two components are modulated by the Sigmoid function and then fused element-wise by multiplication to obtain the enhanced feature output.

[0027] The calculation formula is as follows:

[0028]

[0029] in, This is the intermediate representation of the input X after normalization. CNN The output features of the Gated CNN Block are represented by Conv(·), which is a linear transformation, Norm(·) represents normalization, w1, w2, and w3 represent different weights, σ(·) is the GeLU activation function, and ⊙ represents element-wise multiplication.

[0030] S24. The deep modeling features and shortcut path features output by each DMambaOut module are concatenated along the channel dimension and fused through a 1×1 convolution to form a multi-scale feature representation. This structure can improve the model's sensitivity to minor defects and context modeling ability without increasing network depth, ultimately generating feature maps for use by the detection head.

[0031] Furthermore, to enhance the model's ability to identify minor defects with low contrast and blurred edges, the TAIFI module is introduced;

[0032] In step S3, the improvements to the AIFI module specifically include:

[0033] S31. Divide the multi-scale feature map output in step S2 into several tokens to form a feature sequence. Where B is the batch size, N h Let D be the number of tokens, and D be the feature dimension of each token.

[0034] S32. Calculate the statistical importance weight Π for each token to quantify the significance of the token:

[0035] First, calculate the squared L2 norm of each token feature as the statistical energy;

[0036] The statistical energy is scaled using a learnable temperature parameter T;

[0037] The statistical importance weight of each token is obtained by normalization using the Softmax function.

[0038] S33. The token feature sequence w is weighted and summed using the statistical weights Π to obtain the context vector d, which serves as the global feature summary of the sequence.

[0039] The context vector d is transformed nonlinearly to generate a context conditioning term. Used for uniform feature modulation of all tokens;

[0040] S34. Apply the statistical weights and context adjustment term A to the original feature sequence w:

[0041] Y weighted =w⊙Π⊙A

[0042] Through the above steps, the key token features are dynamically enhanced while suppressing responses in low-information or noisy regions.

[0043] S35. Input the weighted feature sequence output in step S34 into the subsequent Feed-Forward network of the Transformer encoder to form the complete TAIFI encoder layer output, which provides high-quality feature representation for the subsequent decoding stage and significantly improves the detection accuracy of small, low-contrast defects.

[0044] Furthermore, the CAFF module is an improved feature fusion module;

[0045] In step S4, the improved feature fusion module specifically includes:

[0046] S41. For the multi-scale feature maps from the model encoding stage, first increase the dimensionality through 1×1 convolution, then divide them into two paths:

[0047] A direct jump connection is used to preserve shallow textures and edge information;

[0048] The other path serves as the main trunk, passing through two deep modeling units in sequence. Each unit includes a RepNCSP module and a 3×3 convolution to enhance semantic expressiveness.

[0049] S42. The features output by the main path and the jump connection path are concatenated in the channel dimension to form a multi-scale fused feature.

[0050] S43. Apply contextual anchor attention to the fused features, including the following sub-steps:

[0051] Global pooling is performed on the fused features to extract contextual anchor information;

[0052] Use k×1 and 1×k depth separable convolutions to model the context response in the horizontal and vertical directions respectively;

[0053] A spatial saliency map is generated by fusing horizontal and vertical responses through a gating mechanism.

[0054] The spatial saliency map is multiplied element-wise with the fused features to enhance key features and suppress background interference.

[0055] S44. Output the feature map processed by the CAFF module for defect localization and classification in the subsequent DETR decoding stage, so as to improve the detection accuracy of the model in complex backgrounds and small target scenes.

[0056] Furthermore, in order to significantly reduce the computational cost and number of parameters of the model while maintaining high detection accuracy, GSConv convolution is introduced in the neck of the RT-DETR model;

[0057] In step S5, the GSConv convolution specifically includes:

[0058] S51. Divide the input features along the channel dimension into a main path and a secondary path. The main path compresses the channels using standard convolution to extract key semantic information, while the secondary path uses depthwise separable convolution to preserve redundant feature representations and enhance feature diversity.

[0059] S52. The outputs of the main path and the secondary path are concatenated in the channel dimension, and information interaction between different channels is achieved through channel shuffle operation, thereby improving the feature fusion effect.

[0060] S53. Input the processed features into the detection head to significantly reduce the amount of computation and model parameters while maintaining high detection accuracy, so as to realize real-time defect detection suitable for edge devices.

[0061] Furthermore, step S6 specifically includes:

[0062] S61. Input the improved RT-DETR model from steps S2 to S5 into the training set image, train the model, optimize the network parameters through the Adam loss function, so that the model can accurately capture the small defect features in the PCBA image.

[0063] S62. Input the trained improved RT-DETR model into the test set images and output the detection results of small defects, including the category and location information of the defects;

[0064] S63. Evaluate and visualize the test results to enable rapid and accurate identification of minor defects in PCBAs, and make it applicable to real-time inspection tasks of edge devices.

[0065] The beneficial effects of this invention are:

[0066] In terms of accuracy: The GMO-DETR model constructed in this invention achieves a core accuracy index mAP@0.5 of 98.22% on the public PCBA dataset. While ensuring high detection accuracy, it effectively improves the ability to identify minute defects, especially showing excellent detection performance in low contrast and complex texture backgrounds.

[0067] Recognition efficiency: While significantly reducing the number of model parameters and computational load, this invention achieves an inference speed of up to 174 FPS, significantly improving the real-time detection capability of the model on edge devices, and fully demonstrating the great advantages of the detection method in efficient recognition.

[0068] This invention successfully achieves a balance between model complexity and detection performance. Compared with the RT-DETR model, the number of parameters and computational cost are reduced by approximately 39.4% and 28.7% respectively, while maintaining high accuracy and fast inference. It provides a feasible and efficient technical solution for real-time and efficient detection of PCBA defects in industrial vision. Attached Figure Description

[0069] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0070] Figure 1 This is a flowchart of the PCBA micro-defect detection method based on the improved DETR of the present invention.

[0071] Figure 2 This is the network structure diagram of the improved RT-DETR model of this invention.

[0072] Figure 3 This is a network structure diagram of GMONet, the backbone network proposed in this invention.

[0073] Figure 4 This is a network structure diagram of the TAIFI module proposed in this invention.

[0074] Figure 5 This is a network structure diagram of the CAFF module proposed in this invention.

[0075] Figure 6 This is a diagram of the GSConv convolutional network structure shown in this invention.

[0076] Figure 7 This is a bubble chart comparing the performance of this invention with other models.

[0077] Figure 8 This is a heatmap comparing the traditional RT-DETR model.

[0078] Figure 9 This is the ablation experiment results table. Detailed Implementation

[0079] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0080] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0081] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0082] like Figure 1 and Figure 6 As shown, the PCBA minute defect detection method based on improved DETR in this embodiment includes the following steps: S1, acquiring the image dataset of the PCBA to be detected, and dividing the images into training set, test set, and validation set; S2, improving the backbone network of the RT-DETR architecture by constructing a backbone network GMONet that integrates GhostConv and MambaOut mechanisms to improve feature extraction capability and network computation efficiency; S3, improving the AIFI module in the RT-DETR encoding stage by designing a TAIFI module with a statistically perceptual attention mechanism to enhance the model's ability to express minute features; S4, in RT... - In the DETR decoding stage, the original feature fusion module RepC3 is improved by adding the context-aware feature fusion module CAFF. The CAFF module, based on the RepNCSPELAN4 backbone structure, introduces a context anchor attention mechanism to improve the feature fusion effect. S5, GSConv convolution is introduced at all convolution positions in the neck of the RT-DETR model to build a consistent lightweight convolution system to further reduce the number of model parameters and improve inference speed. S6, The RT-DETR model improved by steps S2 to S5 is trained, and the test set images are input into the trained model to obtain the detection results of PCBA minor defects.

[0083] It should be noted that the PCBA-DET dataset is a publicly available detection dataset focused on PCBA defect detection. This dataset encompasses images taken from different angles, including top, side, and tilt angles. The dataset contains 4,000 original images and 2,384 data-augmented images, labeled with eight original defects, such as loose fan screws and missing motherboard screws. In this study, the dataset was randomly partitioned into training, validation, and test sets in a 7:1:2 ratio for experimental purposes.

[0084] This embodiment enables real-time detection of minute defects in PCBAs. On the one hand, it achieves model lightweighting through multi-module collaborative optimization, significantly reducing the amount of computation and parameters. On the other hand, it improves inference speed while maintaining detection accuracy, realizing rapid and accurate identification of minute defects in PCBAs, and is suitable for industrial vision inspection scenarios of edge devices.

[0085] In this embodiment, as Figure 3 As shown, GMONet is the main network module in the backbone network, primarily composed of the GhostConv input guidance module and four-stage dynamic aggregation modules (GMO-Block 1 to GMO-Block 4); Figure 4 As shown, the TAIFI module is a novel token-aware feature interaction integration module. Inspired by the statistical self-attention concept in ToST, this module aims to build a more efficient feature interaction and enhancement path; such as... Figure 5 As shown, the CAFF module, while inheriting the RepNCSPELAN4 backbone structure, introduces a context anchor attention mechanism to improve the context modeling capability and spatial selectivity during the fusion stage; for example... Figure 6 As shown, the lightweight GSConv convolution module achieves a balance between feature modeling capability and computational efficiency by introducing grouped convolution, depthwise separable convolution, and channel rearrangement mechanism, making it particularly suitable for PCBA defect identification tasks with real-time detection and resource constraints.

[0086] In step S2, the improvements to the backbone network include:

[0087] S21. The input PCBA image first undergoes low-level feature extraction via the GhostConv module. The GhostConv module utilizes a cheap operation to reconstruct feature channels, reducing redundant convolution calculations and thus effectively reducing computational complexity while preserving the original spatial structure information, enabling rapid encoding of high-resolution images.

[0088] S22. Multiple dynamic aggregation modules (GMO-Block1 to GMO-Block4) are sequentially set in the backbone network. Each GMO-Block consists of a standard convolutional layer and a DMambaOut module. The DMambaOut module adopts a dual-path feature modeling mechanism, including a shortcut path and a deep modeling path: the shortcut path is used to preserve shallow detail information and directly skips subsequent convolutional operations using identity mapping to ensure stable gradient propagation; the deep modeling path inputs into multiple Gated CNN Blocks and achieves dynamic adjustment and context fusion of features through gated convolution.

[0089] S23, GatedCNNBlock, consists of a normalization layer, two parallel linear paths, and a gating fusion mechanism. Its computation process includes:

[0090] First, the input feature X is normalized.

[0091] Then, feature responses and gating weights are extracted through convolutional paths and gating paths, respectively.

[0092] The two components are modulated by the Sigmoid function and then fused element-wise by multiplication to obtain the enhanced feature output.

[0093] The calculation formula is as follows:

[0094]

[0095] in, This is the intermediate representation of the input X after normalization. CNN The output features of the Gated CNN Block are represented by Conv(·), which is a linear transformation, Norm(·) represents normalization, w1, w2, w3 represent different weights, σ(·) is the GeLU activation function, and ⊙ represents element-wise multiplication.

[0096] S24. The deep modeling features and shortcut path features output by each DMambaOut module are concatenated along the channel dimension and fused through a 1×1 convolution to form a multi-scale feature representation. This structure can improve the model's sensitivity to minor defects and context modeling ability without increasing network depth, ultimately generating feature maps for use by the detection head.

[0097] Based on this, in order to enhance the model's feature interaction capabilities and global context awareness capabilities, a new TAIFI module is proposed to improve the original AIFI module.

[0098] In this embodiment, the improved TAIFI module includes:

[0099] S31. Divide the multi-scale feature map output in step S2 into several tokens to form a feature sequence. Where B is the batch size, N h Let D be the number of tokens, and D be the feature dimension of each token.

[0100] S32. Calculate the statistical importance weight Π for each token to quantify the significance of the token:

[0101] First, calculate the squared L2 norm of each token feature as the statistical energy;

[0102] The statistical energy is scaled using a learnable temperature parameter T;

[0103] The statistical importance weight of each token is obtained by normalization using the Softmax function.

[0104] S33. The token feature sequence w is weighted and summed using the statistical weights Π to obtain the context vector d, which serves as the global feature summary of the sequence.

[0105] The context vector d is transformed nonlinearly to generate a context conditioning term. Used for uniform feature modulation of all tokens;

[0106] S34. Apply the statistical weights Π and the context adjustment term A to the original feature sequence w:

[0107] Y weighted =w⊙Π⊙A

[0108] Through the above steps, the key token features are dynamically enhanced while suppressing responses in low-information or noisy regions.

[0109] S35. Input the weighted feature sequence output in step S34 into the subsequent Feed-Forward network of the Transformer encoder to form the complete TAIFI encoder layer output, which provides high-quality feature representation for the subsequent decoding stage and significantly improves the detection accuracy of small, low-contrast defects.

[0110] After completing the feature optimization in the encoding stage, in order to further improve the feature fusion capability and spatial context modeling performance of the model in the decoding stage, this embodiment improves the decoding structure of RT-DETR and proposes a feature fusion module CAFF with a context-aware mechanism to replace the original RepC3 module.

[0111] In this embodiment, the improved CAFF module includes:

[0112] S41. For the multi-scale feature maps from the model encoding stage, first increase the dimensionality through 1×1 convolution, then divide them into two paths:

[0113] A direct jump connection is used to preserve shallow textures and edge information;

[0114] The other path serves as the main trunk, passing through two deep modeling units in sequence. Each unit includes a RepNCSP module and a 3×3 convolution to enhance semantic expressiveness.

[0115] S42. The features output by the main path and the jump connection path are concatenated in the channel dimension to form a multi-scale fused feature.

[0116] S43. Apply contextual anchor attention to the fused features, including the following sub-steps:

[0117] Global pooling is performed on the fused features to extract contextual anchor information;

[0118] Use k×1 and 1×k depth separable convolutions to model the context response in the horizontal and vertical directions respectively;

[0119] A spatial saliency map is generated by fusing horizontal and vertical responses through a gating mechanism.

[0120] The spatial saliency map is multiplied element-wise with the fused features to enhance key features and suppress background interference.

[0121] S44. Output the feature map processed by the CAFF module for defect localization and classification in the subsequent DETR decoding stage, so as to improve the detection accuracy of the model in complex backgrounds and small target scenes.

[0122] After optimizing the CAFF module, in order to further reduce the overall computational load and parameter scale of the model and improve the real-time detection performance on edge devices, this embodiment introduces a lightweight convolution module GSConv into the neck structure of the RT-DETR model to build a unified and efficient convolution system.

[0123] In this embodiment, the introduction of GSConv convolution specifically includes:

[0124] S51. Divide the input features along the channel dimension into a main path and a secondary path. The main path compresses the channels using standard convolution to extract key semantic information, while the secondary path uses depthwise separable convolution to preserve redundant feature representations and enhance feature diversity.

[0125] S52. The outputs of the main path and the secondary path are concatenated in the channel dimension, and information interaction between different channels is achieved through channel shuffle operation, thereby improving the feature fusion effect.

[0126] S53. Input the processed features into the detection head to significantly reduce the amount of computation and model parameters while maintaining high detection accuracy, so as to realize real-time defect detection suitable for edge devices.

[0127] After improving the network structure, the RT-DETR model optimized through steps S2 to S5 is trained.

[0128] In this embodiment, model training and result display specifically include:

[0129] S61. Input the improved RT-DETR model from steps S2 to S5 into the training set image, train the model, optimize the network parameters through the Adam loss function, so that the model can accurately capture the small defect features in the PCBA image.

[0130] S62. Input the trained improved RT-DETR model into the test set images and output the detection results of small defects, including the category and location information of the defects;

[0131] S63. Evaluate and visualize the test results to enable rapid and accurate identification of minor defects in PCBAs, and make it applicable to real-time inspection tasks of edge devices.

[0132] It should be noted that a fixed hyperparameter configuration was used to ensure consistency across model training, validation, and testing. Specifically, the input image resolution was set to 640×640 pixels. The model ran for a total of 200 epochs. The optimizer used was AdamW, with configuration parameters including a batch size of 8, a warm-up epoch of 2000 iterations, and a learning rate of 0.0001. This configuration helps the model learn more stably and adapt to the complexity of small object detection tasks, thus exhibiting excellent performance under various test conditions.

[0133] In this embodiment, as Figure 7 As shown, this invention maintains its advantage in mAP metrics compared to RT-DETR and other mainstream models, while significantly reducing model complexity. Figure 8 As shown in the heatmap, the response of this invention to minute defect areas is more concentrated, suppressing background noise interference; as Figure 9 As shown in the ablation experiment, each improved module plays a key role in enhancing the overall performance. In summary, this embodiment enables rapid and accurate detection of minute defects in PCBAs on edge devices, providing an efficient and feasible technical solution for real-time industrial vision inspection.

[0134] It should be noted that this example evaluated the effectiveness of nine object detection models in PCBA detection. The evaluated models included RT-DETR, YOLOv5 to YOLO13, and MambaYOLO. Compared to these models, our improved method achieves higher practical inference efficiency and accuracy while maintaining a compact structure. Furthermore, this example uses common object detection metrics, including mAP50 (mean accuracy at IoU = 0.5) and mAP50-95 (mean accuracy at IoU = 0.5:0.05:0.95). Computational efficiency metrics are also reported: FLOPs (G), number of parameters (M), and inference speed (FPS). These multi-dimensional comparisons validate the model's ability to balance performance and efficiency from multiple perspectives.

[0135] In summary, the method for detecting material shortages in flexible circuit board tape mounting according to the present invention has the following advantages:

[0136] In terms of accuracy: The GMO-DETR model constructed in this invention achieves a core accuracy index mAP@0.5 of 98.22% on the public PCBA dataset. While ensuring high detection accuracy, it effectively improves the ability to identify minute defects, especially showing excellent detection performance in low contrast and complex texture backgrounds.

[0137] Recognition efficiency: While significantly reducing the number of model parameters and computational load, this invention achieves an inference speed of up to 174 FPS, significantly improving the real-time detection capability of the model on edge devices, and fully demonstrating the great advantages of the detection method in efficient recognition.

[0138] This invention successfully achieves a balance between model complexity and detection performance. Compared with the RT-DETR model, the number of parameters and computational cost are reduced by approximately 39.4% and 28.7% respectively, while maintaining high accuracy and fast inference. It provides a feasible and efficient technical solution for real-time and efficient detection of PCBA defects in industrial vision.

[0139] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined by the scope of the claims.

Claims

1. A PCBA micro-defect detection method based on improved DETR, characterized in that, Includes the following steps: S1. Obtain the image dataset of the PCBA to be detected, and divide the images into training set, test set and validation set; S2. Improve the backbone network of the RT-DETR architecture and build a backbone network GMONet that integrates GhostConv and MambaOut mechanisms to enhance feature extraction capabilities and network computation efficiency. S3. Improve the AIFI module in the RT-DETR encoding stage by designing a TAIFI module with a statistically perceptual attention mechanism to enhance the model's ability to express subtle features. S4. In the RT-DETR decoding stage, the original feature fusion module RepC3 is improved by adding a context-aware feature fusion module CAFF. The CAFF module introduces a context anchor attention mechanism on the basis of inheriting the RepNCSPELAN4 backbone structure to improve the feature fusion effect. S5. Introduce GSConv convolutions at all convolution positions in the neck of the RT-DETR model to build a consistent lightweight convolution system, thereby further reducing the number of model parameters and improving inference speed. S6. Train the RT-DETR model improved by steps S2 to S5, and input the test set images into the trained model to obtain the detection results of PCBA micro-defects.

2. The PCBA micro-defect detection method based on improved DETR as described in claim 1, characterized in that, Step S1 specifically includes: S11. Use circuit board defect image samples from the publicly available PCBA-DET dataset. The samples include circuit board images from multiple angles and under different lighting conditions, and label the defect types. The defect types include: loose fan screws, missing fan screws, loose motherboard screws, missing motherboard screws, loose fan wiring, missing fan wiring, fan scratches, and motherboard scratches. S12. Divide the image samples according to the ratio of training set, validation set and test set, wherein the ratio is 7:1:2; S13. Perform data augmentation processing on the training set and validation set samples. The data augmentation includes rotation, translation, brightness adjustment and noise perturbation to improve the generalization ability of the detection model.

3. The PCBA micro-defect detection method based on improved DETR as described in claim 1, characterized in that, Step S2 includes the design and feature extraction process of the backbone network structure. The backbone network is GMONet, which is used to extract multi-scale spatial features and deep semantic information from the input PCBA image to balance detection accuracy and model lightweighting. Specifically, it includes the following steps: S21. The input PCBA image first undergoes low-level feature extraction via the GhostConv module. The GhostConv module utilizes a cheap operation to reconstruct feature channels, reducing redundant convolution calculations and thus effectively reducing computational complexity while preserving the original spatial structure information, enabling rapid encoding of high-resolution images. S22. Multiple dynamic aggregation modules (GMO-Block1 to GMO-Block4) are sequentially set in the backbone network. Each GMO-Block consists of a standard convolutional layer and a DMambaOut module. The DMambaOut module adopts a dual-path feature modeling mechanism, including a shortcut path and a deep modeling path: the shortcut path is used to preserve shallow detail information and directly skips subsequent convolutional operations using identity mapping to ensure stable gradient propagation; the deep modeling path inputs into multiple Gated CNN Blocks and achieves dynamic adjustment and context fusion of features through gated convolution. S23, the Gated CNN Block, consists of a normalization layer, two parallel linear paths, and a gating fusion mechanism. Its computation process includes: First, the input feature X is normalized. Then, feature responses and gating weights are extracted through convolutional paths and gating paths, respectively. The two components are modulated by the Sigmoid function and then fused element-wise by multiplication to obtain the enhanced feature output. The calculation formula is as follows: in, This is the intermediate representation of the input X after normalization. CNN The output features of the Gated CNN Block are represented by Conv(·), which is a linear transformation, Norm(·) represents normalization, w1, w2, and w3 represent different weights, σ(·) is the GeLU activation function, and ⊙ represents element-wise multiplication. S24. The deep modeling features and shortcut path features output by each DMambaOut module are concatenated along the channel dimension and fused through a 1×1 convolution to form a multi-scale feature representation. This structure can improve the model's sensitivity to minor defects and context modeling ability without increasing network depth, ultimately generating feature maps for use by the detection head.

4. The PCBA micro-defect detection method based on improved DETR as described in claim 1, characterized in that, Step S3 includes introducing a TAIFI module with a statistically perceptual attention mechanism into the feature map during the model encoding stage. The specific steps are as follows: S31. Divide the multi-scale feature map output in step S2 into several tokens to form a feature sequence. Where B is the batch size, N h Let D be the number of tokens, and D be the feature dimension of each token. S32. Calculate the statistical importance weight Π for each token to quantify the significance of the token: First, calculate the squared L2 norm of each token feature as the statistical energy; The statistical energy is scaled using a learnable temperature parameter T; The statistical importance weight of each token is obtained by normalization using the Softmax function. S33. The token feature sequence w is weighted and summed using the statistical weights Π to obtain the context vector d, which serves as the global feature summary of the sequence. The context vector d is transformed nonlinearly to generate a context conditioning term. Used for uniform feature modulation of all tokens; S34. Apply the statistical weights Π and the context adjustment term A to the original feature sequence w: AND weighted =w⊙Π⊙A Through the above steps, the key token features are dynamically enhanced while suppressing responses in low-information or noisy regions. S35. Input the weighted feature sequence output in step S34 into the subsequent Feed-Forward network of the Transformer encoder to form the complete TAIFI encoder layer output, which provides high-quality feature representation for the subsequent decoding stage and significantly improves the detection accuracy of small, low-contrast defects.

5. The method for detecting material shortages in flexible circuit board tape mounting as described in claim 1, characterized in that, Step S4 introduces a feature fusion module CAFF with a context-aware mechanism to improve the global expressive power of features while maintaining the integrity of multi-scale information. Specifically, it includes: S41. For the multi-scale feature maps from the model encoding stage, first increase the dimensionality through 1×1 convolution, then divide them into two paths: A direct jump connection is used to preserve shallow textures and edge information; The other path serves as the main trunk, passing through two deep modeling units in sequence. Each unit includes a RepNCSP module and a 3×3 convolution to enhance semantic expressiveness. S42. The features output by the main path and the jump connection path are concatenated in the channel dimension to form a multi-scale fused feature. S43. Apply contextual anchor attention to the fused features, including the following sub-steps: Global pooling is performed on the fused features to extract contextual anchor information; Use k×1 and 1×k depth separable convolutions to model the context response in the horizontal and vertical directions respectively; A spatial saliency map is generated by fusing horizontal and vertical responses through a gating mechanism. The spatial saliency map is multiplied element-wise with the fused features to enhance key features and suppress background interference. S44. Output the feature map processed by the CAFF module for defect localization and classification in the subsequent DETR decoding stage, so as to improve the detection accuracy of the model in complex backgrounds and small target scenes.

6. The PCBA micro-defect detection method based on improved DETR as described in claim 1, characterized in that, Step S5 introduces GSConv convolutions at all convolutional positions in the neck region of the RT-DETR model, specifically including: S51. Divide the input features along the channel dimension into a main path and a secondary path. The main path compresses the channels using standard convolution to extract key semantic information, while the secondary path uses depthwise separable convolution to preserve redundant feature representations and enhance feature diversity. S52. The outputs of the main path and the secondary path are concatenated in the channel dimension, and information interaction between different channels is achieved through channel shuffle operation, thereby improving the feature fusion effect. S53. Input the processed features into the detection head to significantly reduce the amount of computation and model parameters while maintaining high detection accuracy, so as to realize real-time defect detection suitable for edge devices.

7. The PCBA micro-defect detection method based on improved DETR as described in claim 1, characterized in that, Step S6 specifically includes: S61. Input the improved RT-DETR model from steps S2 to S5 into the training set image, train the model, optimize the network parameters through the Adam loss function, so that the model can accurately capture the small defect features in the PCBA image. S62. Input the trained improved RT-DETR model into the test set images and output the detection results of small defects, including the category and location information of the defects; S63. Evaluate and visualize the test results to enable rapid and accurate identification of minor defects in PCBAs, and make it applicable to real-time inspection tasks of edge devices.

Images