DETR-based printed circuit board defect detection method and device

Abstract

The invention relates to the technical field of printed circuit board detection, and discloses a DETR-based printed circuit board defect detection method and device, and the method comprises the steps: capturing and extracting a feature map from a to-be-detected image based on a preset backbone network model; adopting a preset RTDETR model, combining a multi-head self-attention mechanism to perform deep fusion on same-scale features in the feature map, and performing cross-scale feature fusion on different-scale features in the feature map to obtain a multi-scale feature sequence; screening out a multi-scale feature sequence containing a target object from the multi-scale feature sequence by using an IoU perception query selection mechanism to obtain a target feature sequence; and predicting the target feature sequence through an RTDETR model, and adjusting the boundary of a prediction frame by using a bounding box regression technology to obtain a bounding box of the printed circuit board defect and a corresponding confidence score. The method has the effect of realizing real-time rapid detection and stability of the defects of the printed circuit board.

Description

Technical Field

[0001] The present application relates to the technical field of printed circuit board defect detection, and in particular to a printed circuit board defect detection method and device based on DETR. Background Art

[0002] Printed circuit boards (PCBs) are an indispensable core component of electronic devices. The quality they carry is directly related to the reliability and durability of electronic products. With the rapid development of electronic technology, the market's quality standards for PCBs are becoming more and more stringent, which makes PCB defect detection particularly critical.

[0003] The rapid development of deep learning technology has injected new vitality into the field of PCB defect detection. Among them, the YOLO (You Only Look Once) series of algorithms have emerged in the field of real-time target detection with its excellent detection speed and accuracy. Since the advent of YOLOv5 and the continuous innovation of YOLOv10, each generation of algorithm iteration has made significant progress in detection speed, accuracy and model lightweight, greatly broadening the application scope of PCB defect detection. However, the YOLO series of algorithms still face challenges that cannot be ignored in terms of technical adaptability, scalability, real-time, stability, and generalization and robustness of small sample data. Algorithms based on the Transformer architecture, such as DETR (Detection with Transformers) and ViT (Vision Transformer), have also shown great potential in the research and application of defect detection. In particular, DETR, an innovative target detection model, relies on the Transformer mechanism, breaks the traditional R-CNN-based detection mode, and provides a new perspective for processing complex image features and analyzing the relationship between targets. However, the Transformer architecture algorithm has problems of delay and instability in the real-time inspection task of printed circuit boards, and has low accuracy in detecting various defect types of printed circuit boards.

[0004] With respect to the above-mentioned related technologies, the inventors have found that the existing printed circuit board defect detection methods have the problems of being unable to meet the real-time requirements of industrial production and having low detection accuracy. Summary of the invention

[0005] In order to achieve real-time rapid detection and stability of printed circuit board defects and improve the accuracy of detection of various defect types, the present application provides a printed circuit board defect detection method and device based on DETR.

[0006] In a first aspect, the present application provides a printed circuit board defect detection method based on DETR.

[0007] This application is achieved through the following technical solutions:

[0008] A printed circuit board defect detection method based on DETR comprises the following steps:

[0009] Acquire the image to be inspected of the printed circuit board;

[0010] Based on a preset backbone network model, capturing and extracting a feature map from the image to be detected;

[0011] The preset RT_DETR model is used in combination with the multi-head self-attention mechanism to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence;

[0012] Using an IoU-aware query selection mechanism, a multi-scale feature sequence containing a target object is selected from the multi-scale feature sequence to obtain a target feature sequence;

[0013] The target feature sequence is predicted by the RT_DETR model, the boundary of the prediction box is adjusted using the bounding box regression technology, and the confidence score of each prediction box is predicted to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

[0014] In a preferred example, the present application can be further configured as follows: the structure of the backbone network model includes:

[0015] An input layer, used for receiving the image to be detected;

[0016] A plurality of different convolutional layers are filled with zero values ​​to obtain a plurality of 3×3 convolutional layers, each of which is connected to the output end of the input layer to capture and extract local features of the image to be detected;

[0017] The batch normalization layer includes a plurality of normalization units, each of which is connected to the output end of each convolutional layer and is used to normalize each local feature.

[0018] The Sigmoid linear layer is connected to the output of each normalization unit and is used to model each normalization result using the Swish activation function to obtain a feature map;

[0019] The output layer is connected to the output end of the Sigmoid linear layer and is used to output the feature map.

[0020] In a preferred example, the present application can be further configured as follows: the structure of the RT_DETR model includes:

[0021] An intra-scale feature interaction module, configured to receive the feature map, capture local region connections between features of the same scale, aggregate the features of the same scale, and obtain fused features of the same scale;

[0022] A cross-scale feature fusion module is used to upsample and downsample the fused same-scale features, align the same-scale features, and perform splicing or addition operations on the aligned same-scale features to obtain fused multi-scale features;

[0023] The encoder is used to encode the fused multi-scale features to obtain the multi-scale feature sequence.

[0024] In a preferred example, the present application can be further configured as follows: the step of predicting the target feature sequence by the RT_DETR model, adjusting the boundary of the prediction box by using the bounding box regression technology, and predicting the confidence score of each prediction box, and obtaining the bounding box of the printed circuit board defect and the corresponding confidence score includes:

[0025] The RT_DETR model includes a decoder, which decodes the multi-scale feature sequence and outputs a set of prediction boxes of different scales, each of which includes a width, a height, and a center position;

[0026] The decoder obtains an offset of each of the prediction boxes, and determines an actual boundary of the prediction box based on the offset and in combination with the bounding box parameters.

[0027] In a preferred example, the present application can be further configured as follows: it also includes the following steps:

[0028] The RT_DETR model includes a prediction head, which outputs a classification matrix of (N, C+1) according to the feature map in the actual bounding box of each prediction box, wherein N is the number of prediction boxes, and C+1 is the total number of all defect categories of the printed circuit board and a background class; and uses the classification matrix in combination with a softmax function to predict the classification confidence score of each prediction box;

[0029] And, the prediction head predicts the offset of each of the prediction boxes and determines the border regression parameters of each prediction box.

[0030] In a preferred example, the present application may be further configured as follows: the step of using the IoU-aware query selection mechanism to filter out a multi-scale feature sequence containing a target object from the multi-scale feature sequence, and obtaining the target feature sequence includes:

[0031] generating an initial query from the multi-scale feature sequence to determine a candidate region of a target object;

[0032] Predict the corresponding IoU values ​​for the candidate regions respectively;

[0033] Based on a preset threshold, the candidate regions whose IoU values ​​are greater than the threshold are screened out to construct a multi-scale feature sequence containing the target object.

[0034] In a preferred example, the present application can be further configured as follows: the training steps of the RT_DETR model include:

[0035] Iteratively predicting the boundary of the prediction box of the target feature sequence, and gradually reducing the boundary of the prediction box;

[0036] The preset loss function is used to evaluate the difference between the boundary of the predicted box and the true annotation box, and the model parameters are updated using back propagation;

[0037] When the loss value of the loss function is less than a preset threshold, the model prediction is determined to be stable and the RT_DETR model is output.

[0038] In a preferred example, the present application may be further configured as follows: the loss function includes a border regression loss function for measuring the difference between the boundary of the predicted box and the true labeled box and a classification loss function for measuring the difference between the predicted defect category probability and the true defect category probability;

[0039] The expression of the loss function includes:

[0040] L=λ cls L cls +λ bbox L bbox

[0041] In the formula, L represents the total loss function, L cls represents the classification loss function, L bbox represents the bounding box regression loss function, λ cls and λ bbox Represent the weight coefficients of classification loss and border regression loss, respectively, which are used to adjust the relative weights of classification loss and border regression loss, λ cls +λ bbox =1.

[0042] In a second aspect, the present application provides a printed circuit board defect detection device based on DETR.

[0043] This application is achieved through the following technical solutions:

[0044] A printed circuit board defect detection device based on DETR, comprising:

[0045] An acquisition module, used for acquiring an image of a printed circuit board to be inspected;

[0046] A feature extraction module, used to capture and extract a feature map from the image to be detected based on a preset backbone network model;

[0047] A feature fusion module is used to use a preset RT_DETR model and a multi-head self-attention mechanism to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence;

[0048] A matching module, configured to filter out a multi-scale feature sequence containing a target object from the multi-scale feature sequence by using an IoU-aware query selection mechanism to obtain a target feature sequence;

[0049] The defect detection module is used to predict the target feature sequence through the RT_DETR model, adjust the boundary of the prediction box using the bounding box regression technology, and predict the confidence score of each prediction box to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

[0050] In a third aspect, the present application provides a computer device.

[0051] This application is achieved through the following technical solutions:

[0052] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned printed circuit board defect detection methods based on DETR when executing the computer program.

[0053] In a fourth aspect, the present application provides a computer-readable storage medium.

[0054] This application is achieved through the following technical solutions:

[0055] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of any one of the above-mentioned printed circuit board defect detection methods based on DETR.

[0056] In a fifth aspect, the present application provides a computer program product.

[0057] This application is achieved through the following technical solutions:

[0058] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the above-mentioned printed circuit board defect detection methods based on DETR.

[0059] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0060] The image to be inspected of the printed circuit board is obtained, and based on the preset backbone network model, the feature map is captured and extracted from the image to be inspected. The backbone network is used to conduct in-depth analysis of the input image, capture and extract the feature map, and provide rich visual data for subsequent inspection of the printed circuit board; the preset RT_DETR model is used, combined with the multi-head self-attention mechanism, to deeply fuse the same-scale features in the feature map to enhance the context information; the cross-scale feature fusion of the different-scale features in the feature map is performed to construct visual data of different scales, which provides strong feature support for downstream tasks such as target detection and semantic segmentation, and finally generates a coherent and reliable Then, the IoU-aware query selection mechanism is used to filter out the multi-scale feature sequence containing the target object from the multi-scale feature sequence to obtain the target feature sequence, so as to further narrow the regional scope of target detection and improve the detection efficiency; the target feature sequence is predicted by the RT_DETR model, the boundary of the prediction box is adjusted by the bounding box regression technology, and the confidence score of each prediction box is predicted to obtain the bounding box of the printed circuit board defect and the corresponding confidence score; the real-time rapid detection and stability of printed circuit board defects are realized, the real-time requirements of industrial production are met, and the accuracy of detection of diversified defect types is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the overall architecture of a model of a printed circuit board defect detection method based on DETR is provided as an exemplary embodiment of the present application.

[0062] Figure 2 A schematic structural diagram of a backbone network model of a printed circuit board defect detection method based on DETR is provided as another exemplary embodiment of the present application.

[0063] Figure 3 A structural block diagram of an intra-scale feature interaction module of a DETR-based printed circuit board defect detection method provided by another exemplary embodiment of the present application.

[0064] Figure 4 A structural block diagram of a cross-scale feature fusion module of a printed circuit board defect detection method based on DETR is provided as an exemplary embodiment of the present application.

[0065] Figure 5 A schematic diagram of the detection effect of a printed circuit board defect detection method based on DETR on an image to be detected of a printed circuit board is provided as an exemplary embodiment of the present application.

[0066] Figure 6A DETR-based printed circuit board defect detection method is provided as an exemplary embodiment of the present application, and a schematic diagram of the detection effect on an image to be inspected of a printed circuit board after an enhanced data training model is adopted. DETAILED DESCRIPTION

[0067] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0068] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0070] In the research of fabric defect detection technology, the research team of Xi'an Polytechnic University successfully applied the Swin-Unet model based on Transformer architecture in "A method for detecting defective areas of color texture fabrics based on self-attention". The model uses its deep learning ability of self-attention layer to accurately capture the details and contextual information of the image, greatly improving the ability to identify tiny defects in complex pattern fabrics.

[0071] In the paper "A Transformer-Based Wafer Defect Detection Method and System", researchers from Shanghai University of Electric Power proposed an innovative simplified detection process, which not only significantly improved the detection efficiency and accuracy, but also enhanced the versatility and adaptability of the detection system, bringing a breakthrough quality control solution to the semiconductor industry.

[0072] In the paper "Transformer-based Defect Detection Method and Electronic Device", the research team of East China Jiaotong University effectively captured the global contextual information of the image through an innovative encoder structure. This innovation significantly improved the detection accuracy of defects in small targets and complex backgrounds, and provided strong technical support for the quality inspection of electronic equipment.

[0073] The convergence of these research results illustrates the broad application prospects of Transformer-based defect detection technology and its important role in the wave of industrial automation and intelligence, but it also encounters a series of challenges.

[0074] In "A self-attention-based method for detecting defective regions in colored textured fabrics", researchers pioneered an unsupervised model that effectively detects fabric defects through image reconstruction technology without relying on defect samples or manual labeling. However, the model may encounter recognition difficulties when dealing with fabrics with different texture complexity and color changes, which still needs further research.

[0075] In the study of "A Transformer-based Wafer Defect Detection Method and System", an end-to-end Transformer structure method was proposed, which not only improved the detection accuracy but also simplified the detection process. However, the robustness and scalability of this method in actual industrial environments have not been fully discussed, and the model's generalization ability for different wafer defects and its recognition performance for complex or subtle defects need more empirical research to verify.

[0076] In the study of "Defect Detection Method and Electronic Device Based on Transformer", a defect detection method that can effectively capture global context information and reduce interference from irrelevant information is proposed by combining Transformer and convolutional neural network. Although this method has made progress in feature extraction and classification network design, how to further optimize the model to adapt to image inputs of different resolutions and sizes, and how to improve the accuracy and efficiency of the model in detecting diverse defect types are still topics that need to be explored in depth in future research.

[0077] To this end, in the defect detection of printed circuit boards, it is necessary to conduct in-depth exploration in enhancing the model's adaptability to complex textures and color changes, improving its robustness and real-time response capabilities in changing industrial environments, strengthening the model's generalized recognition capabilities for a variety of defect types, and optimizing the model structure to adapt to image inputs of different resolutions and sizes.

[0078] An embodiment of the present application provides a printed circuit board defect detection method based on DETR, and the main steps of the method are described as follows.

[0079] Acquire the image to be inspected of the printed circuit board;

[0080] Based on a preset backbone network model, capturing and extracting a feature map from the image to be detected;

[0081] The preset RT_DETR model is used in combination with the multi-head self-attention mechanism to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence;

[0082] Using an IoU-aware query selection mechanism, a multi-scale feature sequence containing a target object is selected from the multi-scale feature sequence to obtain a target feature sequence;

[0083] The target feature sequence is predicted by the RT_DETR model, the boundary of the prediction box is adjusted using the bounding box regression technology, and the confidence score of each prediction box is predicted to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

[0084] In one embodiment, the structure of the backbone network model includes:

[0085] An input layer, used for receiving the image to be detected;

[0086] A plurality of different convolutional layers are filled with zero values ​​to obtain a plurality of 3×3 convolutional layers, each of which is connected to the output end of the input layer to capture and extract local features of the image to be detected;

[0087] The batch normalization layer includes a plurality of normalization units, each of which is connected to the output end of each convolutional layer and is used to normalize each local feature.

[0088] The Sigmoid linear layer is connected to the output of each normalization unit and is used to model each normalization result using the Swish activation function to obtain a feature map;

[0089] The output layer is connected to the output end of the Sigmoid linear layer and is used to output the feature map.

[0090] In one embodiment, the structure of the RT_DETR model includes:

[0091] An intra-scale feature interaction module, configured to receive the feature map, capture local region connections between features of the same scale, aggregate the features of the same scale, and obtain fused features of the same scale;

[0092] A cross-scale feature fusion module is used to upsample and downsample the fused same-scale features, align the same-scale features, and perform splicing or addition operations on the aligned same-scale features to obtain fused multi-scale features;

[0093] The encoder is used to encode the fused multi-scale features to obtain the multi-scale feature sequence.

[0094] In one embodiment, the step of predicting the target feature sequence by the RT_DETR model, adjusting the boundary of the prediction box by using the bounding box regression technology, and predicting the confidence score of each prediction box to obtain the bounding box of the printed circuit board defect and the corresponding confidence score includes:

[0095] The RT_DETR model includes a decoder, which decodes the multi-scale feature sequence and outputs a set of prediction boxes of different scales, each of which includes a width, a height, and a center position;

[0096] The decoder obtains an offset of each of the prediction boxes, and determines an actual boundary of the prediction box based on the offset and in combination with the bounding box parameters.

[0097] In one embodiment, the following steps are also included:

[0098] The RT_DETR model includes a prediction head, which outputs a classification matrix of (N, C+1) according to the feature map in the actual bounding box of each prediction box, wherein N is the number of prediction boxes, and C+1 is the total number of all defect categories of the printed circuit board and a background class; and uses the classification matrix in combination with a softmax function to predict the classification confidence score of each prediction box;

[0099] And, the prediction head predicts an offset of each of the prediction boxes.

[0100] In one embodiment, the step of using the IoU-aware query selection mechanism to filter out a multi-scale feature sequence containing a target object from the multi-scale feature sequence to obtain a target feature sequence includes:

[0101] generating an initial query from the multi-scale feature sequence to determine a candidate region of a target object;

[0102] Predict the corresponding IoU values ​​for the candidate regions respectively;

[0103] Based on a preset threshold, the candidate regions whose IoU values ​​are greater than the threshold are screened out to construct a multi-scale feature sequence containing the target object.

[0104] In one embodiment, the training step of the RT_DETR model includes:

[0105] Iteratively predicting the boundary of the prediction box of the target feature sequence, and gradually reducing the boundary of the prediction box;

[0106] The preset loss function is used to evaluate the difference between the boundary of the predicted box and the true annotation box, and the model parameters are updated using back propagation;

[0107] When the loss value of the loss function is less than a preset threshold, the model prediction is determined to be stable and the RT_DETR model is output.

[0108] In one embodiment, the loss function includes a border regression loss function for measuring the difference between the boundary of the predicted box and the true annotation box and a classification loss function for measuring the difference between the predicted defect category probability and the true defect category probability;

[0109] The expression of the loss function includes:

[0110] L=λ cls L cls +λ bbox L bbox

[0111] In the formula, L represents the total loss function, L cls Represents the classification loss function, L bbox represents the bounding box regression loss function, λ cls and λ bbox Represent the weight coefficients of classification loss and border regression loss, respectively, which are used to adjust the relative weights of classification loss and border regression loss, λ cls +λ bbox =1.

[0112] The above embodiments are further described in detail below in conjunction with the accompanying drawings.

[0113] Reference Figure 1 , an image of the printed circuit board is captured by a camera, and the image to be inspected of the printed circuit board is collected to obtain an input image.

[0114] The backbone network model is used to capture the input image and extract feature maps of different scales. These feature maps are like multi-angle lenses that capture the visual world, providing rich visual data for printed circuit board defect detection.

[0115] The backbone network model includes:

[0116] An input layer, used for receiving the image to be detected;

[0117] A plurality of different convolutional layers are filled with zero values ​​to obtain a plurality of 3×3 convolutional layers, each of which is connected to the output end of the input layer to capture and extract local features of the image to be detected;

[0118] The batch normalization layer includes a plurality of normalization units, each of which is connected to the output end of each convolutional layer and is used to normalize each local feature.

[0119] The Sigmoid linear layer is connected to the output of each normalization unit and is used to model each normalization result using the Swish activation function to obtain a feature map;

[0120] The output layer is connected to the output end of the Sigmoid linear layer and is used to output the feature map.

[0121] Specifically, by improving the HGStem in the original Stem module in the backbone network model, the HGStem in the Stem module was originally designed as a single 3×3 convolutional layer. Instead of using a single 3x3 convolutional layer, this application indirectly uses n identical 3x3 convolutional layers. The specific process is to decompose the original 3×3 convolution into different convolutions (3×3, 1×1…), and then fill different convolutional layers with zero values ​​to turn it back into a 3×3 convolutional layer. Figure 2 As shown in the figure. Each convolutional layer is followed by a normalization unit, and several normalization units form a batch normalization layer. This design can provide more flexibility in the model training phase, because each convolutional layer can learn and adjust its parameters independently; in the inference phase, the parameters of several convolutional layers and batch normalization layers are fused to form an efficient, parameter-shared convolutional layer, thereby reducing the computational burden of the model and improving the inference speed. As a result, the backbone network model can not only maintain the effectiveness of the original module HGStem in feature extraction, but also reduce the amount of calculation through parameter fusion, showing advantages in application scenarios with high real-time requirements, so that the model can maintain high accuracy while optimizing the model's operating efficiency, providing new possibilities for the application of deep learning models in fields such as real-time target detection.

[0122] Next, the preset RT_DETR (Real-Time Detection Transformer) model is used, combined with the multi-head self-attention mechanism (Self-Attention) to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence. The RT_DETR model includes an intra-scale feature interaction module, a cross-scale feature fusion module and an encoder, wherein the intra-scale feature interaction module is used to receive the feature map, capture the local region connection between the same-scale features, aggregate the same-scale features, and obtain the fused same-scale features; the cross-scale feature fusion module is used to upsample and downsample the fused same-scale features, align the same-scale features, and perform splicing or addition operations on the aligned same-scale features to obtain fused multi-scale features; the encoder is used to encode the fused multi-scale features to obtain the multi-scale feature sequence.

[0123] Reference Figure 3Specifically, the intra-scale feature interaction module (AIFI) includes five parallel units (a), (b), (c), (d) and (e), and the feature maps are input into units a, b, c, d and e respectively. Unit a is used to perform connection operation and index scaling operation on the feature map in sequence. Unit b is used to perform square sum error operation, connection operation and index scaling operation on the feature map in sequence. Unit c is used to perform connection operation, mean square error operation and index scaling operation on the feature map in sequence. Unit d is used to perform square sum error operation, feature fusion operation and index scaling operation on the feature map in sequence. Unit e is used to perform intra-scale feature interaction operation (i.e., capturing the local region connection between the same-scale features) and cross-channel feature fusion operation (i.e., aggregating the same-scale features) on the feature map in sequence, and then perform index scaling operation on each same-scale feature obtained by fusion. The deep fusion step between the same-scale features is realized by the self-attention mechanism, which uses multi-head self-attention to capture the connection between local regions, and enhances the context information through feature aggregation to achieve deep fusion of the same-scale features.

[0124] Reference Figure 4 , the cross-scale feature fusion module (CCFM) includes a convolution layer, a feature fusion unit, and a flattening unit. The fused same-scale features are respectively input into the convolution layer for upsampling and downsampling to obtain aligned same-scale features; then they are input into the feature fusion unit for splicing operation to obtain fused multi-scale features; finally, they are input into the flattening unit for flattening operation to convert the multi-dimensional feature map into a one-dimensional vector for subsequent image processing. In the figure, C represents the convolution operation, and the number of 1×1 convolutions N is set according to the actual feature scale length value. The cross-scale feature fusion module cleverly combines features of different scales.

[0125] The efficient hybrid encoder of the RT_DETR model processes these feature maps in detail. The fused features flow through the encoder, where the self-attention mechanism enables the model to focus on important feature areas, and finally generates a coherent and informative multi-scale feature sequence, providing strong feature support for downstream tasks such as object detection and semantic segmentation.

[0126] Next, the IoU-aware query selection mechanism is used to filter out the multi-scale feature sequence containing the target object from the multi-scale feature sequence to obtain the target feature sequence. Specifically:

[0127] generating an initial query from the multi-scale feature sequence to determine a candidate region of a target object;

[0128] Predict the corresponding IoU values ​​for the candidate regions respectively;

[0129] Based on a preset threshold, the candidate regions whose IoU values ​​are greater than the threshold are screened out to construct a multi-scale feature sequence containing the target object.

[0130] According to the predicted IoU value (intersection over union) of the features in the multi-scale feature sequence and the preset target object IoU threshold, the relevance of the features is evaluated, and the multi-scale feature sequence that is most likely to contain the target object is selected. These multi-scale feature sequences are initialized as the starting point of the decoder to prepare for the subsequent object detection iterative optimization process.

[0131] Finally, the target feature sequence is predicted through the RT_DETR model, the boundary of the prediction box is adjusted using the bounding box regression technology, and the confidence score of each prediction box is predicted to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

[0132] The RT_DETR model also includes a decoder, which decodes the multi-scale feature sequence and outputs a set of prediction boxes of different scales, each of which includes a width, a height, and a center position;

[0133] The decoder obtains an offset of each of the prediction boxes, and determines an actual boundary of the prediction box based on the offset and in combination with the bounding box parameters.

[0134] In this embodiment, the RT_DETR model is an end-to-end object detection model based on Transformer, which can directly predict the category and bounding box of the target without using anchor boxes to predict the bounding box, and the prediction efficiency is faster. Among them, the prediction of the bounding box is based on the output of the encoder-decoder architecture of the model.

[0135] The decoder output of the RT_DETR model includes category prediction and bounding box prediction. The bounding box parameters include four values ​​(Δx, Δy, Δw, Δh), corresponding to the offset of the center point of the bounding box (Δx, Δy) and the logarithmic transformation of the width and height of the bounding box (Δw, Δh). With the help of the prediction head of the RT_DETR model, the actual bounding box coordinates are calculated by the following formula to obtain the actual boundary of the predicted box:

[0136] Determine the center point coordinates of the prediction box (cx, cy):

[0137] cx = Δx at the decoder output

[0138] cy = Δy output by the decoder

[0139] Determine the width w and height h of the prediction box:

[0140] w = exp(Δw output by the decoder)

[0141] h = exp(Δh output by the decoder)

[0142] Then, based on the offset of the prediction box and the bounding box parameters, the coordinates of the upper left corner and lower right corner of the prediction box are calculated to determine the actual prediction box coordinates. Assuming (cx, cy) is the coordinate of the center point of the prediction box, then the coordinate of the upper left corner of the prediction box (x min ,y min ) and the lower right corner coordinate (x max ,y max ) can be calculated as follows:

[0143]

[0144] For defect location detection of printed circuit boards (PCBs), the decoder output of the RT_DETR model can be used to directly predict the bounding box and determine the actual boundary of the prediction box. It can handle detection objects of different shapes and sizes, with higher detection efficiency and greater applicability. The RT_DETR model also includes a prediction head, which outputs a classification matrix of (N, C+1) based on the feature map within the actual bounding box of each prediction box, where N is the number of prediction boxes and C+1 is the total number of all defect categories of the printed circuit board and a background class; and uses the classification matrix in combination with the softmax function to predict the classification confidence score of each prediction box;

[0145] And, the prediction head predicts the offset of each of the prediction boxes and determines the border regression parameters of each prediction box.

[0146] In this embodiment, the classification confidence score of each prediction box refers to the confidence of the RT_DETR model that the prediction box belongs to a certain category, which can be calculated by the softmax function. Using the softmax function, the original logits (i.e., the original classification score output by the model) can be converted into a probability distribution. For printed circuit board (PCB) defect category detection, assuming that the logits output by the model is a vector of length C, where C is the total number of all defect categories of the printed circuit board, for the prediction box i, the probability that it belongs to category c is It can be calculated by the following formula:

[0147]

[0148] in, is the raw logit score of the predicted box i for category c.

[0149] Furthermore, the classification confidence score of the predicted box i is calculated, and its probability for the highest scoring category is taken as its confidence:

[0150]

[0151] The calculated score represents the classification confidence score of the RT_DETR model for predicting that box i belongs to a certain category c.

[0152] In practical applications, other factors, such as the confidence of the bounding box of the prediction box, can also be combined to obtain the final classification confidence score.

[0153] In order to achieve accurate labeling of the target object, the decoder then starts the prediction, uses the encoder-decoder architecture to extract and transform features, uses the bounding box regression technology to adjust the boundaries of the prediction box, performs the prediction, and determines the bounding box of the printed circuit board defect and the corresponding confidence score.

[0154] This continuous prediction and fine-tuning process ensures that the model can gradually approach the true location and category of the target object.

[0155] In one embodiment, the training steps of the RT_DETR model include:

[0156] Iteratively predicting the boundary of the prediction box of the target feature sequence, and gradually reducing the boundary of the prediction box;

[0157] The preset loss function is used to evaluate the difference between the boundary of the predicted box and the true annotation box, and the model parameters are updated using back propagation;

[0158] When the loss value of the loss function is less than a preset threshold, the model prediction is determined to be stable and the RT_DETR model is output.

[0159] The decoder then starts an iterative optimization process. In each iteration, the Transformer decoder captures the internal connections of the image through the self-attention mechanism in the object detection task to gradually refine the position and size of the bounding box. In each iteration, the model adjusts the predicted box through the bounding box regression technology and predicts the confidence score of each box, indicating the certainty of detecting a specific object. In addition, by designing a loss function to evaluate the difference between the boundary of the predicted box and the true annotated box, back-propagation updates the model parameters, and performs multi-scale predictions to ensure the prediction accuracy of features of different sizes.

[0160] In addition, the anchor box mechanism assists in the initialization of the bounding box. During the whole process, the model is continuously iteratively refined until the prediction is stable, achieving precise adjustment of the object bounding box and accurate prediction of the confidence score.

[0161] After a series of iterative optimizations, the model finally determines and outputs the bounding box and confidence score of each object, completing the precise object labeling process. This achievement marks the successful completion of the object detection task.

[0162] Reference Figure 5,The data detection results after the modified model are shown in Figure .,The detection accuracy of each defect type and bounding box is shown in Table 1.

[0163] Table 1

[0164] Defect Type Bounding box accuracy Missing_hole 0.999 Mouse_bite 0.937 Open_circuit 0.960 Short 0.988 Spur 0.976 Spurious_copper 0.970

[0165] In one embodiment, the loss function includes a border regression loss function for measuring the difference between the boundary of the predicted box and the true annotation box and a classification loss function for measuring the difference between the predicted defect category probability and the true defect category probability;

[0166] The expression of the loss function includes:

[0167] L=λ cls L cls +λ bbox L bbox

[0168] In the formula, L represents the total loss function, L cls represents the classification loss function, L bbox represents the bounding box regression loss function, λ cls and λ bbox Represent the weight coefficients of classification loss and border regression loss, respectively, which are used to adjust the relative weights of classification loss and border regression loss, λ cls +λ bbox =1.

[0169] By introducing classification loss and bounding box regression loss, the defect area can be located more accurately. Among them, the classification loss function adopts the Focal Loss function, including:

[0170] L focal = -α(1-p t ) γ log(p t )

[0171] The border regression loss function uses the GIoU (Generalized Intersection over Union) loss function, and the formula is as follows:

[0172]

[0173] In one embodiment, since PCB defects are usually rare events and class imbalance may occur, the popular focal loss is used to help the model better focus on defective samples that are difficult to classify.

[0174] Throughout the labeling process, the Transformer decoder demonstrates flexibility in real-time object detection tasks, and can also balance the model's inference speed and prediction accuracy by adjusting the number of decoder layers. According to the specific requirements of the task for speed and accuracy, the optimal number of layers can be determined through experimental tuning, and model pruning and knowledge distillation techniques can be used to reduce the number of layers while maintaining accuracy performance, as well as hardware optimization and parallel processing to improve efficiency.

[0175] In addition, the number of layers and feature reuse strategies are dynamically adjusted, and lightweight Transformer variants are developed to adapt to the needs of real-time tasks, ensuring efficient and accurate target detection and controlling the inference speed under limited computing resources, achieving a perfect balance between speed and accuracy.

[0176] The object labeling process of the RT-DETR model is not only automated, but also efficient and accurate. It can achieve rapid object recognition and positioning while ensuring detection accuracy. It is very suitable for application scenarios of real-time object detection, demonstrating its advancement and practicality in the field of modern computer vision.

[0177] The RT-DETR model uses the Transformer real-time target detection model and processes data through the self-attention mechanism, which significantly improves the speed, as shown in Table 2 below.

[0178] Table 2

[0179]

[0180] By using the RT_DETR model, the inference speed is increased by more than 100 times, which can greatly improve the efficiency of the model. At the same time, it can save resource consumption to a certain extent and speed up research and development.

[0181] In one embodiment, only a certain part of the model to be trained can be modified on the originally trained RT_DETR model, i.e., "frozen training". By loading the pre-trained weights and loading new modules, there is no need to retrain the entire model, which not only saves training time but also improves the quality of training. Through frozen training, when there are some pre-trained weights, the part of the network to which these pre-trained weights are applied is universal, such as a backbone network model, then the training of these weights can be frozen first, and more resources can be put into the network parameters of the latter part of the training, so that both time and resource utilization can be greatly improved. Then, after a period of training for the latter network parameters, these frozen parts are unfrozen, and then all are trained together.

[0182] In one embodiment, the training data of the model is enhanced by modifying the scale (zoom range) and degrees (angle size) of the training data, as well as flipud (probability of flipping up and down), cropping, to enhance the performance and generalization ability of the model, and then comparing it with normal data, as shown in Table 3 below.

[0183] Table 3

[0184]

[0185] By comparing the data of the backbone network model before and after the improvement, it can be clearly seen that the average accuracy of PCB board defect detection before the improvement is 95.6%, while the average accuracy of PCB board defect detection after the improvement is 97.2%. In the table, except for the decrease in the accuracy of "mouse_bite", the other five aspects have been improved, and even the average accuracy of Missing_hole has reached 0.999.

[0186] By comparison, it is found that the average accuracy of the data before and after enhancement does not fluctuate much, which proves that the model is less sensitive to the influence of interference elements in the data, and can maintain an excellent detection rate even when facing PCB boards in real life, as shown in Table 4.

[0187] Table 4

[0188]

[0189] In the table, degrees are all 10 (original data 0.0), flipud is all 0.1 (original data 0.0), and the others are default values. The mean values ​​of various defects are 0.969, 0.953, 0.963, and 0.972, respectively. It can be seen that their average values ​​do not fluctuate very much, which proves that this application can still maintain a high level of judgment in the face of complex environments.

[0190] The data detection results of the model obtained by augmented data training are as follows Figure 6 The detection accuracy of each defect type and bounding box is shown in Table 5.

[0191] Table 5

[0192]

[0193]

[0194] Different strategies are adopted. Strategy 1: Pretrian model: rtder-l.pt (the most original model), scale: 0.75;

[0195] Strategy 2: Pretrian model: Train7 / best.model (model without data enhancement): rtder-l.pt, scale: 0.25;

[0196] Strategy 3: Pretrian model: Train8 / best.model (model of strategy 2), scale: 0.25;

[0197] Strategy 4: Pretrian model: Train8 / best.model (model of strategy 2), scale: 0.9;

[0198] Strategy 5: Pretrian model: Trai10 / best.model (model of strategy 4), scale: 0.25;

[0199] Strategy 6: Pretrian model Train8 / best.model (model of strategy 2), scale: 0.5;

[0200] Strategy 7: Pretrian model: rtder-l.pt (the most original model), scale: 0.5;

[0201] The defect accuracy table is shown in Table 6 below.

[0202] Table 6

[0203]

[0204] It is not difficult to see from different strategies that the data enhancement effect is the best on the original data-enhanced model, especially strategy 5, which is optimized on the basis of strategies 2, 3, and 4. It not only inherits the advantages of the previous strategies, but also achieves a breakthrough in defect detection accuracy through fine-tuning.

[0205] In summary, a DETR-based printed circuit board defect detection method obtains the image to be detected of the printed circuit board, captures and extracts the feature map from the image to be detected based on the preset backbone network model, uses the backbone network to conduct in-depth analysis of the input image, captures and extracts the feature map, and can provide rich visual data for subsequent detection of the printed circuit board; adopts the preset RT_DETR model, combined with the multi-head self-attention mechanism, deeply fuses the same-scale features in the feature map to enhance the context information; cross-scale feature fusion is performed on the different-scale features in the feature map, and visual data of different scales is constructed, which provides powerful features for downstream tasks such as target detection and semantic segmentation. feature support, and finally generate a coherent and information-rich multi-scale feature sequence; then, use the IoU-aware query selection mechanism to filter out the multi-scale feature sequence containing the target object from the multi-scale feature sequence to obtain the target feature sequence, so as to further narrow the regional scope of target detection and improve the detection efficiency; through the RT_DETR model, the target feature sequence is predicted, the boundary of the prediction box is adjusted by using the bounding box regression technology, and the confidence score of each prediction box is predicted to obtain the bounding box of the printed circuit board defect and the corresponding confidence score; the real-time rapid detection and stability of printed circuit board defects are realized, the real-time requirements of industrial production are met, and the accuracy of detection of diversified defect types is improved.

[0206] The backbone network model of this application not only effectively reduces the number of model parameters, reduces the complexity of the model and memory usage, but also significantly improves the accuracy of the model in target detection tasks by optimizing parameter configuration and structural design, so that the model can process data faster while maintaining high performance, and is suitable for printed circuit board defect detection scenarios that require real-time feedback. In addition, the reduced number of model parameters requires fewer computing resources during training and inference, which helps to deploy the model on resource-constrained devices, expands the scope of application of the model, and provides a new efficient, lightweight and high-performance solution for the field of computer vision, which is expected to promote the development and application of related technologies.

[0207] Existing technologies often perform poorly when processing small sample data sets, while this application introduces innovative data augmentation techniques and small sample learning strategies, which significantly improve the generalization ability of the model in small sample conditions and ensure that the accuracy of the detection results is not limited by the number of samples.

[0208] The existing technology has delay and instability problems in real-time detection tasks, while this application adopts a lightweight network design and optimizes the reasoning process, which not only ensures high accuracy, but also realizes fast and stable printed circuit board defect detection, meeting the real-time requirements of industrial production.

[0209] Taking into account the changeable lighting and occlusion conditions in industrial environments, this application significantly improves the model's adaptability to complex environmental changes through adaptive feature fusion and attention mechanisms, ensuring high detection performance under various environmental conditions.

[0210] This application improves the detection performance of the model in complex scenarios through an innovative multi-task learning framework and feature extraction strategy, ensures comprehensive coverage of various defect types, and improves the accuracy and efficiency of the model in detecting diverse defect types in printed circuit boards. It provides a more efficient, accurate, and reliable technical method for the field of printed circuit board defect detection, and has important industrial application value and market prospects.

[0211] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0212] The embodiment of the present application also provides a printed circuit board defect detection device based on DETR, which corresponds one-to-one to a printed circuit board defect detection method based on DETR in the above embodiment. The printed circuit board defect detection device based on DETR includes:

[0213] An acquisition module, used for acquiring an image of a printed circuit board to be inspected;

[0214] A feature extraction module, used to capture and extract a feature map from the image to be detected based on a preset backbone network model;

[0215] A feature fusion module is used to use a preset RT_DETR model and a multi-head self-attention mechanism to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence;

[0216] A matching module, configured to filter out a multi-scale feature sequence containing a target object from the multi-scale feature sequence by using an IoU-aware query selection mechanism to obtain a target feature sequence;

[0217] The defect detection module is used to predict the target feature sequence through the RT_DETR model, adjust the boundary of the prediction box using the bounding box regression technology, and predict the confidence score of each prediction box to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

[0218] For the specific definition of a printed circuit board defect detection device based on DETR, please refer to the definition of a printed circuit board defect detection method based on DETR above, which will not be repeated here.

[0219] Each module in the above-mentioned DETR-based printed circuit board defect detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0220] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, any one of the above-mentioned printed circuit board defect detection methods based on DETR is implemented.

[0221] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, any one of the above-mentioned printed circuit board defect detection methods based on DETR is implemented.

[0222] In one embodiment, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, any one of the above-mentioned printed circuit board defect detection methods based on DETR is implemented.

[0223] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0224] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A printed circuit board defect detection method based on DETR, characterized in that: The following steps are included: Acquire the image to be inspected of the printed circuit board; Based on a preset backbone network model, capturing and extracting a feature map from the image to be detected; The preset RT_DETR model is used in combination with the multi-head self-attention mechanism to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence; Using an IoU-aware query selection mechanism, a multi-scale feature sequence containing a target object is selected from the multi-scale feature sequence to obtain a target feature sequence; The target feature sequence is predicted by the RT_DETR model, the boundary of the prediction box is adjusted using the bounding box regression technology, and the confidence score of each prediction box is predicted to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

2. The printed circuit board defect detection method based on DETR according to claim 1, characterized in that: The structure of the backbone network model includes: An input layer, used for receiving the image to be detected; A plurality of different convolutional layers are filled with zero values ​​to obtain a plurality of 3×3 convolutional layers, each of which is connected to the output end of the input layer to capture and extract local features of the image to be detected; The batch normalization layer includes a number of normalization units, each of which is connected to the output end of each convolutional layer and is used to normalize each local feature. The Sigmoid linear layer is connected to the output of each normalization unit and is used to model each normalization result using the Swish activation function to obtain a feature map; The output layer is connected to the output end of the Sigmoid linear layer and is used to output the feature map.

3. The printed circuit board defect detection method based on DETR according to claim 1, characterized in that: The structure of the RT_DETR model includes: An intra-scale feature interaction module, configured to receive the feature map, capture local region connections between features of the same scale, aggregate the features of the same scale, and obtain fused features of the same scale; A cross-scale feature fusion module is used to upsample and downsample the fused same-scale features, align the same-scale features, and perform splicing or addition operations on the aligned same-scale features to obtain fused multi-scale features; The encoder is used to encode the fused multi-scale features to obtain the multi-scale feature sequence.

4. The printed circuit board defect detection method based on DETR according to claim 1, characterized in that: The step of predicting the target feature sequence by the RT_DETR model, adjusting the boundary of the prediction box by using the bounding box regression technology, and predicting the confidence score of each prediction box to obtain the bounding box of the printed circuit board defect and the corresponding confidence score includes: The RT_DETR model includes a decoder, which decodes the multi-scale feature sequence and outputs a set of prediction boxes of different scales, each of which includes a width, a height, and a center position; The decoder obtains an offset of each of the prediction boxes, and determines an actual boundary of the prediction box based on the offset and in combination with the bounding box parameters.

5. The printed circuit board defect detection method based on DETR according to claim 4, characterized in that: The following steps are also included: The RT_DETR model includes a prediction head, which outputs a classification matrix of (N, C+1) according to the feature map in the actual bounding box of each prediction box, wherein N is the number of prediction boxes, and C+1 is the total number of all defect categories of the printed circuit board and a background class; and uses the classification matrix in combination with a softmax function to predict the classification confidence score of each prediction box; And, the prediction head predicts an offset of each of the prediction boxes.

6. The printed circuit board defect detection method based on DETR according to claim 1, characterized in that: The step of using the IoU-aware query selection mechanism to filter out a multi-scale feature sequence containing a target object from the multi-scale feature sequence to obtain a target feature sequence includes: generating an initial query from the multi-scale feature sequence to determine a candidate region of a target object; Predict the corresponding IoU values ​​for the candidate regions respectively; Based on a preset threshold, the candidate regions whose IoU values ​​are greater than the threshold are screened out to construct a multi-scale feature sequence containing the target object.

7. The printed circuit board defect detection method based on DETR according to any one of claims 1 to 6, characterized in that: The training steps of the RT_DETR model include: Iteratively predicting the boundary of the prediction box of the target feature sequence, and gradually reducing the boundary of the prediction box; The preset loss function is used to evaluate the difference between the boundary of the predicted box and the true annotation box, and the model parameters are updated using back propagation; When the loss value of the loss function is less than a preset threshold, the model prediction is determined to be stable and the RT_DETR model is output.

8. The printed circuit board defect detection method based on DETR according to claim 7, characterized in that: The loss function includes a border regression loss function for measuring the difference between the boundary of the predicted box and the true annotation box and a classification loss function for measuring the difference between the predicted defect category probability and the true defect category probability; The expression of the loss function includes: L=λ cls L cls +λ bbox L bbox In the formula, L represents the total loss function, L cls Represents the classification loss function, L bbox represents the bounding box regression loss function, λ cls and λ bbox Represent the weight coefficients of classification loss and border regression loss, respectively, which are used to adjust the relative weights of classification loss and border regression loss, λ cls +λ bbox =1.

9. A printed circuit board defect detection device based on DETR, characterized in that: include, An acquisition module, used for acquiring an image of a printed circuit board to be inspected; A feature extraction module, used to capture and extract a feature map from the image to be detected based on a preset backbone network model; A feature fusion module is used to use a preset RT_DETR model and a multi-head self-attention mechanism to deeply fuse the same-scale features in the feature map, and to perform cross-scale feature fusion on the different-scale features in the feature map to obtain a multi-scale feature sequence; A matching module, configured to filter out a multi-scale feature sequence containing a target object from the multi-scale feature sequence by using an IoU-aware query selection mechanism to obtain a target feature sequence; The defect detection module is used to predict the target feature sequence through the RT_DETR model, adjust the boundary of the prediction box using the bounding box regression technology, and predict the confidence score of each prediction box to obtain the bounding box of the printed circuit board defect and the corresponding confidence score.

10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 8 when being executed by a processor.

Images