Multi-scale fusion complex precision part machining quality defect intelligent detection method

Abstract

The invention relates to the field of computer vision and defect detection, in particular to a multi-scale fused complex precision part machining quality defect intelligent detection method, which comprises the following steps: inputting a to-be-detected PCB image into a trained defect detection model, and outputting corresponding PCB defect prediction information; the processing steps comprise: performing feature extraction on the PCB image through the backbone network; performing bidirectional feature fusion from top to bottom and from bottom to top on the initial feature maps of the multiple scales through a bidirectional feature fusion module; after the fused feature maps of all the scales are decoded, PCB defect prediction information is obtained through a detection head; according to the invention, the bidirectional feature fusion module (BiFM) carries out bidirectional feature fusion from top to bottom and from bottom to top on initial feature maps of multiple scales, and a multi-scale C2F module (MS-C2F) and an efficient up-sampling module (EUSB) are introduced, so that feature expression is enriched, and meanwhile, the model calculation amount is effectively reduced, thereby improving the precision and real-time performance of small-size defect detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and defect detection, specifically to an intelligent detection method for quality defects in the machining of complex and precision parts using multi-scale fusion. Background Technology

[0002] Printed circuit boards (PCBs) are core components of electronic products. Surface defects (such as missing holes, rodent bites, open circuits, short circuits, burrs, and copper straying) reduce product yield and affect the performance and lifespan of subsequent electronic products. Therefore, automated detection of PCB surface defects is of great significance. Currently, PCB defect detection mainly relies on automated optical inspection (AOI) or machine vision-based algorithms. Traditional target detection algorithms (such as the YOLO series based on convolutional neural networks and Faster R-CNN) have been applied in the field of defect detection, but these methods typically require post-processing such as non-maximum suppression (NMS) to remove redundant detection boxes and rely on pre-defined anchor boxes, which still pose challenges for the detection of small-sized defects.

[0003] The Detection Transformer (DETR) model, proposed in recent years, introduces the Transformer into object detection, realizing an end-to-end detection framework that eliminates the need for Non-Maximum Sampling (NMS) and pre-defined anchor boxes, greatly simplifying the detection process. While DETR demonstrates excellent detection accuracy, it also suffers from slow training convergence and poor performance with small objects. To address these issues, numerous improved algorithms have been proposed, such as Deformable DETR, Conditional-DETR, and SMCA-DETR, which accelerate convergence through improved attention mechanisms, and CF-DETR, which leverages local features to enhance small object detection. Notably, Baidu's Real-Time Detection Transformer (RT-DETR), as the first real-time end-to-end Transformer detector, effectively handles multi-scale features through an efficient hybrid encoder, achieving superior speed and accuracy compared to similarly scaled YOLO models on the COCO dataset. RT-DETR demonstrates the potential of the DETR architecture in real-time detection.

[0004] However, directly applying the DETR model to PCB defect detection still faces challenges. PCB defects are typically small in size and subtle in appearance, and the compact layout and complex background of PCBs easily lead to missed or false detections. Existing real-time detection models have achieved a balance between accuracy and speed, but there is still room for improvement in the fine-grained identification of minute PCB defects. For example, the original RT-DETR model exhibits missed and false detections when detecting certain PCB defect categories, such as burrs, open circuits, and rodent bites, which are not sensitive enough. Therefore, an improved detection method is urgently needed to enhance the extraction and fusion of multi-scale, especially small-scale, defect features while maintaining real-time performance, thereby improving detection accuracy. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide an intelligent detection method for quality defects in the machining of complex precision parts using multi-scale fusion. This method uses a bidirectional feature fusion module (BiFM) to perform bidirectional feature fusion of initial feature maps at multiple scales from top to bottom and from bottom to top. Furthermore, a multi-scale C2F module (MS-C2F) and an efficient upsampling module (EUSB) are introduced into the bidirectional feature fusion module, which effectively reduces the computational load of the model while enriching the feature representation, thereby improving the accuracy and real-time performance of small-size defect detection.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A multi-scale fusion-based intelligent detection method for quality defects in the machining of complex precision parts includes:

[0008] S1: Obtain the PCB image to be inspected;

[0009] S2: Input the PCB image to be detected into the trained defect detection model, and output the corresponding PCB defect prediction information; the defect prediction information includes the bounding box coordinates and category label of the PCB defect;

[0010] The processing steps of the defect detection model include:

[0011] S201: Feature extraction of PCB images is performed through the backbone network to obtain initial feature maps at multiple scales;

[0012] S202: A bidirectional feature pyramid is constructed from top to bottom and bottom to top to iteratively fuse initial feature maps at multiple scales, thereby obtaining a fused feature map that has both details and semantics.

[0013] In the bidirectional feature fusion module: the multi-scale C2F module performs depthwise separable convolution and channel shuffling at multiple scales to extract target features at multiple scales and resolutions; the efficient upsampling module upsamples the feature map step by step, upsampling the feature map at the current scale to the same resolution as the feature map at the next scale and then fusing them.

[0014] S203: After decoding the fused feature maps at various scales by the detection head, the bounding box coordinates and category labels of PCB defects are obtained, which are the PCB defect prediction information.

[0015] S3: Use the PCB defect prediction information output by the defect detection model as the intelligent detection result of the processing quality defects of the PCB image to be detected.

[0016] Preferably, in step S201, a pre-trained ResNet-18 is used as the backbone network to extract features from the PCB image.

[0017] Preferably, initial feature maps at four scales are obtained through the backbone network.

[0018] Preferably, in step S202, the bidirectional feature fusion module includes a cascaded encoder, a first feature fusion module, a first multi-scale C2F module, a first high-efficiency upsampling module, a second feature fusion module, a second multi-scale C2F module, a second high-efficiency upsampling module, a third feature fusion module, a third multi-scale C2F module, a fourth feature fusion module, a fourth multi-scale C2F module, a fifth feature fusion module, a fifth multi-scale C2F module, a sixth feature fusion module, and a sixth feature fusion module.

[0019] A convolutional combination module 1 is connected between the encoder and the first feature fusion module; the convolutional combination module 1 includes a cascaded 1×1 convolutional layer, a batch normalization layer and a SiLU activation function;

[0020] A second convolutional combination module is connected between the fourth multi-scale C2F module and the fifth feature fusion module, as well as between the fifth multi-scale C2F module and the sixth feature fusion module. The second convolutional combination module includes cascaded 3×3 convolutional layers, batch normalization layers, and SiLU activation functions.

[0021] Specifically, the initial feature map at the fourth scale (the smallest scale) is processed by convolutional combination module one, and the result is input to the encoder; the initial feature map at the third scale is processed by convolutional combination module one and convolutional combination module two, and the result is input to the first feature fusion module; the initial feature map at the second scale is processed by convolutional combination module one and convolutional combination module two, and the result is concatenated with the initial feature map at the third scale processed by convolutional combination module one and convolutional combination module two, and then input to the second feature fusion module; the initial feature map at the first scale is processed by convolutional combination module two, and the result is concatenated with the initial feature map at the second scale processed by convolutional combination module one and convolutional combination module two, and then input to the third feature fusion module.

[0022] The output of the second multi-scale C2F module, after being processed by the second convolutional combination module, is added element-wise to the output of the first multi-scale C2F module and the result is input to the sixth feature fusion module.

[0023] The outputs of the second and third multi-scale C2F modules, after being processed by the second convolutional combination module, are added element-wise to the output of the first efficient upsampling module and then input into the fifth feature fusion module.

[0024] The output of the second high-efficiency upsampling module is input to the fourth feature fusion module;

[0025] Finally, the fused feature maps of the three scales are output through the fourth, fifth, and sixth multi-scale C2F modules.

[0026] Preferably, the encoder is an intra-scale feature interaction encoder based on an attention mechanism.

[0027] Preferably, the feature fusion module is a fast normalization fusion module.

[0028] Preferably, the multi-scale C2F module includes cascaded 1×1 convolutional layers, batch normalization layers, ReLU6 activation function layers, segmentation modules, several multi-scale deep convolutional modules, splicing modules, channel rearrangement operation modules, 1×1 pointwise convolutional layers, and batch normalization layers.

[0029] The segmentation module divides the feature map into two feature sub-maps: one feature sub-map is input into multiple cascaded multi-scale deep convolutional modules for processing and the output of each multi-scale deep convolutional module is input into the concatenation module, where the output of all multi-scale deep convolutional modules is concatenated; the other feature sub-map is residually concatenated with the output of the next batch normalization layer.

[0030] The formula for the multi-scale C2F module is expressed as follows:

[0031]

[0032] In the formula: MS-C2F represents a multi-scale C2F module; PWC1 represents a pointwise convolutional layer; BN represents a batch normalization layer; R6 represents a ReLU6 activation function layer; N represents N multi-scale depth convolutional modules; CS represents channel rearrangement operation; MSDC represents a multi-scale depth convolutional module.

[0033] Preferably, the multi-scale deep convolution module includes three branches, each branch including a cascaded deep convolutional layer of the corresponding kernel size, a batch normalization layer, and a ReLU6 activation function layer; the outputs of the three branches are added element by element and then input to the channel rearrangement operation module to obtain the output of the multi-scale deep convolution module;

[0034] The formula for a multi-scale depthwise convolution module is expressed as follows:

[0035]

[0036] Each multi-scale deep convolutional module has a residual connection, i.e., x = x + MSDC(x);

[0037] In the formula: MSDC represents a multi-scale depthwise convolutional module; x represents the input feature map; DWCB ks The depthwise convolutional layers with different kernel sizes are followed by batch normalization layers and ReLU6 activation function layers. ks represents the kernel size, and KS indicates that there are convolutional kernels of different sizes.

[0038] Preferably, the efficient upsampling module includes a cascaded upsampling module, a 3×3 depth convolutional layer, a channel rearrangement operation module, and a 1×1 convolutional layer;

[0039] The formula for the high-efficiency upsampling module is expressed as:

[0040] EUSB(x) = C 1x1 (CS(DWC 3x3 (UP(x))));

[0041] In the formula: EUSB represents the high-efficiency upsampling module; UP represents the ordinary upsampling layer (x2); DWC 3x3 This indicates a depthwise convolutional layer with a kernel size of 3×3; C 1x1 This represents a 1×1 convolutional layer.

[0042] Preferably, in step S2, when training the defect detection model, the normalized Wasserstein distance loss is used as the loss function for bounding box regression.

[0043] The predicted bounding boxes and the ground truth bounding boxes are modeled as two-dimensional Gaussian distributions, and the Wasserstein distance between them is calculated to measure the regression error of the bounding boxes. The formula is expressed as follows:

[0044]

[0045] Where: N a and N b Let c and y represent the Gaussian distributions modeled by the ground truth bounding boxes and the predicted bounding boxes, respectively. The ground truth bounding boxes and the predicted bounding boxes are denoted as (cx... a ,cy a ,w a ,h a ) and (cx b ,cy b ,w b ,h b ), cx a cy a w represents the x and y coordinates of the center point of bounding box a. a and h b Let C represent the width and height of the center point of bounding box a; C is a constant closely related to the training dataset.

[0046] Compared with existing technologies, the intelligent detection method for quality defects in the machining of complex precision parts using multi-scale fusion in this invention has the following advantages:

[0047] This invention utilizes a bidirectional feature fusion module (BiFM) to perform top-down and bottom-up bidirectional feature fusion on initial feature maps at multiple scales, resulting in fused feature maps at multiple scales. The BiFM module applies an attention mechanism to the smallest-scale feature map, and then uses a feature fusion module (FN-fusion) to perform weighted feature fusion with larger-scale features, effectively preserving both low-level details and high-level semantic information. This significantly enhances the defect detection model's ability to perceive minute defects, thereby improving the accuracy of small-sized defect detection. Furthermore, the introduction of the BiFM module significantly improves the defect detection model's attention to small defects on PCB boards, and experimental results demonstrate a significant improvement in the average accuracy for small target defects.

[0048] This invention introduces a multi-scale C2F module (MS-C2F) and an efficient upsampling module (EUSB) into the bidirectional feature fusion module. This enriches feature representation while effectively reducing model computation, thereby improving the real-time performance of small-sized defect detection. The multi-scale C2F module performs depthwise separable convolution and channel shuffling at multiple scales to extract multi-scale, multi-resolution target features. By employing depthwise separable convolution and channel shuffling strategies in feature enhancement, it better obtains multi-scale information about the target. The efficient upsampling module performs stepwise upsampling of the feature map, upsampling the current scale feature map to the same resolution as the next scale feature map and fusing them to obtain enhanced high-resolution features. Using depthwise convolution instead of traditional convolution upsampling, the defect detection model reduces redundant computation and parameter count while maintaining detection accuracy, thus improving detection efficiency. Experiments show that after introducing the multi-scale C2F module and the efficient upsampling module, the model's characterization of target boundaries is more refined, and the model's inference speed remains essentially unchanged while improving localization accuracy. Attached Figure Description

[0049] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0050] Figure 1 This is a network structure diagram of the defect detection model.

[0051] Figure 2 This is a network structure diagram of the Bidirectional Feature Fusion Module (BiFM).

[0052] Figure 3 This is a network structure diagram of the multi-scale C2F module (MS-C2F).

[0053] Figure 4 This is a network structure diagram of a multi-scale deep convolutional module (MSDC).

[0054] Figure 5 This is a network structure diagram of the high-efficiency upsampling module (EUSB).

[0055] Figure 6 This document contains data and schematic diagrams of PCB surface defects.

[0056] Figure 7 A comparison of initial RT-DETR and EMSF-DETR detection. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and illustrated in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0058] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not mean that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0059] The following detailed explanation illustrates the specific implementation methods:

[0060] Example:

[0061] This embodiment discloses an intelligent detection method for machining quality defects of complex precision parts using multi-scale fusion.

[0062] like Figure 1 As shown, a multi-scale fusion intelligent detection method for machining quality defects in complex and precision parts includes:

[0063] S1: Obtain the PCB image to be inspected;

[0064] S2: Input the PCB image to be detected into the trained defect detection model, and output the corresponding PCB defect prediction information; the defect prediction information includes the bounding box coordinates and category label of the PCB defect;

[0065] The processing steps of the defect detection model include:

[0066] S201: Feature extraction of PCB images is performed through the backbone network to obtain initial feature maps at multiple scales;

[0067] S202: A bidirectional feature pyramid is constructed from top to bottom and bottom to top through a bidirectional feature fusion module (BiFM) to iteratively fuse initial feature maps at multiple scales to obtain a fused feature map that has both details and semantics.

[0068] In the bidirectional feature fusion module: the multi-scale C2F module (MS-C2F) performs depthwise separable convolution and channel shuffling at multiple scales to extract multi-scale, multi-resolution target features; the efficient upsampling module (EUSB) upsamples the feature map step by step, upsampling the feature map at the current scale to the same resolution as the feature map at the next scale and then fusing them to obtain enhanced high-resolution features;

[0069] Through the synergistic effect of the above modules, the fused features output by the model include both low-level defect details and high-level semantic context, providing sufficient basis for accurate defect identification.

[0070] S203: After decoding the fused feature maps at various scales by the detection head, the bounding box coordinates and category labels of PCB defects are obtained, which are the PCB defect prediction information.

[0071] In this embodiment, the fused features are mapped to a series of detection targets through the Transformer detection head (i.e., encoder-decoder structure). The bounding box coordinates and category label of each defect target are directly regressed through the interaction between the query and features in the decoder.

[0072] S3: Use the PCB defect prediction information output by the defect detection model as the intelligent detection result of the processing quality defects of the PCB image to be detected.

[0073] This invention utilizes a bidirectional feature fusion module (BiFM) to perform top-down and bottom-up bidirectional feature fusion on initial feature maps at multiple scales, resulting in fused feature maps at multiple scales. The BiFM module applies an attention mechanism to the smallest-scale feature map, and then uses a feature fusion module (FN-fusion) to perform weighted feature fusion with larger-scale features, effectively preserving both low-level details and high-level semantic information. This significantly enhances the defect detection model's ability to perceive minute defects, thereby improving the accuracy of small-sized defect detection. Furthermore, the introduction of the BiFM module significantly improves the defect detection model's attention to small defects on PCB boards, and experimental results demonstrate a significant improvement in the average accuracy for small target defects.

[0074] This invention introduces a multi-scale C2F module (MS-C2F) and an efficient upsampling module (EUSB) into the bidirectional feature fusion module. This enriches feature representation while effectively reducing model computation, thereby improving the real-time performance of small-sized defect detection. The multi-scale C2F module performs depthwise separable convolution and channel shuffling at multiple scales to extract multi-scale, multi-resolution target features. By employing depthwise separable convolution and channel shuffling strategies in feature enhancement, it better obtains multi-scale information about the target. The efficient upsampling module performs stepwise upsampling of the feature map, upsampling the current scale feature map to the same resolution as the next scale feature map and fusing them to obtain enhanced high-resolution features. Using depthwise convolution instead of traditional convolution upsampling, the defect detection model reduces redundant computation and parameter count while maintaining detection accuracy, thus improving detection efficiency. Experiments show that after introducing the multi-scale C2F module and the efficient upsampling module, the model's characterization of target boundaries is more refined, and the model's inference speed remains essentially unchanged while improving localization accuracy.

[0075] To better illustrate the technical solution of the present invention, this embodiment is described in the following parts.

[0076] I. Backbone Network

[0077] In this embodiment, a pre-trained ResNet-18 is used as the backbone network to extract features from PCB images, and the feature maps output by different layers of ResNet-18 are used as initial feature maps at multiple scales.

[0078] Initial feature maps at four scales are obtained through the backbone network.

[0079] II. Bidirectional Feature Fusion Module (BiFM)

[0080] The bidirectional feature fusion module introduces intermediate-scale features from the backbone of the convolutional neural network and incorporates them into the initial feature pyramid to participate in the fusion of high-level and low-level features. At the same time, it adopts a top-down and bottom-up bidirectional iterative fusion mechanism to perform feature fusion on features at each scale and uses a weighted feature fusion strategy to integrate feature information from different levels. The resulting multi-scale feature map is output, which contains both high-level semantic information and low-level detail information to enhance the representation ability of small defects.

[0081] like Figure 2 As shown, the bidirectional feature fusion module includes a cascaded encoder, a first feature fusion module, a first multi-scale C2F module, a first high-efficiency upsampling module, a second feature fusion module, a second multi-scale C2F module, a second high-efficiency upsampling module, a third feature fusion module, a third multi-scale C2F module, a fourth feature fusion module, a fourth multi-scale C2F module, a fifth feature fusion module, a fifth multi-scale C2F module, a sixth feature fusion module, and a sixth feature fusion module.

[0082] The encoder is an Attention-based Intra-scale Feature Interaction (AIFI) encoder. The feature fusion module is a Fast Normalized Fusion (FN-fusion) module. Both AIFI and the feature fusion module are existing modules.

[0083] A convolutional combination module 1 is connected between the encoder and the first feature fusion module; the convolutional combination module 1 includes a cascaded 1×1 convolutional layer, a batch normalization layer and a SiLU activation function;

[0084] There is a convolutional combination module 2 between the fourth multi-scale C2F module and the fifth feature fusion module, and between the fifth multi-scale C2F module and the sixth feature fusion module; the convolutional combination module 2 includes cascaded 3×3 convolutional layers, batch normalization layers and SiLU activation functions;

[0085] Specifically, the initial feature map at the fourth scale (the smallest scale) is processed by convolutional combination module one, and the result is input to the encoder; the initial feature map at the third scale is processed by convolutional combination module one and convolutional combination module two, and the result is input to the first feature fusion module; the initial feature map at the second scale is processed by convolutional combination module one and convolutional combination module two, and the result is concatenated with the initial feature map at the third scale processed by convolutional combination module one and convolutional combination module two, and then input to the second feature fusion module; the initial feature map at the first scale is processed by convolutional combination module two, and the result is concatenated with the initial feature map at the second scale processed by convolutional combination module one and convolutional combination module two, and then input to the third feature fusion module.

[0086] The output of the second multi-scale C2F module, after being processed by the second convolutional combination module, is added element-wise to the output of the first multi-scale C2F module, and the result is input to the sixth feature fusion module.

[0087] The outputs of the second and third multi-scale C2F modules, after being processed by the second convolutional combination module, are added element-wise to the output of the first efficient upsampling module and then input into the fifth feature fusion module.

[0088] The output of the second high-efficiency upsampling module is input to the fourth feature fusion module;

[0089] Finally, the fused feature maps of the three scales are output through the fourth, fifth, and sixth multi-scale C2F modules.

[0090] III. Multi-scale C2F module (MS-C2F)

[0091] The multi-scale C2F module further processes the fused features. The MSDC submodule provides multi-scale convolution operators to extract features from different receptive fields and blends feature channels through a channel shuffle mechanism to enhance the expressive power of the features.

[0092] like Figure 3 As shown, the multi-scale C2F module includes cascaded 1×1 convolutional layers, batch normalization layers, ReLU6 activation function layers, segmentation modules, several multi-scale depth convolutional modules, splicing modules, channel rearrangement operation modules, 1×1 pointwise convolutional layers, and batch normalization layers.

[0093] The segmentation module divides the feature map into two feature sub-maps: one feature sub-map is input into multiple cascaded multi-scale deep convolutional modules for processing and the output of each multi-scale deep convolutional module is input into the concatenation module, where the output of all multi-scale deep convolutional modules is concatenated; the other feature sub-map is residually concatenated with the output of the next batch normalization layer.

[0094] The formula for the multi-scale C2F module is expressed as follows:

[0095]

[0096] In the formula: MS-C2F represents a multi-scale C2F module; PWC1 represents a pointwise convolutional layer (1×1); BN represents a batch normalization layer; R6 represents a RELU6 activation function layer; N represents N multi-scale deep convolutional modules; CS represents a channel shuffle operation; MSDC represents a multi-scale deep convolutional module.

[0097] IV. Multi-Scale Depthic Convolutional Module (MSDC)

[0098] The MSDC submodule provides multi-scale convolution operators to extract features from different receptive fields during this process, and blends feature channels through a channel shuffle mechanism to enhance the expressive power of features.

[0099] like Figure 4 As shown, the multi-scale deep convolution module includes three branches. Each branch includes a cascaded deep convolutional layer of the corresponding kernel size, a batch normalization layer, and a ReLU6 activation function layer. The outputs of the three branches are added element by element and then input to the channel rearrangement operation module to obtain the output of the multi-scale deep convolution module.

[0100] The formula for a multi-scale depthwise convolution module is expressed as follows:

[0101]

[0102] Each multi-scale depthwise convolutional module has a residual connection, i.e., x = x + MSDC(x);

[0103] In the formula: MSDC represents a multi-scale depthwise convolutional module; x represents the input feature map; DWCB ks The depthwise convolutional layers with different kernel sizes are followed by batch normalization layers and ReLU6 activation function layers. ks represents the kernel size, and KS indicates that there are convolutional kernels of different sizes.

[0104] V. High-Efficiency Upsampling Module (EUSB)

[0105] The efficient upsampling module upsamples the feature map layer by layer. After each stage of EUSB, the upsampled features are fused with the corresponding jumper features from the backbone network, gradually building a high-resolution feature map for fine detection. Because the EUSB module uses depthwise convolution instead of ordinary convolution for upsampling, the computational cost is lower, enabling the entire model to run efficiently on edge devices.

[0106] like Figure 5 As shown, the high-efficiency upsampling module includes a cascaded upsampling module, a 3×3 depthwise convolutional layer, a channel rearrangement operation module, and a 1×1 convolutional layer;

[0107] The formula for the high-efficiency upsampling module is expressed as:

[0108] EUSB(x) = C 1x1 (CS(DWC 3x3 (UP(x))));

[0109] In the formula: EUSB represents the high-efficiency upsampling module; UP represents the ordinary upsampling layer (x2); DWC 3x3 This indicates a depthwise convolutional layer with a kernel size of 3×3; C 1x1 This represents a 1×1 convolutional layer.

[0110] VI. Loss Function

[0111] When training the defect detection model, the normalized Wasserstein distance (NWD) loss is used as the loss function for bounding box regression;

[0112] The predicted bounding boxes and the ground truth bounding boxes are modeled as two-dimensional Gaussian distributions, and the Wasserstein distance between them is calculated to measure the regression error of the bounding boxes. The formula is expressed as follows:

[0113]

[0114] Where: N a and N b Let c and y represent the Gaussian distributions modeled by the ground truth bounding boxes and the predicted bounding boxes, respectively. The ground truth bounding boxes and the predicted bounding boxes are denoted as (cx... a ,cy a ,w a ,h a ) and (cx b ,cy b ,w b ,h b ), cx a cy a w represents the x and y coordinates of the center point of the bounding box a. a and h brepresents the width and height of the center point of bounding box a; C is a constant closely related to the training dataset and affects the optimal performance of distance measurement.

[0115] The Normalized Wasserstein Distance (NWD) loss employed in this invention during model training effectively alleviates the problem of inaccurate localization of small targets. Compared to traditional GIoU or IoU losses, which only focus on the overlap between the predicted and ground truth bounding boxes, NWD loss treats the bounding boxes as a probability distribution to calculate distances, thus being more robust to positional deviations of small defects. Introducing NWD loss significantly improves the model's accuracy and recall in locating small defects, ensuring the reliability of PCB defect detection.

[0116] VII. Experimental Instructions

[0117] To better illustrate the advantages of the technical solution of the present invention, the following experiment is disclosed in this embodiment.

[0118] This experiment tested the Transformer algorithm for real-time detection of precision component defects using a publicly available dataset of PCB surface defects from Peking University.

[0119] 1. Dataset

[0120] This experiment uses the public dataset from the Intelligent Robotics Laboratory of Peking University for training and testing the network, combined with... Figure 6 As shown, the image defect types are classified as missing holes, rodent bites, open circuits, short circuits, burrs, and copper scrap. This dataset contains 693 PCB defect images and corresponding annotation files. First, the 693 defect images were randomly split into training, validation, and test sets according to a ratio of 7:1:2. However, due to the small dataset size, this would lead to overfitting. Therefore, we expanded the defect images in the training set by flipping, rotating, and filtering to reduce noise. Finally, the total number of images in the training set was expanded to 1940.

[0121] This experiment employed several evaluation metrics, including frames per second (FPS) and average precision (AP). These metrics are widely used in object detection to assess the accuracy and efficiency of models. FPS is often used indirectly as a metric to evaluate model speed, while AP is widely considered a key performance indicator for object detection tasks. AP is obtained by calculating the area under the accuracy curve, providing a measure of overall detection accuracy. The formula for calculating AP is as follows:

[0122]

[0123] Where X TP X FP and X FNThese represent true positive, false positive, and false negative, respectively. AP 50 and AP 95 These represent the average precision for IoU thresholds of 0.50 and 0.50 to 0.95, respectively.

[0124] 2. Training details and parameter settings

[0125] The algorithm model used in this experiment was trained on an Ubuntu 22.04 operating system. The training setup included an Intel Xeon(R) Platinum 8352V CPU, an NVIDIA RTX 4090 GPU with 24GB VRAM, and 120GB of system memory. CUDA version 11.8 and Python version 3.10 were used. The deep learning framework PyTorch version 2.1.2 was employed. The AdamW optimizer was used during model training. The configuration included: 200 epochs, a batch size of 8, 8 worker processes, an initial learning rate of 0.001, a momentum parameter of 0.9, and a weight decay parameter of 0.001.

[0126] 3. Experimental results

[0127] 1) Backbone test results

[0128] Visual inspection aims to achieve high accuracy and real-time performance. In the RT-DETR model, ResNet-18 is the smallest backbone network included in the original model. However, its parameter size is still larger than many recently released lightweight networks, such as Convnextv2 and FasterNet. The goal of this experiment is to reduce the parameter size of the detection model by performing backbone replacement experiments. The results are shown in Table 1.

[0129] Table 1 Comparison of Backbone network performance

[0130]

[0131] This experiment compared the ResNet-18 backbone network by replacing it with EfficientViT M0, Fasternet t0, EfficientFormer2, and Convnextv2atto. Table 1 shows that the lightweight networks performed poorly in defect detection. In PCB defects, which are mainly small, the FPS (frames per second) did not improve despite a significant reduction in model parameters. Furthermore, it was observed that networks with a stacked structure of 3×3 convolutions and residual connections had an advantage in extracting small objects. Therefore, this invention uses ResNet-18 as the backbone network and enhances it.

[0132] 2) Ablation experiment

[0133] Ablation experiments were conducted to verify the effectiveness of various optimization strategies. The experimental results are shown in Table 2. It can be observed that the BIFM structure enhances the focus on small defects, improving AP... S It improved by 4.0%. Although MS-C2F did not improve AP... S However, by introducing the MS-C2F structure on top of the BIFM structure, improvements were made in accuracy, recall, AP, and AP. S Further improvements have been made in this area; it is worth noting that AP S The accuracy increased by 8.0%. Finally, the introduction of NWD Loss significantly reduced the box function loss and decreased the sensitivity to small target locations. Compared with the original model, the improved EMSF-DETR model of this invention improved accuracy by 0.9%, recall by 1.6%, and AP. 50 It increased by 1.2%, AP 95 It increased by 1.4%, AP S It increased by 10%.

[0134] Table 2 Ablation Experiment

[0135]

[0136] a represents the introduction of BIFM; b represents the introduction of MS-C2F; c represents the introduction of NWD; AP s AP representing small goals

[0137] 3) Model Comparison Experiment

[0138] This experiment compared the RT-DETR model and the EMSF-DETR model using a test set containing 139 images. Figure 7 The presentation shows a sample randomly selected from each of six defect categories: voids, rodent bites, open circuits, short circuits, burrs, and copper inclusions. The descriptions show that the original RT-DETR model, due to the small size of the defects and their similarity in features, resulted in both false negatives and false positives for burrs, open circuits, and rodent bites. In contrast, the EMSF-DETR model demonstrated a higher confidence score in defect detection and accurately identified all present defects.

[0139] To further verify the superiority of the proposed EMSC-DETR model in terms of detection accuracy and generalization performance, we implemented eight other detection models and compared their collective performance with that of EMSC-DETR. These comparison models included CNN-based and Transformer-based models. To ensure fairness, the parameters or GFLOPs of the selected comparison models were very close to those of EMSC-DETR. The detection results of all comparison models are shown in Table 3.

[0140] Table 3 Model Comparison Experiment

[0141]

[0142] As shown in Table 3, among CNN-based models, the TOOD and YOLO series consistently outperform EMSF-DETR in terms of detection capability and generalization performance. It can be observed that the AP of the EMSF-DETR model... 50 These figures were 4.1%, 2.4%, 2.0%, 2.8%, and 2.7% higher than other models, respectively; AP 95 The AP values ​​were 5.0%, 2.1%, 2.5%, 2.7%, and 3.0% higher than other models, respectively. s These figures are 30%, 25%, 30%, 35%, and 42% higher than other models, respectively. Furthermore, EMSF-DETR boasts the highest FPS.

[0143] In the Transformer-based model, the AP of EMSF-DETR 50 These figures were 13.9%, 20.4%, 8.6%, and 1.2% higher than other models, respectively; AP 95 The AP values ​​were 10.7%, 16.5%, 13.0%, 2.7%, and 3.0% higher than other models, respectively. s These figures are 35%, 30%, 28%, and 10% higher than other models, respectively. Although its inference speed is 13.8 frames per second lower than the original RT-DETR, it is still sufficient for real-time PCB detection.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the technical solutions. Those skilled in the art should understand that modifications or equivalent replacements of the technical solutions of the present invention that do not depart from the purpose and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A multi-scale fusion intelligent detection method for quality defects in the machining of complex precision parts, characterized in that, include: S1: Obtain the PCB image to be inspected; S2: Input the PCB image to be detected into the trained defect detection model and output the corresponding PCB defect prediction information; Defect prediction information includes the bounding box coordinates and category labels of PCB defects; The processing steps of the defect detection model include: S201: Feature extraction of PCB images is performed through the backbone network to obtain initial feature maps at multiple scales; S202: A bidirectional feature pyramid is constructed from top to bottom and bottom to top to iteratively fuse initial feature maps at multiple scales, thereby obtaining a fused feature map that has both details and semantics. In the bidirectional feature fusion module: the multi-scale C2F module performs depthwise separable convolution and channel shuffling at multiple scales to extract multi-scale, multi-resolution target features; the efficient upsampling module upsamples the feature map step by step, upsampling the feature map at the current scale to the same resolution as the feature map at the next scale and then fusing them. S203: After decoding the fused feature maps at various scales by the detection head, the bounding box coordinates and category labels of PCB defects are obtained, which are the PCB defect prediction information. S3: Use the PCB defect prediction information output by the defect detection model as the intelligent detection result of the processing quality defects of the PCB image to be detected.

2. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 1, characterized in that: In step S201, the pre-trained ResNet-18 is used as the backbone network to extract features from the PCB image.

3. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 1, characterized in that: Initial feature maps at four scales are obtained through the backbone network.

4. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 3, characterized in that: In step S202, the bidirectional feature fusion module includes a cascaded encoder, a first feature fusion module, a first multi-scale C2F module, a first high-efficiency upsampling module, a second feature fusion module, a second multi-scale C2F module, a second high-efficiency upsampling module, a third feature fusion module, a third multi-scale C2F module, a fourth feature fusion module, a fourth multi-scale C2F module, a fifth feature fusion module, a fifth multi-scale C2F module, a sixth feature fusion module, and a sixth feature fusion module. A convolutional combination module 1 is connected between the encoder and the first feature fusion module; the convolutional combination module 1 includes a cascaded 1×1 convolutional layer, a batch normalization layer and a SiLU activation function; There is a convolutional combination module 2 between the fourth multi-scale C2F module and the fifth feature fusion module, and between the fifth multi-scale C2F module and the sixth feature fusion module; the convolutional combination module 2 includes cascaded 3×3 convolutional layers, batch normalization layers and SiLU activation functions; Specifically, the initial feature map at the fourth scale (the smallest scale) is processed by convolutional combination module one, and the result is input to the encoder; the initial feature map at the third scale is processed by convolutional combination module one and convolutional combination module two, and the result is input to the first feature fusion module; the initial feature map at the second scale is processed by convolutional combination module one and convolutional combination module two, and the result is concatenated with the initial feature map at the third scale processed by convolutional combination module one and convolutional combination module two, and then input to the second feature fusion module; the initial feature map at the first scale is processed by convolutional combination module two, and the result is concatenated with the initial feature map at the second scale processed by convolutional combination module one and convolutional combination module two, and then input to the third feature fusion module. The output of the second multi-scale C2F module, after being processed by the second convolutional combination module, is added element-wise to the output of the first multi-scale C2F module, and the result is input to the sixth feature fusion module. The outputs of the second and third multi-scale C2F modules, after being processed by the second convolutional combination module, are added element-wise to the output of the first efficient upsampling module and then input into the fifth feature fusion module. The output of the second high-efficiency upsampling module is input to the fourth feature fusion module; Finally, the fused feature maps of the three scales are output through the fourth, fifth, and sixth multi-scale C2F modules.

5. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 4, characterized in that: The encoder is an intra-scale feature interaction encoder based on an attention mechanism.

6. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 4, characterized in that: The feature fusion module is a fast normalization fusion module.

7. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 4, characterized in that: The multi-scale C2F module includes cascaded 1×1 convolutional layers, batch normalization layers, ReLU6 activation function layers, segmentation modules, several multi-scale deep convolutional modules, splicing modules, channel rearrangement operation modules, 1×1 pointwise convolutional layers, and batch normalization layers. The segmentation module divides the feature map into two feature sub-maps: one feature sub-map is input into multiple cascaded multi-scale deep convolutional modules for processing and the output of each multi-scale deep convolutional module is input into the concatenation module, where the output of all multi-scale deep convolutional modules is concatenated; the other feature sub-map is residually concatenated with the output of the next batch normalization layer. The formula for the multi-scale C2F module is expressed as follows: In the formula: MS-C2F represents a multi-scale C2F module; PWC1 represents a pointwise convolutional layer; BN represents a batch normalization layer; R6 represents a ReLU6 activation function layer; N represents N multi-scale depthwise convolutional modules; CS stands for Channel Rearrangement; MSDC stands for Multi-Scale Depth Convolution Module.

8. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 7, characterized in that: The multi-scale deep convolution module includes three branches. Each branch includes a cascaded deep convolutional layer of the corresponding kernel size, a batch normalization layer, and a ReLU6 activation function layer. The outputs of the three branches are summed element by element and then fed into the channel rearrangement operation module to obtain the output of the multi-scale deep convolution module. The formula for a multi-scale depthwise convolution module is expressed as follows: Each multi-scale deep convolutional module has a residual connection, i.e., x = x + MSDC(x); In the formula: MSDC represents a multi-scale depthwise convolutional module; x represents the input feature map; DWCB ks The depthwise convolutional layers with different kernel sizes are followed by batch normalization layers and ReLU6 activation function layers. ks represents the kernel size, and KS indicates that there are convolutional kernels of different sizes.

9. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 4, characterized in that: The high-efficiency upsampling module includes a cascaded upsampling module, a 3×3 depthwise convolutional layer, a channel rearrangement operation module, and a 1×1 convolutional layer; The formula for the high-efficiency upsampling module is expressed as: EUSB(x)=C 1x1 (CS(DWC 3x3 (UP(x)))); In the formula: EUSB represents the high-efficiency upsampling module; UP represents the ordinary upsampling layer (x2); DWC 3x3 This indicates a depthwise convolutional layer with a kernel size of 3×3; C 1x1 This represents a 1×1 convolutional layer.

10. The intelligent detection method for quality defects in the machining of complex precision parts by multi-scale fusion as described in claim 1, characterized in that: In step S2, when training the defect detection model, the normalized Wasserstein distance loss is used as the loss function for bounding box regression. The predicted bounding boxes and the ground truth bounding boxes are modeled as two-dimensional Gaussian distributions, and the Wasserstein distance between them is calculated to measure the regression error of the bounding boxes. The formula is expressed as follows: Where: N a and N b Let c and y represent the Gaussian distributions modeled by the ground truth bounding boxes and the predicted bounding boxes, respectively. The ground truth bounding boxes and the predicted bounding boxes are denoted as (cx... a ,cy a ,w a ,h a ) and (cx b ,cy b ,w b ,h b ), cx a cy a w represents the x and y coordinates of the center point of bounding box a. a and h b Let C represent the width and height of the center point of bounding box a; C is a constant closely related to the training dataset.

Images