PCB defect detection method and system based on improved RT-DETR

Abstract

The invention relates to a PCB defect detection method based on improved RT-DETR, and belongs to the technical field of PCB detection, and the method comprises the steps: S1, obtaining a PCB image data set, processing the PCB image data set through employing a mixed object migration HOM data enhancement strategy, and generating an enhanced training sample; s2, an improved RT-DETR model is constructed, wherein the improved RT-DETR model comprises a hybrid fast Fourier transform network HybFFT-Net backbone network and a cross-layer interaction neck CLI-Neck; s3, training the improved RT-DETR model by using the enhanced training sample, and optimizing model parameters to obtain a trained PCB defect detection model; and S4, the to-be-detected PCB image is input into the trained PCB defect detection model, a PCB defect detection result is obtained through model reasoning, and the defect detection result comprises a defect position and a defect category. According to the method, cross-image migration and reconstruction of a real defect target are utilized, and the adaptability of the model to diversified backgrounds and complex scenes is effectively improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of PCB detection, in particular to a PCB defect detection method based on an improved RT-DETR. BACKGROUND

[0002] With the rapid development of electronic devices towards high density, multifunction and miniaturization, the manufacturing process complexity of printed circuit boards (PCB) as core components is continuously improved, leading to the diversification of defect types, and higher requirements for the precision, efficiency and reliability of defect detection. At present, the industrial field widely adopts automated optical inspection (AOI, Automated Optical Inspection) technology for PCB defect recognition. This technology realizes non-contact rapid detection by combining high-resolution imaging with traditional image processing algorithms. However, the AOI system has poor detection robustness in practical application, and it is difficult to effectively identify defects with low contrast, similar color or complex background, especially when facing small defects such as pinholes, burrs, and virtual welding, which are prone to missed detection and false detection.

[0003] The original RT-DETR model still has the following limitations when applied to high-precision PCB defect detection tasks: (1) In the case of limited sample size and uneven class distribution, the model is prone to overfitting during training, which leads to insufficient generalization ability on unseen samples, making it difficult to adapt to complex and variable industrial detection scenarios; (2) The traditional convolutional backbone has limited multi-scale modeling capability in feature extraction, making it difficult to balance local detail modeling and global semantic dependency, thereby weakening the model's representation ability for different types of defects; (3) In the feature fusion stage, most methods still rely on simple addition or splicing operations, and the cross-scale information interaction is insufficient, which not only easily leads to the details of small targets being submerged, but also to some extent restricts the robustness and computational efficiency of the overall feature representation. SUMMARY

[0004] The purpose of the present application is to solve the problems in the background art, and to provide a PCB defect detection method based on an improved RT-DETR, which effectively improves the model's adaptability to diversified backgrounds and complex scenes by using cross-image migration and reconstruction of real defect targets.

[0005] The technical scheme of the present application: a PCB defect detection method based on an improved RT-DETR, comprising, S1, obtaining a PCB image dataset, and processing the PCB image dataset using a hybrid object migration (HOM) data enhancement strategy to generate enhanced training samples; S2. Construct an improved RT-DETR model, which includes a hybrid fast Fourier transform network HybFFT-Net backbone network and a cross-layer interactive neck CLI-Neck; S3. Train the improved RT-DETR model using enhanced training samples, optimize the model parameters, and obtain the trained PCB defect detection model. S4. Input the PCB image to be detected into the trained PCB defect detection model, and obtain the PCB defect detection result through model inference. The defect detection result includes the defect location and defect category.

[0006] Preferably, in step S2, the HybFFT-Net backbone network is divided into five hierarchical stages, with the feature resolution of each stage decreasing to 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image resolution, respectively. Each hierarchical stage includes a Hybrid Fourier Feature Block (HFF-Block). The HFF-Block includes a Hybrid Fourier Mixer (HF-Mixer) and an Adaptive Convolutional Feedforward Network (AK-FFN). The CLI-Neck includes a Multi-Scale Feature Encoder (MSFE), a Feature Concatenation Module (FMS), and a lightweight structure.

[0007] Preferably, in step S1, the specific process of the Hybrid Object Migration (HOM) data augmentation strategy is as follows: extract the target region containing defects from the first training image in the PCB image dataset, and separate the real pixel blocks of the target region; migrate and paste the real pixel blocks to any position in the second training image in the PCB image dataset, wherein the second training image is different from the first training image; perform validity verification on the pasted second training image, and after confirming that there are no pixel conflicts, use it as an augmentation training sample.

[0008] Preferably, the working process of HF-Mixer is as follows: A1. Receive the input features from the corresponding layer stage in the HybFFT-Net backbone network, and expand the number of channels of the input features to twice the original number of channels through convolution operations; A2. Divide the expanded channel features into two groups along the channel dimension, which are used as local branch features and global branch features, respectively. A3. Processing local branch features: split them into two groups of sub-channel features, apply depthwise separable convolutions with dilation rates of 1 and 2 respectively, and then concatenate the two groups of processed sub-channel features along the channel dimension to obtain local enhancement features; A4. Processing the global branch feature: mapping it to the frequency domain by two-dimensional discrete Fourier transform (DFT) to obtain the frequency domain feature; using frequency domain position encoding (FPE) to integrate position information, FPE separates the real part and the imaginary part of the frequency domain feature and merges them in the channel dimension, and then learns the local neighborhood relationship in the frequency domain dimension through deep separable convolution; using frequency domain dynamic convolution (FDC) to generate an adaptive filter to weight the frequency domain feature integrated with position information, highlighting the discriminative frequency components; mapping the weighted frequency domain feature back to the spatial domain through inverse Fourier transform to obtain the global enhanced feature; A5. Concatenating the local enhanced feature and the global enhanced feature in the channel dimension, fusing through a nonlinear activation function, and then compressing the channel number through 1x1 convolution to obtain the output feature of HF-Mixer.

[0009] Preferably, the working process of AK-FFN is: B1. Receiving the output feature of HF-Mixer, expanding its channel number to twice the original channel number through 1x1 convolution to obtain an intermediate feature; B2. Dividing the intermediate feature into multiple sub-features along the channel dimension; B3. Applying deep separable convolution with different kernel sizes to each sub-feature, including 3x3, 5x5 and 7x7; B4. Concatenating all sub-features processed by deep separable convolution in the channel dimension, enhancing nonlinear expression through GELU activation function, and then compressing the channel number to the original channel number through 1x1 convolution to obtain the output feature of AK-FFN.

[0010] Preferably, the working process of MSFE is: C1. Receiving multi-scale feature maps output by different hierarchical stages in the HybFFT-Net backbone network, including first scale feature map, second scale feature map and third scale feature map; C2. Convolutionally transforming the first scale feature map and the second scale feature map respectively, adjusting their spatial sizes through nearest neighbor interpolation to align the spatial sizes of the three with the third scale feature map; C3. Extending the dimensions of the three aligned scale feature maps respectively, converting the two-dimensional feature maps to three-dimensional features; C4. Concatenating the three three-dimensional features in the extended dimension, processing the concatenated three-dimensional combined feature using 3D convolution to extract multi-scale correlation information; C5. Normalizing and max-pooling the feature processed by 3D convolution, and then converting it to a two-dimensional feature map through dimension compression to obtain a multi-scale fusion feature.

[0011] Preferably, the working process of FMS is: D1. receiving the multi-scale fusion features output by the MSFE and the different scale feature maps output by the HybFFT-Net backbone network, determining large-scale feature maps, medium-scale feature maps and small-scale feature maps; D2. performing convolution operations on the large-scale feature maps, the medium-scale feature maps and the small-scale feature maps respectively, adjusting the channel numbers of the three to unify the channel numbers to the original channel number of the medium-scale feature maps; D3. adopting a hybrid downsampling structure of "max pooling + average pooling" for the large-scale feature maps after adjusting the channels to reduce the spatial dimensions thereof; adopting nearest neighbor interpolation for upsampling of the small-scale feature maps after adjusting the channels to expand the spatial dimensions thereof, so that the spatial dimensions of the three are aligned with those of the medium-scale feature maps; D4. concatenating the three feature maps after aligning the spatial dimensions and the channel dimensions in the channel dimension to obtain the output features of the FMS.

[0012] Preferably, the lightweight structure includes a GSConv and a VoV-GSCSP module; the working process of the GSConv is as follows: the input features are divided into main branch features and auxiliary branch features; the main branch features are adjusted from an input channel number c1 to half of an output channel number c2 through standard convolution (SC) to extract cross-channel relationships; the auxiliary branch features are extracted through deep separable convolution DSConv with a large convolution kernel to extract spatial features; the features processed by the main branch and the features processed by the auxiliary branch are fused, cross-channel information interaction is promoted through channel shuffle ChannelShuffle, and GSConv output features are obtained; the working process of the VoV-GSCSP module is as follows: the input features are divided into first path features and second path features; the first path features are subjected to deep feature extraction through a GSBottleneck layer; the second path features are subjected to lightweight processing only through simple convolution operation; the features processed by the two paths are concatenated in the channel dimension, and are integrated and optimized through a final convolution layer to obtain VoV-GSCSP module output features.

[0013] An improved RT-DETR-based PCB defect detection system, comprising, An image acquisition module: configured to acquire a PCB image dataset and a PCB image to be detected, the PCB image dataset comprising a plurality of PCB training images with or without defects; A data enhancement module: configured to process the PCB image dataset using a hybrid object migration (HOM) data enhancement strategy to generate enhanced training samples; a model construction and training module: configured to construct an improved RT-DETR model and train the improved RT-DETR model using the enhanced training samples to obtain a trained PCB defect detection model; Defect detection module: for inputting the PCB image to be detected into the trained PCB defect detection model, and obtaining the PCB defect detection result through model inference Result output and management module: for outputting the defect detection result, and having the functions of model project management, detection result export and local saving, the defect detection result including the image with defect annotation box and category label.

[0014] Preferably, the model construction and training module comprises a network structure definition submodule, a loss function configuration submodule and a parameter optimization submodule; the network structure definition submodule is used for defining the specific structures of HFF-Block in the HybFFT-Net backbone network and MSFE, FMS and lightweight structure in CLI-Neck; the loss function configuration submodule is used for configuring the combined loss function, i.e., cross-entropy loss function + intersection over union loss function; and the parameter optimization submodule is used for iteratively optimizing the model parameters based on the enhanced training samples through the gradient descent algorithm.

[0015] Compared with the prior art, the present application has the following beneficial technical effects: In the present application, a PCB defect detection model with high precision, strong robustness and low resource consumption is established, which has been actually deployed in a PCB defect detection system designed independently, realizing the full-process automation from image input, intelligent analysis to result output, and the system integrates the functions of visual detection interface, model project management, detection result export and local saving, etc., thereby improving the intelligent level and application convenience of industrial quality inspection.

[0016] In the present embodiment, the cross-image migration and reconstruction of real defect targets effectively improve the adaptability of the model to diversified backgrounds and complex scenes. The feature distortion problem introduced by traditional geometric or color disturbance is avoided, and the sample category balance can be realized through active copying of rare defects, thereby reducing overfitting and enhancing the generalization performance.

[0017] The core HFF-Block introduces frequency domain position encoding and dynamic convolution through HF-Mixer to capture global semantics and long-range dependencies; meanwhile, the multi-scale convolution mechanism of AK-FFN is combined to enhance the sensitivity of local edges and textures. This design takes into account both local details and global semantics, effectively improving the discriminability and robustness of feature representation, and is especially suitable for detection tasks of small defects and complex backgrounds.

[0018] The Cross-Layer Interaction Neck (CLI-Neck) realizes sufficient fusion of cross-scale features through a multi-scale feature encoder (MSFE) and a feature splicing module (FMS), breaking through the limitations of traditional FPN relying only on addition or splicing. At the same time, combined with GSConv and VoV-GSCSP and other lightweight structures, the computational overhead and parameter size are greatly reduced while ensuring the sufficiency of feature expression and detection accuracy. This structure not only enhances the model's detection ability for small targets and complex scenes, but also improves the feasibility and practical value of deploying industrial edge devices.

[0019] The system has high integration and strong practicality: the improved model is deployed in the PCB defect detection system designed independently, integrating image import, intelligent analysis, visual result display, model project management, detection result export and local storage and other functions, improving the quality inspection efficiency and intelligent level, and having good engineering application prospect and industrialization value.

[0020] In summary, the present application is superior to the prior art in terms of detection accuracy, generalization ability, model lightweight and system practicality, solves the problems of low precision, difficulty in identifying small targets and large model in current PCB defect detection, has outstanding substantial characteristics, and has broad application prospect and industrialization value in the field of industrial quality inspection. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 The mixed object migration data enhancement diagram in the present application is shown in the figure; Figure 2 The mixed fast Fourier transform network in the present application is shown in the figure; Figure 3 The mixed Fourier mixer structure in the present application is shown in the figure; Figure 4 The adaptive convolution feedforward network structure in the present application is shown in the figure; Figure 5 The cross-layer interaction neck structure in the present application is shown in the figure; Figure 6 The multi-scale feature encoder structure in the present application is shown in the figure; Figure 7 The feature map splicing module structure in the present application is shown in the figure; Figure 8 This is a schematic diagram of the GSConv lightweight convolutional structure in this invention; Figure 9 This is a schematic diagram of the VoV-GSCSP structure in this invention; Figure 10 This is a schematic diagram of the operation interface of the detection system in this invention. Detailed Implementation

[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0025] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive, either alone or selectively, with other embodiments.

[0026] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0027] Example 1 like Figures 1-10 As shown, the present invention proposes a PCB defect detection method based on improved RT-DETR, comprising: S1. Obtain the PCB image dataset and use the Hybrid Object Transfer (HOM) data augmentation strategy to process the PCB image dataset and generate augmented training samples. S2. Construct an improved RT-DETR model, which includes a hybrid fast Fourier transform network HybFFT-Net backbone network and a cross-layer interactive neck CLI-Neck; S3. Train the improved RT-DETR model using enhanced training samples, optimize the model parameters, and obtain the trained PCB defect detection model. S4, input the PCB image to be detected into the trained PCB defect detection model, and obtain a PCB defect detection result through model inference. The defect detection result includes a defect position and a defect category.

[0028] In step S2, the HybFFT-Net backbone network is divided into five hierarchical stages, and the feature resolution of each hierarchical stage is 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image resolution in turn. Each hierarchical stage includes a hybrid Fourier feature block HFF-Block. The HFF-Block includes a hybrid Fourier mixer HF-Mixer and an adaptive convolution feed-forward network AK-FFN. The CLI-Neck includes a multi-scale feature encoder MSFE, a feature splicing module FMS and a lightweight structure. As shown in Figure 2 The module is composed of a spatial convolution branch and a frequency domain modeling branch: the former preserves fine-grained details and edge information, and the latter uses Fourier transform to capture long-range dependencies and global semantics. The shallow layer highlights local structures, the middle layer coordinates local and global feature interactions, and the deep layer strengthens semantic consistency. The core components of the HFF-Block include a hybrid Fourier mixer HF-Mixer for frequency domain feature integration and an adaptive kernel feed-forward network AK-FFN for multi-scale representation learning.

[0029] In step S1, the specific process of the hybrid object migration HOM data augmentation strategy is as follows: a target region containing a defect is extracted from a first training image in a PCB image dataset, and real pixel blocks of the target region are separated; the real pixel blocks are migrated and pasted to any position of a second training image in the PCB image dataset, and the second training image is different from the first training image; the pasted second training image is verified for effectiveness, and after confirming that there is no pixel conflict, it is used as an augmented training sample. As shown in Figure 1 The strategy extracts a target region containing a defect from a training image, migrates and pastes real pixel blocks of the target region to any position of another image to generate a new training sample. This method avoids the feature distortion problem caused by geometric transformation or color disturbance in traditional data augmentation, and directly uses real defect samples for reconstruction, so that the model learns the natural performance of the defect in different contexts; The HOM (Host-Oriented Model) strategy not only enhances the model's adaptability to background changes but also effectively alleviates overfitting during training. Especially when defect sample categories are unevenly distributed, it can balance the data distribution by actively replicating rare defects, thus improving the model's ability to identify tail-end categories. Furthermore, this strategy encourages the model to focus more on the spatial relationship between the target and its surrounding environment, rather than relying on fixed background patterns, thereby improving detection robustness in complex industrial scenarios.

[0030] The working process of HF-Mixer is as follows: A1. Receive the input features from the corresponding layer stage in the HybFFT-Net backbone network, and expand the number of channels of the input features to twice the original number of channels through convolution operations; A2. Divide the expanded channel features into two groups along the channel dimension, which are used as local branch features and global branch features, respectively. A3. Processing local branch features: split them into two groups of sub-channel features, apply depthwise separable convolutions with dilation rates of 1 and 2 respectively, and then concatenate the two groups of processed sub-channel features along the channel dimension to obtain local enhancement features; A4. Processing global branch features: Map the features to the frequency domain using a two-dimensional discrete Fourier transform (DFT) to obtain frequency domain features; incorporate location information into the frequency domain features using frequency domain position encoding (FPE). FPE separates the real and imaginary parts of the frequency domain features and merges them along the channel dimension. Then, learn local neighborhood relationships in the frequency domain dimension through depthwise separable convolution; generate an adaptive filter using frequency domain dynamic convolution (FDC) to weight the frequency domain features incorporating location information, highlighting discriminative frequency components; map the weighted frequency domain features back to the spatial domain using an inverse Fourier transform to obtain global enhanced features; A5. The local enhancement features and global enhancement features are concatenated along the channel dimension, fused through a non-linear activation function, and then compressed by a 1×1 convolution to obtain the output features of HF-Mixer.

[0031] like Figure 3 As shown, the input features are first doubled in number through convolution, and then divided into two branches along each channel: a local branch and a global branch based on frequency domain modeling. The local branch uses depthwise separable convolution with multiple dilation rates to acquire spatial neighborhood information under different receptive fields, thus taking into account both fine-grained edges and a wider range of context. The global branch maps the features to the frequency domain through a two-dimensional discrete Fourier transform (DFT), performs adaptive dynamic filtering in the frequency domain to highlight informative frequency components and suppress redundancy, and then performs an inverse transform back to the spatial domain to model long-range dependencies and global semantics. The outputs of the two branches are finally concatenated along the channel dimension and fused into a unified representation through nonlinear activation.

[0032] Local branches: The input features are further split into two groups after channel division, and depth separable convolutions with dilation rates of 1 and 2 are applied respectively, and then the local enhanced features are obtained by concatenating the two groups in the channel dimension:

[0033] wherein represents a depth separable convolution with a dilation rate of , , are two groups of sub-channel features. This design expands the effective receptive field without significantly increasing the computation, and improves the sensitivity to small defect edges and textures, aiming to enhance the local modeling capability in the spatial domain; Global branch: The other part of the input after channel division is mapped to the frequency domain, focusing on long-range dependencies and overall semantic relationships in the frequency domain:

[0034] wherein, represents the result of two-dimensional discrete Fourier transform, and are spatial dimensions, is the frequency domain coordinate, and the transformation expresses the spatial features as a combination of frequency components: high frequencies correspond to detailed textures, and low frequencies correspond to global structures; In the frequency domain representation, the network first incorporates position information through frequency-conditional positional encoding (FPE), and then uses adaptive filters generated by frequency dynamic convolution (FDC) for weighting to highlight key frequency components and suppress redundancy. FPE separates the real and imaginary parts and merges them in the channel dimension, and learns local neighborhood relationships in each channel through depth separable convolution. As shown in the formula, represents local depth convolution in the frequency domain dimension. This is equivalent to introducing neighborhood dependence on frequency components, so that the model not only depends on the amplitude spectrum and phase spectrum itself, but also captures the spatial corresponding relationship between adjacent components in the frequency domain:

[0035] After obtaining the frequency domain features with position information, the network further generates adaptive filters through frequency dynamic convolution (FDC) to weight different frequency components. Specifically, the frequency weights are first predicted through 1x1 convolution and Softmax function

[0036] At each frequency position, the convolution kernels of different groups are adjusted adaptively according to , so as to highlight discriminative frequency components and suppress irrelevant noise;

[0037] wherein denotes the group of frequency domain convolution kernels, is the corresponding dynamic weight. In this way, the model can achieve more fine adaptive adjustment in the frequency domain, thereby enhancing the modeling ability of global semantic information; The global branch is mapped back to the spatial domain through inverse Fourier transform and spliced with the local branch in the channel dimension. Through the channel attention mechanism, adaptive fusion is realized, effectively highlighting key information and suppressing redundant features. Finally, 1x1 convolution is used to complete channel compression, generating an output representation that simultaneously has local details and global context, thereby providing more robust and discriminative feature support for subsequent tasks.

[0038] The working process of AK-FFN is as follows: B1. Receive the output features of HF-Mixer, expand the channel number to twice the original channel number through 1x1 convolution, and obtain intermediate features; B2. Divide the intermediate features into multiple sub-features along the channel dimension; B3. Apply depth separable convolution with different kernel sizes to each sub-feature, including 3x3, 5x5 and 7x7; B4. Concatenate all sub-features processed by depth separable convolution in the channel dimension, enhance non-linear expression through GELU activation function, and then compress the channel number to the original channel number through 1x1 convolution to obtain the output features of AK-FFN.

[0039] AK-FFN achieves more rich spatial feature expression through multi-scale convolution kernels, and the structure is as shown in Figure 4 ; The input features are first expanded in the channel dimension through 1x1 convolution to obtain an intermediate representation with twice the number of channels. Then, the representation is divided into multiple sub-features, each of which is fed into a depth separable convolution with different kernel sizes including 3x3, 5x5 and 7x7 to capture spatial dependencies under different receptive fields. Finally, the branch output is concatenated, activated by GELU, and compressed in the channel through 1x1 convolution to restore the original dimension, forming a multi-scale spatial perception capable of forward output;

[0040] wherein, denotes the sub-set after dividing the input features in the channel dimension; corresponds to , and Depthwise separable convolutions are used to extract spatial dependencies under different receptive fields; through Activation functions enhance nonlinear expressive power, and finally utilize... Convolution completes channel compression, yielding the final feedforward output. Compared to traditional feedforward structures with a single convolutional kernel, AK-FFN can dynamically integrate spatial representations at different scales, thereby achieving richer and more discriminative feature reconstruction. This design not only improves the model's adaptability in complex scenarios but also provides support for the collaborative modeling of frequency and spatial domain features.

[0041] The working process of MSFE is as follows: C1. Receive multi-scale feature maps output from different hierarchical stages in the HybFFT-Net backbone network, including first-scale feature maps, second-scale feature maps, and third-scale feature maps; C2. Perform convolution transformations on the first-scale feature map and the second-scale feature map respectively, and adjust their spatial dimensions through nearest neighbor interpolation so that the spatial dimensions of the three maps are aligned with the third-scale feature map; C3. Perform dimensional expansion on the three scale feature maps after alignment to convert the two-dimensional feature maps into three-dimensional features; C4. The three 3D features are concatenated in the extended dimension, and 3D convolution is used to process the concatenated 3D combined features to extract multi-scale correlation information; C5. Normalize and max pool the features after 3D convolution, and then convert them into two-dimensional feature maps through dimensionality compression to obtain multi-scale fused features.

[0042] The working process of FMS is as follows: D1. Receive the multi-scale fusion features output by MSFE and the feature maps of different scales output by the HybFFT-Net backbone network, and determine the large-scale feature map, medium-scale feature map and small-scale feature map; D2. Perform convolution operations on the large-scale feature map, medium-scale feature map, and small-scale feature map respectively, and adjust the number of channels of the three to unify the number of channels to the original number of channels of the medium-scale feature map; D3. For the large-scale feature map after channel adjustment, a hybrid downsampling structure of "max pooling + average pooling" is used to reduce its spatial dimension; for the small-scale feature map after channel adjustment, nearest neighbor interpolation is used for upsampling to expand its spatial dimension, so that the spatial dimensions of the three are aligned with the medium-scale feature map. D4. After aligning the three feature maps in both spatial and channel dimensions, concatenate them along the channel dimension to obtain the output features of FMS.

[0043] HybFFT-Ne realizes the collaborative representation of local details and global semantics by combining spatial and frequency domain modeling in a unified framework. The core HFF-Block uses HF-Mixer to introduce position encoding and dynamic convolution in the frequency domain to highlight key signals, while enhancing the modeling of fine-grained edges and textures in the spatial domain. Combined with the multi-scale convolution mechanism of AK-FFN, the capture ability of features under different receptive fields is effectively improved. This structure not only enhances the adaptability of the model to small defects and complex scenes, but also significantly improves the discriminability and robustness of feature reconstruction, providing technical support for high-precision and real-time industrial detection tasks.

[0044] The overall architecture of the Cross-Layer Interaction Neck (CLI-Neck) is shown in Figure 5 . This structure aims to improve the sufficiency and robustness of feature fusion, and realizes efficient transmission of cross-scale information through multi-layer interaction mechanism. Unlike the traditional FPN simple addition or splicing method, the CLI-Neck designs a Multi-Scale Feature Encoder (MSFE) and a Feature Map Stitch (FMS), the former uses three-dimensional convolution to model the correlation between different levels of features, and the latter realizes efficient splicing of cross-scale features through upsampling and downsampling, thereby enhancing the expression ability of fused features. At the same time, in order to reduce the computational overhead and adapt to edge device deployment, the CLI-Neck integrates GSConv and VoV-GSCSP and other lightweight structures, which reduces the parameter quantity and computational complexity while maintaining detection accuracy.

[0045] To more effectively improve the model's ability to detect targets of different sizes and obtain more robust feature representations in complex scenes, the MSFE is designed, as shown in Figure 6 . This module first performs convolutional transformation on and feature maps, and adjusts their spatial dimensions through nearest neighbor interpolation to align with feature maps for subsequent multi-scale feature fusion. In the fusion stage, each feature map is expanded to a three-dimensional feature through operation, and then spliced in the expanded dimension. Subsequently, 3D convolution is used to uniformly process the three-dimensional combined features to extract more rich multi-scale information. After completing the 3D convolution, normalization and max pooling are further introduced to optimize feature expression, and finally operation is used to compress it into a two-dimensional feature map to obtain the fused multi-scale representation;

[0046] The design breaks through the limitation of traditional FPN which only fuses features by simple addition or concatenation, can more fully excavate the space-channel correlation between different levels of features, and effectively enhance the sensitivity to small targets and the ability to understand the context semantics. After fusion, the features are normalized, maximally pooled and dimensionally compressed, and the output is a 2D feature map rich in multi-scale information.

[0047] As shown in Figure 7 , the FMS module is used to concatenate three different scale feature maps: large scale, medium scale and small scale. First, the channel number of each layer feature map is adjusted to unify the feature expression of the main scale. For the large scale feature map, after adjusting the channel number by convolution operation, a hybrid downsampling structure combining maximum pooling and average pooling is used, which can not only reduce the spatial dimension, but also enhance the translation invariance. For the small scale feature map, first adjust the channel number through the convolution module, and then use nearest neighbor interpolation for upsampling to maintain local features and avoid loss of small target information.

[0048] In the formula, FMS represents the output fusion feature map, , and correspond to the large scale, medium scale and small scale feature maps respectively. Finally, the three feature maps processed and aligned to the same dimension are concatenated in the channel dimension, and the output channel number is the same as the medium scale feature Figure 1 , thereby enhancing the model's ability to capture fine-grained feature information.

[0049] The lightweight structure includes GSConv and VoV-GSCSP modules; the working process of GSConv is: the input features are divided into main branch features and auxiliary branch features; the main branch features are adjusted from the input channel number c1 to half of the output channel number c2 through standard convolution (SC), to extract cross-channel relationships; the auxiliary branch features are extracted through deep separable convolution DSConv with large kernel to extract spatial features; the features processed by the main branch and the features processed by the auxiliary branch are fused, and cross-channel information interaction is promoted through channel shuffle ChannelShuffle, to obtain the GSConv output features; the working process of VoV-GSCSP module is: the input features are divided into first path features and second path features; the first path features are processed through GSBottleneck layer for deep feature extraction; the second path features are only processed through simple convolution operation for lightweight processing; the features processed by the two paths are concatenated in the channel dimension, and are integrated and optimized through the final convolution layer to obtain the VoV-GSCSP module output features.

[0050] As the model scales up, the computational overhead and storage consumption increase significantly, especially when deploying real-time detectors on resource-constrained edge devices. To address this challenge, we introduce GSConv to reduce the additional parameter overhead brought by model enhancement, thereby reducing the amount of computation and the number of parameters. On this basis, we also use an improved VoV-GSCSP module to replace the traditional bottleneck structure to speed up the inference process. This design effectively reduces the parameter size and complexity of the model while maintaining detection accuracy.

[0051] As Figure 8 shown, the structure of GSConv consists of two branches for optimizing convolution calculation. The main branch reduces the channel dimension by half through standard convolution (SC), adjusting the input channel number c1 to the output channel number c2. This not only reduces the channel dimension, but also reduces the amount of data for subsequent calculations. At the same time, standard convolution can effectively extract spatial information and channel information of input features, so this branch is responsible for capturing potential cross-channel relationships. The auxiliary branch uses depth-wise separable convolution (DSConv) with a large kernel to focus on extracting spatial features. The use of a large kernel allows it to cover a larger receptive field, enhancing feature representation capabilities without significantly increasing parameter size. After processing by both branches, the features are fused, and then a channel shuffle operation is performed to promote cross-channel information exchange, further improving feature utilization efficiency. The VoV-GSCSP module divides the input features into two parallel paths. One path uses the GSBottleneck layer to extract deep features, while the other path bypasses the complex GSBottleneck and only uses simple convolution operations for lightweight processing. This design strategy ensures that the model can effectively reduce computational overhead while still efficiently learning and utilizing different levels of feature information. After processing by both paths, their outputs are concatenated in the channel dimension and then integrated and optimized by a final convolution layer. This process not only improves gradient flow but also enhances feature reuse efficiency. With this structure, VoV-GSCSP exhibits superior performance in tasks that require multi-level feature learning, especially for real-time detector architectures. It reduces computational costs while maintaining high accuracy, making it suitable for deployment in resource-constrained industrial edge computing environments.

[0052] The CLI-Neck realizes more sufficient interaction and fusion between cross-scale features through the collaborative design of a multi-scale feature encoder (MSFE) and a feature splicing module (FMS), effectively improves the detection capability of defects in small targets and complex scenes, and at the same time, in combination with GSConv and VoV-GSCSP and other lightweight structures, the calculation overhead and the parameter scale are greatly reduced on the premise of maintaining high detection accuracy and real-time performance. The design not only enhances the sufficiency and robustness of feature representation, but also ensures the deployability and practical value of the model on industrial edge devices.

[0053] The application constructs a PCB defect detection model with high precision, strong robustness and low resource consumption. The model has been actually deployed in a PCB defect detection system designed independently, realizing the full-process automation from image input, intelligent analysis to result output. The system integrates a visual detection interface, model project management, detection result export and local saving functions, improving the intelligent level and application convenience of industrial quality inspection.

[0054] In the embodiment, the cross-image migration and reconstruction of real defect targets effectively improve the adaptability of the model to diversified backgrounds and complex scenes. The feature distortion problem introduced by traditional geometric or color disturbance is avoided, and the sample class balance can be realized through active copying of rare defects, thereby reducing overfitting and enhancing generalization performance.

[0055] The core HFF-Block introduces frequency domain position encoding and dynamic convolution through the HF-Mixer to capture global semantics and long-range dependencies. At the same time, combined with the multi-scale convolution mechanism of AK-FFN, the sensitivity of local edges and textures is enhanced. This design takes into account local details and global semantics, effectively improving the discriminability and robustness of feature representation, especially suitable for detection tasks of small defects and complex backgrounds.

[0056] The Cross-Layer Interaction Neck (CLI-Neck) realizes sufficient fusion of cross-scale features through a multi-scale feature encoder (MSFE) and a feature splicing module (FMS), breaking through the limitations of traditional FPN relying only on addition or splicing. At the same time, in combination with GSConv and VoV-GSCSP and other lightweight structures, the calculation overhead and parameter scale are greatly reduced on the premise of ensuring the sufficiency of feature expression and detection accuracy. This structure not only enhances the detection capability of the model for small targets and complex scenes, but also improves the feasibility and practical value of deployment on industrial edge devices.

[0057] High system integration, strong practicability: the improved model is deployed in the self-designed PCB defect detection system, integrating image import, intelligent analysis, visual result display, model project management, detection result export and local storage functions, improving the quality inspection efficiency and intelligent level, and having good engineering application prospect and industrialization value.

[0058] In summary, the present application is superior to the prior art in terms of detection accuracy, generalization ability, model lightweight and system practicability, solves the problems of low precision, small target difficult to identify and large model difficult to deploy in current PCB defect detection, has outstanding substantial characteristics, and has broad application prospect and industrialization value in the field of industrial quality inspection.

[0059] Example two As Figures 1-10 shown, the PCB defect detection system based on the improved RT-DETR proposed by the present application comprises, An image acquisition module is used to acquire a PCB image dataset and a PCB image to be detected, and the PCB image dataset comprises a plurality of PCB training images with or without defects. A data enhancement module is used to process the PCB image dataset by using a hybrid object migration HOM data enhancement strategy to generate enhanced training samples; a model construction and training module is used to construct an improved RT-DETR model and train the improved RT-DETR model by using the enhanced training samples to obtain a trained PCB defect detection model. A defect detection module is used to input the PCB image to be detected into the trained PCB defect detection model to obtain a PCB defect detection result by model inference. A result output and management module is used to output the defect detection result, and has the functions of model project management, detection result export and local saving, and the defect detection result comprises an image with defect bounding box and category label.

[0060] The model construction and training module comprises a network structure definition submodule, a loss function configuration submodule and a parameter optimization submodule; the network structure definition submodule is used to define the specific structure of HFF-Block in HybFFT-Net backbone network and MSFE, FMS and lightweight structure in CLI-Neck; the loss function configuration submodule is used to configure the combined loss function, cross entropy loss function+intersection over union loss function; and the parameter optimization submodule is used to iteratively optimize the model parameters by gradient descent algorithm based on the enhanced training samples.

[0061] The result output and management module comprises a visual interface unit, which is used for displaying a PCB image to be detected, a detection result image of a defect marking frame and a category label, and supports visual operations of image import, model selection, detection start and result query.

[0062] The embodiments of the present application are described in detail above with reference to the drawings, but the present application is not limited thereto, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

Claims

1. An improved RT-DETR-based PCB defect detection method, characterized in that: The application relates to a PCB defect detection method based on an improved RT-DETR model. S1, acquiring a PCB image dataset, adopting a hybrid object migration HOM data enhancement strategy to process the PCB image dataset, and generating an enhanced training sample; S2, constructing an improved RT-DETR model, the improved RT-DETR model comprising a hybrid fast Fourier transform network HybFFT-Net backbone network and a cross-layer interaction neck CLI-Neck; S3, training the improved RT-DETR model by using the enhanced training sample, optimizing model parameters, and obtaining a trained PCB defect detection model; S4, inputting a to-be-detected PCB image into the trained PCB defect detection model, and obtaining a PCB defect detection result through model reasoning, wherein the defect detection result comprises a defect position and a defect category.

2. The PCB defect detection method based on improved RT-DETR according to claim 1, wherein, In step S2, the HybFFT-Net backbone network is divided into five hierarchical stages, the feature resolution of each hierarchical stage is sequentially reduced to 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image resolution, and each hierarchical stage comprises a hybrid Fourier feature block HFF-Block; the HFF-Block comprises a hybrid Fourier mixer HF-Mixer and an adaptive convolution feedforward network AK-FFN; the CLI-Neck comprises a multi-scale feature encoder MSFE, a feature splicing module FMS and a lightweight structure.

3. The PCB defect detection method based on improved RT-DETR according to claim 2, characterized in that, In step S1, the specific process of the hybrid object migration HOM data enhancement strategy is as follows: a target region containing a defect is extracted from a first training image in the PCB image dataset, and a real pixel block of the target region is separated; The real pixel block is migrated and pasted to an arbitrary position of a second training image in the PCB image dataset, the second training image is different from the first training image; after the second training image after pasting is verified to be free of pixel conflicts, the second training image is used as an enhanced training sample.

4. The PCB defect detection method based on improved RT-DETR according to claim 3, characterized in that, The working process of the HF-Mixer is as follows: A1. receiving input features of a corresponding hierarchical stage in the HybFFT-Net backbone network, and expanding the channel number of the input features to twice the original channel number through convolution operation; A2. dividing the features after channel expansion into two groups along the channel dimension, and taking the two groups as local branch features and global branch features respectively; A3. processing the local branch features: splitting the local branch features into two groups of sub-channel features, respectively applying depth separable convolution with inflation rates of 1 and 2, and then splicing the two groups of processed sub-channel features in the channel dimension to obtain local enhanced features; A4. processing the global branch features: mapping the global branch features to the frequency domain through two-dimensional discrete Fourier transform (DFT) to obtain frequency domain features; adopting frequency domain position encoding (FPE) to fuse position information, separating the real part and the imaginary part of the frequency domain features and merging them in the channel dimension, and then learning the local neighborhood relationship in the frequency domain dimension through depth separable convolution; adopting frequency domain dynamic convolution (FDC) to generate an adaptive filter, and weighting the frequency domain features after position information fusion to highlight discriminative frequency components; and mapping the weighted frequency domain features back to the spatial domain through inverse Fourier transform to obtain global enhanced features; A5. The local enhancement features and the global enhancement features are spliced in the channel dimension, fused through a nonlinear activation function, and then compressed in channel number through 1x1 convolution to obtain the output features of the HF-Mixer.

5. The PCB defect detection method based on improved RT-DETR according to claim 4, characterized in that, The working process of the AK-FFN is as follows: B1. The output features of the HF-Mixer are received, and the channel number thereof is expanded to twice the original channel number through 1x1 convolution to obtain intermediate features; B2. The intermediate features are evenly divided into multiple sub-features along the channel dimension; B3. Depth separable convolution with different kernel sizes is applied to each sub-feature, including 3x3, 5x5, and 7x7; B4. All sub-features processed through depth separable convolution are spliced in the channel dimension, enhanced through a GELU activation function for nonlinear expression, and then compressed in channel number to the original channel number through 1x1 convolution to obtain the output features of the AK-FFN.

6. The PCB defect detection method based on improved RT-DETR according to claim 5, characterized in that, The working process of the MSFE is as follows: C1. The multi-scale feature maps output by different hierarchical stages in the HybFFT-Net backbone network are received, including a first-scale feature map, a second-scale feature map, and a third-scale feature map; C2. The first-scale feature map and the second-scale feature map are respectively subjected to convolution transformation, and their spatial sizes are adjusted through nearest neighbor interpolation to align with the spatial size of the third-scale feature map; C3. The three aligned scale feature maps are respectively subjected to dimension expansion to convert two-dimensional features into three-dimensional features; C4. The three three-dimensional features are spliced in the expanded dimension, and the spliced three-dimensional combined features are processed through 3D convolution to extract multi-scale correlation information; C5. The features processed through 3D convolution are subjected to normalization and maximum pooling operations, and then converted into two-dimensional feature maps through dimension compression to obtain multi-scale fusion features.

7. The PCB defect detection method based on improved RT-DETR according to claim 6, characterized in that, The working process of the FMS is as follows: D1. The multi-scale fusion features output by the MSFE and the different scale feature maps output by the HybFFT-Net backbone network are received to determine a large-scale feature map, a medium-scale feature map, and a small-scale feature map; D2. The large-scale feature map, the medium-scale feature map, and the small-scale feature map are respectively subjected to convolution operation to adjust the channel numbers thereof to the original channel number of the medium-scale feature map; D3. The large-scale feature map after channel adjustment is subjected to a hybrid downsampling structure of "maximum pooling + average pooling" to reduce its spatial dimension; the small-scale feature map after channel adjustment is subjected to nearest neighbor interpolation for upsampling to expand its spatial dimension, so that the spatial dimensions of the three are aligned with the medium-scale feature map; D4. The three feature maps after alignment in spatial dimension and channel dimension are spliced in the channel dimension to obtain the output features of the FMS.

8. The PCB defect detection method based on improved RT-DETR according to claim 7, characterized in that, The lightweight structure comprises a GSConv and a VoV-GSCSP module; a working process of the GSConv is as follows: input features are divided into main branch features and auxiliary branch features; the main branch features are adjusted from an input channel number c1 to half of an output channel number c2 through standard convolution (SC) to extract cross-channel relationships; the auxiliary branch features are extracted through deep separable convolution DSConv with a large convolution kernel to extract spatial features; the features processed by the main branch and the features processed by the auxiliary branch are fused, cross-channel information interaction is promoted through channel shuffle ChannelShuffle, and GSConv output features are obtained; a working process of the VoV-GSCSP module is as follows: input features are divided into first path features and second path features; the first path features are subjected to deep feature extraction through a GSBottleneck layer; the second path features are subjected to lightweight processing only through simple convolution operation; the features processed by the two paths are spliced in a channel dimension, integrated and optimized through a final convolution layer, and VoV-GSCSP module output features are obtained.

9. An improved RT-DETR based PCB defect detection system for implementing the improved RT-DETR based PCB defect detection method of any one of claims 1-8. Comprise, An image acquisition module is configured to acquire a PCB image dataset and a PCB image to be detected, and the PCB image dataset comprises a plurality of PCB training images with defects or without defects; A data enhancement module is configured to process the PCB image dataset by using a hybrid object migration (HOM) data enhancement strategy to generate enhanced training samples; A model construction and training module is configured to construct an improved RT-DETR model, train the improved RT-DETR model by using the enhanced training samples, and obtain a trained PCB defect detection model; A defect detection module is configured to input the PCB image to be detected into the trained PCB defect detection model, and obtain a PCB defect detection result by model inference; A result output and management module is configured to output the defect detection result, and has functions of model project management, detection result export and local saving, wherein the defect detection result comprises an image with a defect bounding box and a class label.

10. The PCB defect detection system based on improved RT-DETR according to claim 9, wherein, The model construction and training module comprises a network structure definition submodule, a loss function configuration submodule and a parameter optimization submodule; the network structure definition submodule is configured to define specific structures of HFF-Block in a HybFFT-Net backbone network, MSFE, FMS and a lightweight structure in a CLI-Neck; the loss function configuration submodule is configured to configure a combined loss function: a cross-entropy loss function + an intersection over union loss function; and the parameter optimization submodule is configured to iteratively optimize model parameters by using a gradient descent algorithm based on the enhanced training samples.

Images