PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy

Abstract

The present application relates to small target detection technology, specifically to a PCB surface small target defect detection method based on multi-branch feature extraction and adaptive attention fusion strategy, acquires PCB surface defect dataset, constructs PCB small target defect detection network, introduces C3k2-PPA module in backbone network, relies on local patch feature enhancement mechanism, realizes efficient feature extraction through internal parallel patch perception attention module, adopts multi-branch strategy of local branch + global branch + serial convolution branch, realizes accurate interaction of local details and global context features through patch parameter regulation, reduces calculation redundancy; at the same time, attention mechanism is introduced after multi-branch feature extraction, and key features are adaptively enhanced. The present application significantly improves the precision of PCB surface small target defect detection, effectively reduces the missed detection and false detection, realizes efficient identification of small target defects, and adapts to the actual needs of PCB quality detection in industrial scene.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategies. Background Technology

[0002] With continuous technological advancements, printed circuit boards (PCBs), as a crucial component of electronic devices, directly impact the quality of the entire product. Due to their high integration and complex wiring, PCBs are difficult to manufacture and prone to defects during mass production, thus affecting equipment performance and lifespan. Therefore, PCB surface defect detection is vital in the electronics manufacturing industry, playing a key role in ensuring quality, controlling costs, improving efficiency, and ensuring stable equipment operation. However, in actual inspection processes, the challenge lies in effectively improving the accuracy of detecting small-target defects.

[0003] Traditional PCB defect detection technologies include manual visual inspection and automated inspection. The former relies on subjective human judgment, resulting in low efficiency, large detection errors, and unsuitability for large-scale manufacturing. The latter offers higher detection accuracy but suffers from lower efficiency, expensive equipment, and stringent environmental requirements, limiting its practical application. In recent years, with the rapid development of deep learning, the performance of object detection models has significantly improved. Currently, mainstream object detection algorithms can be broadly categorized into two-stage and one-stage algorithms. Two-stage algorithms typically generate candidate regions first, then classify and regress these regions, improving detection accuracy but sacrificing some real-time performance; examples include Fast R-CNN, Faster R-CNN, and Mask R-CNN network models. Single-stage object detection algorithms treat object detection as a holistic regression problem, directly outputting object category and location information, offering faster speed but potentially lower accuracy compared to two-stage algorithms; examples include the YOLO network series.

[0004] For PCB defect detection, the irregular and minute shapes of defects require the target detection system to not only possess high accuracy and real-time performance but also adaptability to targets of different scales. Current general target detection methods have significant limitations: multi-level downsampling operations lead to severe loss of features of minute defects, and the gradient response of the standard loss function to targets of different scales is unbalanced, ultimately resulting in missed detections and false detections of small target defects. These technical pain points directly restrict the practical deployment efficiency of high-precision PCB defect detection systems.

[0005] Chinese patent application CN118014943A discloses a method for detecting and identifying PCB appearance defects. The backbone feature extraction network has five feature extraction layers, each with a complex structure. For example, the Information Enhancement Perception Module (EPM) involves multiple convolution operations and feature fusion steps. The Multi-Channel Feature Fusion Network (MCFFN) contains multiple modules, and the data processing flow between modules is cumbersome. The model is highly complex, which may result in insufficient adaptability to environmental variables. The solution also fails to fully explain its advantage of being able to adapt to different production environments.

[0006] Chinese patent application CN118644465A discloses a PCB surface defect detection method, system, and storage medium. However, by simply adding a 160x160 pixel detection layer to cover defects larger than 4×4 pixels, dynamic convolution cannot accurately capture minute details, easily missing defects such as short circuits and burrs, and failing to capture small target features sufficiently. Upgrading from F4 to F19 requires more than 10 steps and repeatedly uses three C2f modules. Although the computational load after pruning still reaches 13.4 GFLOPs (Giga Floating Point Operations Per Second), there is more redundant computation at the same precision. Real-time performance is prone to fluctuations on low- and mid-range devices. Furthermore, no comparison with the previous version was provided to demonstrate the improvements in detection accuracy, missed detections, and false detections.

[0007] Chinese patent application CN120182263A discloses a defect detection system, detection method, and model training method. However, the coverage of the detected defect types is insufficient, focusing only on two types of defects: "dirt" and "scratches." It does not mention detection schemes for complex defects commonly found in wafer manufacturing, such as "cracks, discoloration, and mesh contamination." Furthermore, scratches are simply classified into two categories based on "length threshold" and size, without considering key dimensions such as scratch depth and shape. In practical applications, complex defects are easily missed. In addition, the training data requires cropping large wafer images using OpenCV software and then annotating them with polygons using Labelme software. It also requires annotating all visible defects. Wafer defects are small in size, making annotation time-consuming and requiring a high level of expertise from the annotators. In small sample scenarios, the model performance may drop sharply. Summary of the Invention

[0008] Purpose of the Invention: The purpose of this invention is to address the shortcomings of existing technologies and provide a PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategies.

[0009] To address the problems of low efficiency and large errors in manual inspection of small target defects on PCB surfaces, high cost and poor adaptability of traditional automatic inspection equipment, and loss of small target defect features due to multi-level downsampling in general target detection algorithms, which makes it difficult to balance detection accuracy and real-time performance, this invention aims to provide a method for detecting small target defects on PCB surfaces based on multi-branch feature extraction and adaptive attention fusion strategy. Ultimately, this method aims to improve the detection accuracy of small target defects, reduce the false negative and false positive rates, and balance detection real-time performance, thus meeting the actual needs of PCB quality inspection in industrial scenarios for stability, reliability, and efficiency.

[0010] Technical solution: The present invention provides a PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy, comprising the following steps:

[0011] Step 1: Construct a dataset consisting of PCB surface defects and perform preprocessing operations. Take PCB images containing different defect types, classify and filter the PCB images, manually label the filtered PCB images to form a PCB surface defect dataset, and divide it into training set, validation set and test set. Verify that the number of images and labels are consistent to ensure that the data specifications are uniform.

[0012] Step 2: Construct a PCB surface small target defect detection network based on multi-branch feature extraction and feature fusion attention strategy. The backbone network of the PCB surface small target defect detection network consists of one convolutional layer and four identical C3k2-PPA modules; original defect image. The feature map is processed sequentially through convolutional layers and four C3k2-PPA modules. Enter the next level. ;

[0013] In each layer of the C3k2-PPA module, the input feature map First pass through one The convolutional layer performs preliminary feature extraction. The initial feature map output from the convolutional layer is split into n branch feature maps of the same size by the splitting module. For the feature maps of the last n-1 branches, the features from the previous layer are respectively... With current layer features To merge, And each branch feature map Both methods employ a parallelized patch-aware attention module for multi-branch feature extraction and feature fusion attention mechanism to obtain features. Finally, the characteristics of all branches Connect, and through a The convolutional layer adjusts its dimensions and outputs the corresponding feature map of the current layer. ;

[0014] First branch feature map The feature maps are continuously refined through parallel overlay with other branches; Figure 3 It can be seen that regardless of whether C3k=False / True, the first branch feature map does not go through PPA / C3k-PPA, but is combined with other feature maps processed by the PPA / C3k-PPA branch, and finally passes through a... The convolutional layer adjusts its dimensions and outputs the corresponding feature map of the current C3k2-PPA layer. When performing local feature extraction, each PPA module introduces multi-branch feature extraction to capture multi-scale features of defects, and after feature extraction, it uses a series of efficient channel attention and spatial attention mechanisms to perform adaptive feature enhancement to achieve feature aggregation.

[0015] Step 3: Feature map output by the last layer C3k2-PPA module First, the feature is aggregated at multiple scales by the fast spatial pyramid pooling module SPPF. Then, it is processed by the attention-based feature extraction module C2PSA and then sequentially input into the neck network and the detection head network. Feature fusion is achieved through the neck network and the detection head network to realize the detection and classification of small target defects on the PCB surface.

[0016] Step 4: Use the training set constructed in Step 1 as input to train the PCB surface small target defect detection network constructed in Step 2; after each training round, calculate the mean accuracy mAP of the detection network using the validation set; after training, select the network weight corresponding to the highest mAP value during training as the optimal weight of the detection network to ensure optimal detection performance in subsequent rounds.

[0017] Step 5: After preprocessing the PCB surface image to be detected according to the standard in Step 1, input it into the PCB surface small target defect detection network with optimal weights. After a series of processing steps such as multi-branch feature extraction module, the final complete defect detection result is output. The detection result includes defect category, defect location and confidence level.

[0018] The backbone network of the PCB surface small target defect detection network of this invention improves upon the feature extraction module (C3k2) in the original YOLOv11n backbone network. It deeply integrates the parallelized patch-aware attention module (PPA) with the standard C3k2 module at multiple levels, forming a new feature extraction module, C3k2-PPA. The C3k2-PPA module contains n PPA modules connected in series, switching according to the parameter C3k. Specifically: when the parameter C3k of the feature extraction module C3k2-PPA is False, each branch uses one parallelized patch-aware attention module (PPA); when the parameter C3k of C3k2-PPA is True, each branch uses one C3k2-PPA module. This approach is more powerful and refined, suitable for applications requiring extremely high detection accuracy.

[0019] Furthermore, the detailed processing procedure of step 1 is as follows:

[0020] Step 1.1: Determine the PCB defect image acquisition range and basic parameters, covering PCBs with different defect types. Acquire images including missing holes, mouse bites, open circuits, short circuits, stray wires, and spurious copper. The initial acquisition quantity for each type of defect should be no less than 400 images. Use image processing tools to uniformly adjust the resolution of all acquired images to 640×640 pixels to eliminate the interference of resolution differences on subsequent model training.

[0021] Step 1.2: Filter the collected PCB images, remove low-quality images, ensure that the filtered PCB images contain surface defect targets, and verify the size of the defect targets to ensure that the pixel ratio of each defect in the image is between 0.5% and 5%, which meets the requirements of small target detection scenarios, and finally form an initial effective image set.

[0022] Step 1.3: Establish a lowercase English defect label library and assign unique indexes. Use Labelimg software to label defects according to the principle of rectangular boxes closely following the defect edges to ensure that the defects occupy a reasonable proportion within the box. Automatically generate a txt label file with the same name as the image. Verify that the file content is the category index + normalized coordinates / size (all values ​​are in the range of [0,1]) to ensure that it conforms to the YOLO format.

[0023] Step 1.4: Divide the labeled YOLO format dataset into training, validation and test sets in a 7:2:1 ratio. Use a hierarchical randomization strategy to ensure that the proportion of each type of defect in the three subsets is consistent with the total dataset, and avoid model training bias due to uneven class distribution.

[0024] Step 1.5: Create the root directory of PCB_Defect_Dataset, with three subdirectories: "train / val / test". Each subdirectory is further divided into "images" and "labels" to store JPG images and TXT labels. Use a script to verify that the number of "images" and "labels" files in the same subset is consistent to ensure that the images and labels correspond accurately when called by the network.

[0025] Furthermore, in the PCB surface small target defect detection network, during multi-branch feature extraction, the input features in each parallel patch-aware attention module (PPA) of the C3k2-PPA module are divided into three branches after pointwise convolution: a local branch (patch parameter = 2), a global branch (patch parameter = 4), and a serial convolution branch (3 concatenated convolutions). During extraction, the 1 convolutional layer and 4 C3k2-PPA modules in the backbone network are defined as 5 feature extraction layers, denoted as extraction layer E1, extraction layer E2, extraction layer E3, extraction layer E4, and extraction layer E5, respectively. These 5 extraction layers gradually complete the high-dimensional feature extraction of the PCB defect image; the original defect image... The image is a three-channel color image, and its size must be consistent with the resolution (640×640 pixels) after preprocessing in step 1 to ensure uniform input specifications for the feature extraction process; the feature maps extracted by the five extraction layers are denoted as follows. , ;

[0026] Represents the channel dimension of the i-th layer. H and W represent the height and width of the image, respectively. The C, H, and W parameters of each feature map are dynamically adjusted as the extraction layer depth increases to achieve feature refinement from shallow to deep layers.

[0027] Furthermore, the feature maps output by each layer of the C3k2-PPA module... The expression is as follows:

[0028] ;

[0029] in, This represents the feature map of the first branch in the n branches of the i-th layer. This represents the feature map of the j-th branch in the n-th branch of the i-th layer. , This represents the output feature map of the i-th layer C3k2-PPA module. .

[0030] The C3k2-PPA module is obtained by multi-level deep fusion of the feature extraction module C3k2 and the parallelized patch-aware attention module PPA. In the above formula... This refers to the attention weights generated by the parallelized patch-aware attention module (PPA), which are used to weight and adjust the features. Each parallelized PPA performs multi-branch feature extraction and feature fusion attention operations to obtain the feature map. The process is as follows:

[0031] Step 1) For a given tensor Then, it is first adjusted through pointwise convolution to obtain ;

[0032] Step 2) Output the serial convolution feature map by entering the serial convolution branch. ;

[0033] By controlling the patch size parameter p=2, the local branch is entered, and the local feature map is output. ;

[0034] By controlling the patch size parameter p=4, the system enters the global branch and outputs the global feature map. ;

[0035] Step 3) Add the feature maps output from the sequential convolutional branch, local branch, and global branch, and output the summed feature map. ;

[0036] ;

[0037] Step 4) sequentially from one-dimensional channel attention and 2D spatial attention map Perform adaptive feature enhancement processing, and finally output ;

[0038] ;

[0039] ;

[0040] ;

[0041] in, This indicates element-wise multiplication. and This represents the feature map after channel and spatial selection. and These represent the rectified linear unit (ReLU) and batch normalization (BN), respectively. This indicates the final output of the PPA; It is generated The key intermediate features are obtained in each layer of the C3k2-PPA module. After preliminary processing and branch splitting of the input feature map, each branch will obtain... After that, all branches After concatenation and dimensionality adjustment, a more comprehensive feature representation is finally obtained. .

[0042] By controlling the patch size parameter, local and global features are accurately extracted to meet the detection requirements of small target defects. In the form of parallel multi-branch (serial convolution branch, local and global branches with different patch sizes), multi-dimensional and multi-scale feature extraction and fusion attention operation are realized at the same time, which is specifically used for the detection of small target defects on PCB surface.

[0043] Local branch feature map Global branch feature map and serial convolution branch feature maps The specific extraction process is as follows:

[0044] First of all, Perform efficient block processing, converting it into a spatially contiguous block through an unfolding operation. Size patches, formed after dimensional reorganization The structure is then compressed using mean pooling along the channel dimension to obtain patch features. ;

[0045] ;

[0046] ;

[0047] in, This is a block operation function. This is the feature map in the PPA module after pointwise convolution adjustment. For dimensional reshaping operations, This is the patch size parameter. These are the patch features after mean pooling along the channel dimension. The channel index has a value range of [1, C].

[0048] Secondly, a linear transformation is applied to the block features using a feedforward network (FFN), which... 3D feature projection to 3D space, to obtain output features , for features Generate the spatial probability distribution matrix using the Softmax activation function. This reflects the importance weight of features at different spatial locations, enabling preliminary screening and enhancement of features; ;

[0049] in, is the spatial probability distribution matrix, reflecting the feature importance weights at different spatial locations; Softmax is the activation function, used to normalize the input vector into a probability distribution, satisfying that the sum of all elements is 1; z is the output feature of the feedforward network FFN after linearly transforming the block features; d is the dimension of feature z, used to scale feature z to stabilize Softmax calculation.

[0050] Furthermore, a task-aware mechanism is introduced, utilizing the task embedding vector ξ and the projection matrix. The tags are dynamically weighted, and each tag is calculated using a cosine similarity function. The relevance to the task is then determined through matrix multiplication. Weighting, generating reweighted features ;

[0051] ;

[0052] ;

[0053] in, It is a cosine similarity function with boundaries in [0,1]. The i-th label before weighting. Embed vectors for tasks. Let be the projection matrix. This is the i-th weighted label;

[0054] Then, channel selection and spatial reorganization are performed on the weighted features. Channel dimensions are filtered through linear transformation of the P matrix, and the original spatial dimensions are restored through reshaping and interpolation operations. Finally, the local features are output. and global features ;

[0055] Finally, a sequential convolutional branch is constructed to extract fused features: a sequential convolution consisting of three 3x3 convolutions is used to replace the traditional 7x7, 5x5, and 3x3 convolutional layers, thereby extracting features layer by layer and gradually fusing local and contextual information to form a sequential convolutional branch feature map that contains both local details and global contextual information. ;

[0056] ;

[0057] in, The output feature map of the serial convolution branch. This is a 3×3 convolution operation.

[0058] Beneficial effects: Compared with existing technical solutions, the present invention has the following advantages:

[0059] 1. Enhanced small-target defect feature capture and improved detection accuracy: This invention introduces the C3k2-PPA module into the backbone network. Through the multi-branch feature extraction (local branch, global branch, and serial convolutional branch) and adaptive attention fusion strategy of the PPA module, it effectively alleviates the problem of small-target defect feature loss caused by multi-level downsampling in traditional detection models. Specifically, the local branch (p=2) accurately captures local features such as defect edges and details, while the global branch (p=4) obtains defect context information. Combined with channel and spatial attention, key features are dynamically weighted, significantly improving the recognition accuracy of small-target defects (such as micro open circuits and short circuits) in complex backgrounds and reducing the false negative and false positive rates.

[0060] 2. Balancing detection performance and generalization ability, adapting to real-world production scenarios: This invention utilizes a lightweight convolutional design (three 3×3 convolutions replacing traditional large-size convolutions) and a dynamic feature fusion mechanism to control the number of model parameters and computational load while ensuring detection accuracy, meeting the real-time detection needs of industrial scenarios. Furthermore, through multi-branch feature interaction and task-aware weighting, the model can adaptively adapt to real-world production variables such as different lighting conditions and PCB board material differences, maintaining stable detection performance (mAP0.5 reaches 93.4%, superior to mainstream models such as YOLOv11n) in diverse scenarios, and exhibiting stronger generalization ability.

[0061] 3. Optimize feature utilization efficiency and reduce labeling dependence: This invention enhances the extraction and utilization of effective features through operations such as block average pooling and spatial probability distribution filtering, reducing dependence on large-scale labeled data. Simultaneously, the dataset built based on the YOLO format and the standardized training process can be quickly transferred to different PCB defect detection tasks, reducing adaptation costs in practical applications and facilitating industrial deployment. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating the overall process of this invention.

[0063] Figure 2 This is a schematic diagram of the network structure of the present invention;

[0064] Figure 3 This is a schematic diagram of the C3K2-PPA module in an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of the PPA module in an embodiment of the present invention;

[0066] Figure 5 The following is a screenshot showing the detection results using the existing YOLOv11n model as an example.

[0067] Figure 6 This is a schematic diagram illustrating the inspection effect of the technical solution of the present invention in an embodiment. Detailed Implementation

[0068] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0069] like Figure 1 and Figure 2 As shown, the PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy of the present invention includes the following steps:

[0070] Step 1: Obtain PCB images containing different defect types, classify and filter the PCB images, manually annotate the filtered images to form a PCB surface defect dataset, and divide it into training set, validation set and test set;

[0071] Step 2: Construct a PCB surface small target defect detection network based on multi-branch feature extraction and feature fusion attention strategy. The backbone network of the PCB surface small target defect detection network consists of one convolutional layer and four identical C3k2-PPA modules; original defect image. The feature map is processed sequentially through convolutional layers and four C3k2-PPA modules. Enter the next level. ;

[0072] In each layer of the C3k2-PPA module, the input feature map First pass through one The convolutional layer performs preliminary feature extraction. The initial feature map output from the convolutional layer is split into n branch feature maps of the same size by the splitting module. For the feature maps of the last n-1 branches, the features from the previous layer are respectively... With current layer features To merge, And each branch feature map Both methods employ a parallelized patch-aware attention module for multi-branch feature extraction and feature fusion attention mechanism to obtain features. Finally, the characteristics of all branches Connect, and through a The convolutional layer adjusts its dimensions and outputs the corresponding feature map of the current layer. ;

[0073] Step 3: Feature map output by the last layer C3k2-PPA module First, the feature is aggregated at multiple scales by the fast spatial pyramid pooling module SPPF. Then, it is processed by the attention-based feature extraction module C2PSA and then sequentially input into the neck network and the detection head network. Feature fusion is achieved through the neck network and the detection head network to realize the detection and classification of small target defects on the PCB surface.

[0074] Step 4: Use the training set constructed in Step 1 as input to train the PCB surface small target defect detection network constructed in Step 2; after each training round, calculate the mean accuracy mAP of the detection network using the validation set; after training, select the network weight corresponding to the highest mAP value during training as the optimal network weight.

[0075] Step 5: After preprocessing the PCB surface image to be inspected according to the standard in Step 1, input it into the PCB surface small target defect detection network with optimal weights, and output complete defect detection results, including defect category, defect location and confidence level.

[0076] This invention enhances the capture of small target defect features and improves detection accuracy. It primarily utilizes a multi-branch (local branch, global branch, and serial convolutional branch) feature extraction strategy and an adaptive attention fusion strategy within the C3k2-PPA module to more effectively mitigate the loss of small target defect features caused by multi-level downsampling. In complex contexts, the local branch accurately captures defect edges and details, while the global branch acquires contextual information. Combined with channel and spatial attention, key features are dynamically weighted, significantly improving the accuracy of identifying minute open circuits, short circuits, and other defects, while reducing false negatives and false positives. For example, in practical testing on the PKU-Market-PCB public dataset, this invention achieved an mAP of 93.4% with a recall rate of 3.1 percentage points, a 3 percentage point improvement over YOLOv11n.

[0077] The detailed processing procedure for step 1 of this embodiment is as follows:

[0078] Step 1.1: Collect PCB images covering different defect types, including images of vias, rodent bites, open circuits, short circuits, stray copper, and stray copper. The initial number of images collected for each type of defect should not be less than 400.

[0079] Step 1.2: Filter the collected PCB images to ensure that the filtered PCB images contain surface defect targets, and verify the size of the defect targets to ensure that the pixel ratio of each defect target in the image is between 0.5% and 5%, and finally form an initial set of valid images.

[0080] Step 1.3: Establish a lowercase English defect label library and assign unique indexes. Label the defects according to the principle of the rectangle being close to the edge of the defect to ensure that the defect accounts for a reasonable proportion within the rectangle. Automatically generate a txt label file with the same name as the image. Verify that the content of the file is the category index + normalized coordinates / size to ensure that it conforms to the YOLO format.

[0081] Step 1.4: Divide the labeled YOLO format dataset into training set, validation set and test set in a ratio of 7:2:1. Use a hierarchical randomization strategy to ensure that the proportion of each type of defect in the three subsets is consistent with the total dataset.

[0082] Step 1.5: Create the root directory PCB_Defect_Dataset, with three subdirectories: train, val, and test. Each subdirectory is further divided into images and labels to store JPG images and corresponding TXT labels. Use a script to verify that the number of images and labels files in the same subset is consistent.

[0083] In this embodiment, when performing multi-branch feature extraction, the 1 convolutional layer and 4 C3k2-PPA modules in the backbone network are first defined as 5 feature extraction layers, namely extraction layer E1, extraction layer E2, extraction layer E3, extraction layer E4 and extraction layer E5. Through these 5 extraction layers, the high-dimensional feature extraction of the PCB defect image is gradually completed.

[0084] Then input the original defect image. For a three-channel color image, the above five extraction layers are used to extract the output feature map. , ;

[0085] Represents the channel dimension of the i-th layer. H and W represent the height and width of the image, respectively. The C, H, and W parameters of each feature map are dynamically adjusted as the extraction layer depth increases to achieve feature refinement from shallow to deep layers.

[0086] like Figure 3 As shown, in the backbone network of the PCB surface small target defect detection network in this embodiment, each C3k2-PPA module has a parallelized patch perception attention module (PPA). The feature map input to the PPA module is convolved pointwise and then divided into three branches for feature extraction: local branch, global branch, and serial convolution branch. The feature map extracted by each C3k2-PPA module is... The expression is as follows:

[0087] ;

[0088] in, This represents the feature map of the first branch in the n branches of the i-th layer. This represents the feature map of the j-th branch in the n-th branch of the i-th layer. , This represents the output feature map of the i-th layer C3k2-PPA module. ; This refers to the attention weights generated by the parallelized patch-aware attention module (PPA). Each parallelized PPA performs multi-branch feature extraction and feature fusion attention operations to obtain the feature map. The process is as follows:

[0089] Step 1) For a given tensor Then, it is first adjusted through pointwise convolution to obtain ;

[0090] Step 2) Output the serial convolution feature map by entering the serial convolution branch. ;

[0091] By controlling the patch size parameter p=2, the local branch is entered, and the local feature map is output. ;

[0092] By controlling the patch size parameter p=4, the system enters the global branch and outputs the global feature map. ;

[0093] Step 3) Add the feature maps output from the sequential convolutional branch, local branch, and global branch, and output the summed feature map. ;

[0094] ;

[0095] Step 4) sequentially from one-dimensional channel attention and 2D spatial attention map Perform adaptive feature enhancement processing, and finally output ;

[0096] ;

[0097] ;

[0098] ;

[0099] in, This indicates element-wise multiplication. and The feature map represents the result of channel selection and spatial selection. and Representing the rectified linear unit ReLU and batch normalized BN, respectively. This indicates the final output characteristics of the PPA module.

[0100] like Figure 4 As shown, in step 4 of this embodiment, the local branch feature map is extracted. Global branch feature map and serial convolution branch feature maps The specific process is as follows:

[0101] First of all, Perform efficient block processing, converting it into a spatially contiguous block through an unfolding operation. Size patches, formed after dimensional reorganization The structure is then compressed using mean pooling along the channel dimension to obtain patch features. ;

[0102] ;

[0103] ;

[0104] in, This is a block operation function. This is the feature map in the PPA module after pointwise convolution adjustment. For dimensional reshaping operations, This is the patch size parameter. These are the patch features after mean pooling along the channel dimension. The channel index has a value range of [1, C].

[0105] Then, a linear transformation is applied to the block features using a feedforward network (FFN) to transform them. 3D feature projection to 3D space, to obtain output features , for features Generate the spatial probability distribution matrix using the Softmax activation function. ;

[0106] ;

[0107] in, is the spatial probability distribution matrix; Softmax is the activation function; d is the dimension of feature z;

[0108] Next, a task-aware mechanism is introduced, utilizing the task embedding vector ξ and the projection matrix. The tags are dynamically weighted, and each tag is calculated using a cosine similarity function. The relevance to the task is then determined through matrix multiplication. Weighting, generating reweighted features ;

[0109] ;

[0110] ;

[0111] in, It is a cosine similarity function with boundaries in [0,1]. The i-th label before weighting. Embed vectors for tasks. Let be the projection matrix. This is the i-th weighted label;

[0112] Next, channel selection and spatial reorganization are performed on the weighted features. Channel dimensions are filtered through linear transformation of the P matrix, and the original spatial dimensions are restored through reshaping and interpolation operations, ultimately outputting the local features. and global features ;

[0113] Finally, a sequential convolutional branch is constructed to extract fused features. This involves using a sequence of three 3x3 convolutions to gradually fuse local and contextual information, forming a sequential convolutional branch feature map that contains both local details and global contextual information. ;

[0114] ;

[0115] in, The output feature map of the serial convolution branch. This is a 3×3 convolution operation.

[0116] This invention balances detection performance and generalization ability, adapting to real-world production scenarios: Its lightweight convolutional design (three 3×3 convolutions replacing traditional large-size convolutions) and dynamic feature fusion mechanism control the number of model parameters and computational load while ensuring detection accuracy, meeting the real-time detection needs of industrial scenarios. Through multi-branch feature interaction and task-aware weighting, it can adaptively adapt to different lighting conditions, PCB material variations, and other real-world production variables, maintaining stable detection performance and stronger generalization ability in diverse scenarios.

[0117] To further verify the effectiveness of the technical solution of this invention, this embodiment selects a corresponding deep learning model and trains it using stochastic gradient descent with a learning rate of 0.01, a batch size of 8, an epoch of 200, and a weight decay of 0.0005. After completing the model training, the detection model metrics are obtained. The optimal detection model is selected based on performance evaluation metrics such as accuracy, recall, mAP0.5, and mAP0.5:0.95. Then, based on the optimal detection model selected from the performance evaluation metrics, a target detection head is used to detect defects on the PCB surface.

[0118] Precision and recall are primarily used in experiments to measure network performance, and their expressions are:

[0119] ;

[0120] ;

[0121] Where TP (True Positives) represents the number of correctly predicted positive samples, FP (False Positives) represents the number of incorrectly predicted positive samples, and FN (False Negatives) represents the number of positive samples incorrectly predicted as negative samples. Average Precision (AP) is the area under the precision-recall curve; the closer the AP value is to 1, the better the algorithm's detection performance. The expression is: ;

[0122] Mean precision (mAP) is the average AP across all classes, providing a balanced evaluation by combining precision and recall. mAP is particularly important in multi-class detection tasks because it ensures good performance across all classes. Furthermore, mAP is robust to class imbalance and is therefore widely used to evaluate multi-class object detection tasks, making it a key metric for evaluating detection accuracy in our experiments. The expression is:

[0123] ;

[0124] The experimental environment in this embodiment is based on the PyTorch deep learning framework and runs on the Windows operating system. An NVIDIA GeForce RTX 4090 GPU is used for computational acceleration to complete model training and obtain the detection model metrics. Experimental environment parameters: initial learning rate 0.01, training epochs 200, SGD optimizer, momentum coefficient 0.937, weight decay 0.0005, and input image resolution 640x640. In one embodiment of this invention, to verify the effectiveness of the method, the public dataset PKU-Market-PCB is used for experimental verification. The experimental data for the two schemes are shown in Table 1.

[0125] Table 1

[0126] method GFLOPs / G Parameters / M P / % R / % mAP 0.5 / % mAP 0.5: 0.95 / % YOLOv11n 6.3 2.58 91.7 85.2 90.4 45.9 Technical solution of the present invention 7 2.77 93.5 88.3 93.4 49

[0127] In the field testing and verification on the PKU-Market-PCB public dataset, the detection performance of the technical solution of this invention was compared with that of the mainstream YOLOv11n model (e.g., Figure 5 As shown, the following conclusions can be drawn:

[0128] From a qualitative perspective, the YOLOv11n model ( Figure 5The left column (referring to the method used in the left column) has significant limitations in detecting small target defects on PCB surfaces: insufficient ability to capture the features of minute defects, resulting in local missed detections and false detections, and low confidence levels for some defects; while the technical solution of this invention (…) Figure 5 In the right column, the red rectangular area represents a significant improvement in detection accuracy compared to the comparison model, and the red elliptical area represents defects that were missed or falsely detected by the comparison model, all of which were successfully identified by this invention. Through the multi-branch feature fusion and attention enhancement mechanism of the C3k2-PPA module, not only are all target defects accurately identified, but the defect confidence is also significantly improved. The positioning accuracy and category discrimination accuracy of small-sized defects such as short circuits and strays are better than those of the comparison model, and the problems of missed detection and false detection are effectively solved.

[0129] From a quantitative and scenario adaptability perspective, even in scenarios with dense PCB wiring and complex background textures, this invention can still stably extract key features of small target defects, achieving accurate detection and classification. The invention achieves an mAP of 93.4% and a recall of 88.3%, representing improvements of 3 percentage points and 3.1 percentage points respectively compared to YOLOv11n. This further verifies that in small target defect detection tasks, this invention overcomes the feature loss problem caused by downsampling in traditional models while maintaining detection efficiency, demonstrating its ability to adapt to the actual needs of industrial PCB defect detection.

Claims

1. A PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy, characterized in that, Includes the following steps: Step 1: Obtain PCB images containing different defect types, classify and filter the PCB images, manually annotate the filtered images to form a PCB surface defect dataset, and divide it into training set, validation set and test set; Step 2: Construct a PCB surface small target defect detection network based on multi-branch feature extraction and feature fusion attention strategy. The backbone network of the PCB surface small target defect detection network consists of one convolutional layer and four C3k2-PPA modules with the same structure. Original defect image The feature map is processed sequentially through convolutional layers and four C3k2-PPA modules. Enter the next level. ; In each layer of the C3k2-PPA module, the input feature map First pass through one The convolutional layer performs preliminary feature extraction. The initial feature map output from the convolutional layer is split into n branch feature maps of the same size by the splitting module. For the feature maps of the last n-1 branches, the features from the previous layer are respectively... With current layer features To merge, And each branch feature map Both methods employ a parallelized patch-aware attention module for multi-branch feature extraction and feature fusion attention mechanism to obtain features. Finally, the characteristics of all branches Connect, and through a The convolutional layer adjusts its dimensions and outputs the corresponding feature map of the current layer. ; Step 3: Feature map output by the last layer C3k2-PPA module First, the feature is aggregated at multiple scales by the fast spatial pyramid pooling module SPPF. Then, it is processed by the attention-based feature extraction module C2PSA and then sequentially input into the neck network and the detection head network. Feature fusion is achieved through the neck network and the detection head network to realize the detection and classification of small target defects on the PCB surface. Step 4: Use the training set constructed in Step 1 as input to train the PCB surface small target defect detection network constructed in Step 2; after each round of training, calculate the mean accuracy mAP of the detection network using the validation set; After training is completed, the network weights corresponding to the highest mAP values ​​during training are selected as the optimal network weights. Step 5: After preprocessing the PCB surface image to be detected according to the standard in Step 1, input it into the PCB surface small target defect detection network with optimal weights, and output complete defect detection results, including defect category, defect location and confidence level. Each of the C3k2-PPA modules mentioned above includes a parallelized patch-aware attention module (PPA). In the PPA, the input feature map is convolved pointwise and then divided into three branches for feature extraction: a local branch, a global branch, and a sequential convolution branch. The local branch feature map is extracted first. Global branch feature map and serial convolution branch feature maps The specific process is as follows: First of all, Perform efficient block processing, converting it into a spatially contiguous block through an unfolding operation. Size patches, formed after dimensional reorganization The structure is then compressed using mean pooling along the channel dimension to obtain patch features. ; ； ； ； in, This is a block operation function. This is the feature map in the PPA module after pointwise convolution adjustment. For dimensional reshaping operations, This is the patch size parameter. These are the patch features after mean pooling along the channel dimension. The channel index has a value range of [1, C]. Then, a linear transformation is applied to the block features using a feedforward network (FFN) to transform them. 3D feature projection to 3D space, to obtain output features , for features Generate the spatial probability distribution matrix using the Softmax activation function. ; ； in, is the spatial probability distribution matrix; Softmax is the activation function; d is the dimension of feature z; Next, a task-aware mechanism is introduced, utilizing the task embedding vector ξ and the projection matrix. The tags are dynamically weighted, and each tag is calculated using a cosine similarity function. The relevance to the task is then determined through matrix multiplication. Weighting, generating reweighted features ; ； ； in, It is a cosine similarity function with boundaries in [0,1]. The i-th label before weighting. Embed vectors for tasks. For the projection matrix, This is the i-th weighted label; Next, channel selection and spatial reorganization are performed on the weighted features. Channel dimensions are filtered through linear transformation of the P matrix, and the original spatial dimensions are restored through reshaping and interpolation operations, ultimately outputting the local features. and global features ; Finally, a sequential convolutional branch is constructed to extract fused features. This involves using a sequence of three 3x3 convolutions to gradually fuse local and contextual information, forming a sequential convolutional branch feature map that contains both local details and global contextual information. ; ； in, The output feature map of the serial convolution branch. This is a 3×3 convolution operation.

2. The PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy according to claim 1, characterized in that, The detailed processing procedure for step 1 is as follows: Step 1.1: Collect PCB images covering different defect types, including images of vias, rodent bites, open circuits, short circuits, stray copper, and stray copper. The initial number of images collected for each type of defect should not be less than 400. Step 1.2: Filter the collected PCB images to ensure that the filtered PCB images contain surface defect targets, and verify the size of the defect targets to ensure that the pixel ratio of each defect target in the image is between 0.5% and 5%, and finally form an initial set of valid images. Step 1.3: Establish a lowercase English defect label library and assign unique indexes. Label the defects according to the principle of the rectangle being close to the edge of the defect to ensure that the defect accounts for a reasonable proportion within the rectangle. Automatically generate a txt label file with the same name as the image. Verify that the content of the file is the category index + normalized coordinates / size to ensure that it conforms to the YOLO format. Step 1.4: Divide the labeled YOLO format dataset into training set, validation set and test set in a ratio of 7:2:

1. Use a hierarchical randomization strategy to ensure that the proportion of each type of defect in the three subsets is consistent with the total dataset. Step 1.5: Create the root directory PCB_Defect_Dataset, with three subdirectories: train, val, and test. Each subdirectory is further divided into images and labels to store JPG images and corresponding TXT labels. Use a script to verify that the number of images and labels files in the same subset is consistent.

3. The PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy according to claim 1, characterized in that, When performing multi-branch feature extraction, the 1 convolutional layer and 4 C3k2-PPA modules in the backbone network are first defined as 5 feature extraction layers, namely extraction layer E1, extraction layer E2, extraction layer E3, extraction layer E4 and extraction layer E5. The high-dimensional feature extraction of PCB defect images is completed step by step through these 5 extraction layers. Then input the original defect image. For a three-channel color image, the above five extraction layers are used to extract the output feature map. , ; Represents the channel dimension of the i-th layer. H and W represent the height and width of the image, respectively.

4. The PCB defect detection method based on multi-branch feature extraction and adaptive enhancement strategy according to claim 1 or 3, characterized in that, In the backbone network of the PCB surface small target defect detection network, each C3k2-PPA module contains a parallelized patch-aware attention module (PPA). The feature map input to the PPA module is convolved pointwise and then divided into three branches for feature extraction: a local branch, a global branch, and a serial convolution branch. The feature map extracted by each C3k2-PPA module... The expression is as follows: ； in, This represents the feature map of the first branch in the n branches of the i-th layer. This represents the feature map of the j-th branch in the n-th branch of the i-th layer. , This represents the output feature map of the i-th layer C3k2-PPA module. ; This refers to the attention weights generated by the parallelized patch-aware attention module (PPA). Each parallelized PPA performs multi-branch feature extraction and feature fusion attention operations to obtain the feature map. The process is as follows: Step 1) For a given tensor Then, it is first adjusted through pointwise convolution to obtain ; Step 2) Output the serial convolution feature map by entering the serial convolution branch. ; By controlling the patch size parameter p=2, the local branch is entered, and the local feature map is output. ; By controlling the patch size parameter p=4, the system enters the global branch and outputs the global feature map. ; Step 3) Add the feature maps output from the sequential convolutional branch, local branch, and global branch, and output the summed feature map. ; ； Step 4) sequentially from one-dimensional channel attention and 2D spatial attention map Perform adaptive feature enhancement processing, and finally output ; ； ； ； in, This indicates element-wise multiplication. and The feature map represents the result of channel selection and spatial selection. and Representing the rectified linear unit ReLU and batch normalized BN, respectively. This indicates the final output characteristics of the PPA module.

Images