Artificial intelligence industrial PCB defect detection method and system

Abstract

The invention discloses an artificial intelligence industrial PCB defect detection method and system, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining PCB image data and defect labeling data; generating three types of enhanced image sets through directional enhancement, and weighting and extracting small defect samples to construct a training data set; inputting the training data set into an improved YOLOv8 model for training, and performing multi-scale feature fusion on an input image to generate initial feature data; generating filtering characteristic data according to a PCB circuit texture prior rule; the defect area features are enhanced through a Biformer attention mechanism, and enhanced feature data are generated; inputting the enhanced feature data into the detection head network to generate prediction bounding box data; a dynamic NWD loss function is adopted to calculate the distribution distance between the predicted bounding box and the real bounding box, and model parameters are optimized; performing defect reasoning on the target PCB image according to the trained model, and outputting defect position coordinates, types and confidence coefficients; according to the method, the small defect detection precision of the model can be remarkably improved, and the omission ratio is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an artificial intelligence-based method and system for detecting defects in industrial PCBs. Background Technology

[0002] With the automation and refinement of industrial production, product quality inspection has become an important part of the production process. In particular, in the production of printed circuit boards (PCBs), detecting surface defects on PCBs is an important step to ensure their functionality and stability.

[0003] Currently, there has been some progress in PCB defect detection methods based on deep learning, especially in the application of convolutional neural networks (CNN) and YOLO series of target detection algorithms. These methods combine deep learning algorithms with traditional image processing techniques, and can achieve relatively accurate defect detection by automatically extracting features and classifying them.

[0004] However, there are still some shortcomings in the existing technology, mainly in the following aspects:

[0005] Poor small defect detection capability: Many existing methods perform poorly in detecting small defects (such as missing solder joints, short circuits, etc.). Traditional target detection algorithms, especially those based on the IoU loss function, have insufficient accuracy in detecting small defects, resulting in a high false negative rate.

[0006] Low feature extraction efficiency: Current convolutional neural networks and attention mechanisms (such as SE, CBAM, etc.) often struggle to effectively focus on key regions when dealing with minor defects in complex backgrounds, resulting in low feature extraction efficiency and difficulty in adapting to complex backgrounds and diverse defects in industrial environments.

[0007] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides an artificial intelligence-based industrial PCB defect detection method and system.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows:

[0010] In a first aspect, the present invention discloses an artificial intelligence-based industrial PCB defect detection method, comprising the following steps:

[0011] Acquire PCB image data and defect annotation data, wherein the defect annotation data includes actual bounding box data and category labels;

[0012] Based on the PCB image data and defect annotation data, three types of enhanced image sets are generated through targeted enhancement, and a training dataset is constructed by weighted extraction of small defect samples.

[0013] The training dataset is input into the improved YOLOv8 model for training, and multi-scale feature fusion is performed on the input image to generate initial feature data;

[0014] The initial feature data is filtered based on the prior rules of PCB circuit texture to generate filtered feature data;

[0015] Enhanced feature data is generated by strengthening the defective region features in the filtered feature data using the Biformer attention mechanism.

[0016] The enhanced feature data is input into the detection head network to generate predicted bounding box data;

[0017] The dynamic NWD loss function is used to calculate the distribution distance between the predicted bounding box and the true bounding box, and the model parameters are optimized.

[0018] Based on the trained model, perform defect inference on the target PCB image and output the defect location coordinates, category, and confidence level.

[0019] Secondly, this invention discloses an artificial intelligence-based industrial PCB defect detection system, comprising:

[0020] The data acquisition module is used to acquire PCB image data and defect annotation data, wherein the defect annotation data includes actual bounding box data and category labels;

[0021] The data augmentation module is used to generate three types of augmented image sets through targeted augmentation based on the PCB image data and defect annotation data, and to extract small defect samples by weighting to construct a training dataset.

[0022] The model training module is used to input the training dataset into the improved YOLOv8 model for training, perform multi-scale feature fusion, and generate initial feature data.

[0023] The feature filtering module is used to filter the initial feature data based on the prior rules of PCB circuit texture and generate filtered feature data.

[0024] The feature enhancement module is used to enhance the defect region features in the filtered feature data through the Biformer attention mechanism to generate enhanced feature data;

[0025] The parameter optimization module is used to input the enhanced feature data into the detection head network to generate predicted bounding box data; and to calculate the distribution distance between the predicted bounding box and the ground truth bounding box using the dynamic NWD loss function to optimize the model parameters.

[0026] The target detection module is used to perform defect inference on the target PCB image based on the trained model, and output the defect location coordinates, category and confidence level.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. By enhancing the diversity of the training dataset and using the dynamic NWD loss function, the model's detection accuracy for small defects (such as missing solder joints, short circuits, etc.) is significantly improved, and the false negative rate is reduced.

[0029] 2. By integrating the improved YOLOv8 model and the Biformer attention mechanism, the model's feature extraction capability in complex backgrounds is enhanced, thereby improving the model's robustness.

[0030] 3. By adopting efficient deep learning methods to replace traditional manual inspection, the automation and efficiency of PCB defect inspection are significantly improved, meeting the needs of large-scale industrial production;

[0031] 4. By weighted sampling of small defect samples and fusion of multi-scale features, the generalization ability of the model is improved, enabling it to adapt to the inspection needs of different PCB styles and production environments. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is an overall block diagram of the method in Embodiment 1 of the present invention;

[0034] Figure 2 This is a diagram showing the detection effect of the improved model in Embodiment 1 of the present invention;

[0035] Figure 3 This is an architecture diagram of the improved YOLOv8 model in Embodiment 1 of the present invention;

[0036] Figure 4 This is a schematic diagram of the F1 Score of the baseline model before model improvement in Embodiment 1 of the present invention;

[0037] Figure 5 This is a schematic diagram of the F1 Score of the improved YOLOv8 model after model improvement in Embodiment 1 of the present invention;

[0038] Figure 6 This is a schematic diagram of the mAP index results of the baseline model before model improvement in Embodiment 1 of the present invention;

[0039] Figure 7 This is a schematic diagram of the mAP index of the improved YOLOv8 model after model improvement in Embodiment 1 of the present invention;

[0040] Figure 8 This is a schematic diagram of the hole detection results of the baseline model before model improvement in Embodiment 1 of the present invention;

[0041] Figure 9 This is a schematic diagram of the hole detection results of the improved YOLOv8 model after model improvement in Embodiment 1 of the present invention;

[0042] Figure 10 This is a schematic diagram of the short-circuit detection results of the baseline model before model improvement in Embodiment 1 of the present invention;

[0043] Figure 11 This is a schematic diagram of the short-circuit detection results of the improved YOLOv8 model in Embodiment 1 of the present invention.

[0044] Figure 12 This is an overall block diagram of the system in Embodiment 2 of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Application Overview: In traditional deep learning-based PCB defect detection systems, the false negative rate for small-sized defects is significantly higher than that for conventional defects, especially in scenarios involving micron-level defects such as missing solder joints and short circuits, where detection accuracy exhibits a non-linear decreasing trend. Features extracted by convolutional neural networks from complex circuit textures are susceptible to high-frequency noise interference, making it difficult for the attention mechanism to effectively focus on defect areas, resulting in a large amount of redundant information in the feature space.

[0047] For example, in an online inspection system using the YOLOv5 architecture, after feature extraction from a 640×640 pixel PCB image via the backbone network, the resolution of the smallest-scale feature map output by the feature pyramid drops to 20×20 pixels. When the target is a micro-short-circuit defect with an area less than 0.5% of the image area, the IoU metric during bounding box regression becomes significantly less sensitive to positional deviations, resulting in a match between the predicted and ground truth bounding boxes that falls below the detection threshold. Simultaneously, multi-layer convolutional operations generate feature confusion in densely packed circuit trace areas, and the weight matrix generated by the attention mechanism fails to effectively distinguish between normal pads and defect areas, ultimately leading to a multi-peaked confidence distribution in the detection head output.

[0048] To address the aforementioned issues, this application first focuses on the insufficient accuracy of small defect detection. Analysis reveals that traditional data augmentation methods struggle to effectively expand the diversity of small defect samples, leading to inadequate feature learning during model training. To resolve this, a targeted augmentation strategy is employed to generate multiple types of augmented image sets, combined with a weighted sampling mechanism to balance sample distribution. Simultaneously, existing attention mechanisms are observed to have insufficient feature focusing capabilities in complex circuit contexts; therefore, a more efficient attention module is introduced to enhance the feature representation of defect regions. Furthermore, it is found that IoU-based loss functions are less sensitive to positional deviations of small targets; therefore, a distributed distance metric is explored to optimize the bounding box regression process.

[0049] Example 1:

[0050] like Figure 1-11 As shown, an artificial intelligence-based industrial PCB defect detection method includes the following steps:

[0051] Acquire PCB image data and defect annotation data, wherein the defect annotation data includes actual bounding box data and category labels;

[0052] Based on the PCB image data and defect annotation data, three types of enhanced image sets are generated through targeted enhancement, and a training dataset is constructed by weighted extraction of small defect samples.

[0053] The training dataset is input into the improved YOLOv8 model for training, and multi-scale feature fusion is performed on the input image to generate initial feature data;

[0054] The initial feature data is filtered based on the prior rules of PCB circuit texture to generate filtered feature data;

[0055] Enhanced feature data is generated by strengthening the defective region features in the filtered feature data using the Biformer attention mechanism.

[0056] The enhanced feature data is input into the detection head network to generate predicted bounding box data;

[0057] The dynamic NWD loss function is used to calculate the distribution distance between the predicted bounding box and the true bounding box, and the model parameters are optimized.

[0058] Based on the trained model, perform defect inference on the target PCB image and output the defect location coordinates, category, and confidence level.

[0059] Through the above-described scheme, this application effectively improves the accuracy of PCB defect detection, especially the detection capability of small-sized defects. By employing targeted augmentation and weighted sampling, the diversity and proportion of small defect samples are increased, enabling the model to better learn the features of small defects. The introduction of the Biformer attention mechanism enhances the model's ability to focus on defect regions in complex backgrounds and reduces redundant information in the feature extraction process. The application of the dynamic NWD loss function improves the model's sensitivity to small target positional deviations and further optimizes the bounding box regression process. These improvements work together to significantly enhance the performance of the PCB defect detection system, particularly in terms of accuracy and recall when detecting small-sized defects, meeting the requirements of industrial-grade quality standards and improving the yield and efficiency of the production line.

[0060] This application further proposes that the PCB image data includes a matrix of printed circuit board images containing defects, and the defect annotation data includes the true bounding box of the defect and category labels; the process of generating three types of enhanced image sets by targeted enhancement includes: using the CutMix data enhancement method to fuse the defect and background regions of different images to generate a first enhanced image set; using the Mosaic data enhancement method to stitch the defect image and the background image at a first ratio to generate a second enhanced image set; and using the Mosaic data enhancement method to stitch the defect image and the background image at a second ratio to generate a third enhanced image set; wherein the second ratio is higher than the first ratio.

[0061] Specifically, in the training data construction phase, the CutMix method is first used to randomly fuse defect areas and background areas from different PCB images, generating a first enhanced image set with cross-image combination features. This method can simulate the random combination relationship between defects and background in actual production processes. Subsequently, the Mosaic method is used to stitch the defect image and background image at a 1:3 ratio to generate a second enhanced image set. In this case, the defect area occupies a smaller area in the stitched image, simulating the distribution of small-sized defects in a complex background. A third enhanced image set is then generated at a 3:1 ratio, making the defect area occupy the main part of the image and strengthening the model's ability to identify dense defects. By adjusting the stitching ratio, the defect area in the enhanced image generated at the second ratio is three times that of the second enhanced image set, forming a continuous sample distribution from low density to high density. The combined use of these three types of enhanced image sets allows the model to simultaneously learn detection patterns of isolated small defects, scattered multiple defects, and dense defects during training, effectively improving the robustness to detecting defects with significant size differences.

[0062] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0063] PCB image data includes a matrix of printed circuit board images containing defects, and defect annotation data includes the actual defect bounding boxes and category labels. The process of generating three types of enhanced image sets through targeted enhancement includes:

[0064] The CutMix data augmentation method is used to fuse defect and background areas from different images to generate the first augmented image set. Specifically, rectangular regions are randomly cropped from two original PCB images, and the cropped region from one image is used to replace the region in the other image to generate a new fused image.

[0065] The Mosaic data augmentation method is used to stitch the defect image and the background image at a first scale to generate a second augmented image set. Specifically, four original PCB images are selected, each image is scaled to half its original size, and then stitched together in a 2x2 grid to form a new image.

[0066] The Mosaic data augmentation method is used to stitch the defect image and the background image at a second scale to generate a third augmented image set. Specifically, nine original PCB images are selected, each image is scaled to one-third of its original size, and then stitched together in a 3x3 grid to form a new image.

[0067] Through the above technical solutions, this application generates diverse training samples using various data augmentation methods, improving the model's ability to identify defects of different scales, locations, and combinations. The CutMix method enhances the model's learning of local features, while two different proportions of the Mosaic method enhance the model's adaptability to defects of different scales. This diversified data augmentation strategy effectively improves the model's generalization ability and robustness, especially its ability to detect small defects.

[0068] This application further proposes an improved YOLOv8 model, including replacing the original YOLOv8's CIoU loss function with a dynamic NWD loss function, and embedding a Biformer attention mechanism module after the SPPF multi-scale feature fusion module. The basic architecture of the improved YOLOv8 model also includes: a backbone network containing Conv, C2f modules, and an SPPF module; the C2f module extracts multi-level features through cross-stage partial connections, and the SPPF module extracts features through three pooling windows of different sizes to enhance the model's ability to detect multi-scale objects; a feature pyramid network containing bidirectional feature pyramids with top-down and bottom-up paths to fuse feature maps of different scales; a detection head adopting a decoupled head structure to output bounding box coordinates, class probabilities, and confidence scores respectively; and a backbone network including: a basic convolutional module consisting of convolutional units composed of Conv-BN-SiLU; a cross-stage partial connection module that divides the feature map into two parts, processes them separately, and then concatenates them; and a spatial pyramid pooling module that fuses features from different receptive fields through multi-branch pooling operations.

[0069] The backbone network uses a C2f module to achieve cross-stage partial connectivity, dividing the input feature map into two parts. One part is processed through multiple residual blocks, while the other part is directly concatenated with the processed features, thus preserving shallow detail information. The SPPF module employs three parallel max-pooling layers with pooling windows of 5×5, 9×9, and 13×13, respectively, extracting features at multiple scales and concatenating them along the channel dimension. The bidirectional feature pyramid network upsamples deep features and fuses them with shallow features in a top-down path, and downsamples the fused features in a bottom-up path and fuses them with deep features again. The decoupled detection head separates bounding box regression from the classification task, outputting coordinate offsets, class probabilities, and target confidence through independent branches.

[0070] Specifically, the backbone network's basic convolutional module adopts a Conv-BN-SiLU structure. This module extracts local features through convolutional layers, batch normalization layers accelerate model convergence, and the SiLU activation function enhances non-linear expressive power. The cross-stage partial connection module divides the input feature map into two sub-parts. The first part is processed through multiple convolutional layers, and the second part is directly concatenated with the processing result, thus reducing computation while preserving original feature information. The dynamic NWD loss function models the bounding boxes as Gaussian distributions and optimizes the localization accuracy of small targets by calculating the Wasserstein distance between distributions. Combined with the fine-grained features extracted by the C2f module, it alleviates the insufficiency of traditional IoU in dealing with small-sized bounding boxes. For example, during training, when the size difference between the predicted and true bounding boxes is small, dynamic NWD compresses the distance difference through an exponential function, enhancing the model's ability to optimize for small defects. The Biformer attention mechanism, based on the multi-scale features output by the SPPF module, generates attention weights through the interaction of the query matrix and the key matrix. For example, it introduces the feature variance ρ when calculating the query matrix to dynamically adjust the feature contributions of different receptive field branches. The bidirectional feature pyramid network transmits semantic information through a top-down path and detailed information through a bottom-up path. For example, during the fusion process, 1x1 convolutions are used to adjust the number of channels, and then feature complementarity is achieved through element-wise addition. The decoupled detection head outputs bounding box coordinates and class probabilities through independent branches. For example, the bounding box regression branch uses 4 convolutional layers, and the classification branch uses 2 convolutional layers to avoid parameter interference between tasks.

[0071] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0072] The improved YOLOv8 model includes replacing the original YOLOv8's CIoU loss function with a dynamic NWD loss function and embedding a Biformer attention mechanism module after the SPPF multi-scale feature fusion module. The basic architecture of the improved YOLOv8 model also includes a backbone network, a feature pyramid network, and a detection head.

[0073] The backbone network extracts multi-level features through cross-stage partial connections. Specifically, it comprises five stages, each consisting of multiple residual blocks. Each residual block contains 1x1 and 3x3 convolutions, and feature reuse is achieved through skip connections. Cross-stage partial connections divide the input feature map into two parts: one part is directly passed, and the other part is convolved with the directly passed part after convolution processing, effectively extracting multi-scale features.

[0074] The feature pyramid network comprises bidirectional feature pyramids with both top-down and bottom-up paths, fusing feature maps at different scales. The top-down path upsamples high-level semantic features to lower levels, while the bottom-up path uses convolution and pooling to pass low-level detailed features to higher levels. This bidirectional feature fusion allows for the simultaneous utilization of both high-level semantic information and low-level detailed information.

[0075] The detection head employs a decoupled header structure, outputting bounding box coordinates, class probabilities, and confidence scores separately. Specifically, the detection head contains three parallel branches, predicting the bounding box coordinates, class probabilities, and confidence scores respectively. The bounding box coordinates branch outputs four channels, representing the center point coordinates, width, and height; the class probability branch outputs a channel representing the number of classes; and the confidence score branch outputs one channel.

[0076] The backbone network comprises a basic convolutional module, a cross-stage partial connection module, and a spatial pyramid pooling module. The basic convolutional module consists of Conv-BN-SiLU, where Conv is the convolutional layer, BN is the batch normalization layer, and SiLU is the activation function. The cross-stage partial connection module divides the feature map into two parts, processes them separately, and then concatenates them, reducing computation while maintaining information flow. The spatial pyramid pooling module fuses features from different receptive fields through multi-branch pooling operations, including 1x1, 5x5, 9x9, and 13x13 max pooling, effectively expanding the receptive field.

[0077] Through the above technical solutions, the improved YOLOv8 model structure of this application can effectively extract multi-scale features from PCB images, enhancing the detection capability for small target defects. The dynamic NWD loss function can more accurately measure the distribution distance between the predicted bounding box and the true bounding box, improving the bounding box regression accuracy. The Biformer attention mechanism module can enhance the feature extraction capability of key regions, improving the accuracy of defect detection. The decoupled detection head structure can optimize bounding box localization, category classification, and confidence prediction separately, further improving detection performance. Overall, this improved model can better adapt to the needs of PCB defect detection tasks, improving detection accuracy and efficiency.

[0078] This application further proposes specific steps for weighted extraction of small defect samples, including: labeling small defect samples according to the bounding box size in the defect annotation data; assigning a first weight to small defect samples and a second weight to non-small defect samples; and randomly extracting samples according to the weights to form a training dataset to ensure that the proportion of small defect samples is not lower than a preset threshold.

[0079] The identification of small defect samples is achieved by calculating the ratio of the bounding box area to the total image area; samples with a ratio below 0.5% are considered small defect samples. The first weight is set to three times the weight of non-small defect samples, and a preset threshold is dynamically adjusted based on the training set size; for example, it is set to 20% in a training set of hundreds of thousands of samples. The weight allocation process is implemented using a probability distribution function, and the sampling process for the training dataset employs a stratified random sampling strategy to ensure sufficient coverage density of small defect samples in each batch of training data.

[0080] Specifically, during the model training phase, although the image set generated by data augmentation contains small defect samples, their small original number and low probability in random sampling lead to an imbalanced distribution of training data. By using a bounding box size labeling mechanism, small defect samples with an area ratio below a threshold are independently classified and assigned higher sampling weights. In the weight allocation process, the difference ratio between the first and second weights was experimentally verified to be 3:1, which effectively increases the frequency of small defect samples appearing in the training set. A weighted random algorithm is used during sampling, adjusting the probability distribution according to the weight values ​​each time samples are drawn, ensuring that the proportion of small defect samples in the training set remains consistently above 20%. This mechanism significantly increases the frequency of the model learning small defect features during backpropagation, thereby improving the model's ability to identify minor solder joint defects or short-circuit defects.

[0081] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0082] The specific steps for weighted extraction of small defect samples include: labeling small defect samples based on the bounding box size in the defect annotation data. Specifically, a threshold can be set, for example, samples with bounding box areas less than 1% of the total image area are labeled as small defect samples. A first weight is assigned to small defect samples, and a second weight is assigned to non-small defect samples. For example, a weight of 2.0 can be assigned to small defect samples, and a weight of 1.0 to non-small defect samples. Samples are randomly extracted according to the weights to form the training dataset, ensuring that the proportion of small defect samples is not less than a preset threshold. Specifically, a weighted random sampling method can be used, setting the proportion of small defect samples in the final training set to be no less than 30%.

[0083] Through the above technical solution, this application can effectively increase the proportion of small defect samples in the training dataset, enhancing the model's ability to detect small defects. As a result, the model can better learn the characteristics of small defects, reduce the false negative rate, and improve detection accuracy. Furthermore, by adjusting the weights and proportion thresholds, the detection performance of defects of different sizes can be flexibly balanced to adapt to the needs of different application scenarios.

[0084] This application further proposes a priori rules for PCB circuit textures: calculate the angle θ between the edge direction of the feature region and the standard direction of the PCB line, and the matching degree σ between the shape and normal circuit components; retain feature regions that satisfy θ greater than a preset angle threshold or σ less than a preset matching degree threshold.

[0085] In calculating the angle θ between the edge direction and the standard direction, the Canny edge detection algorithm is used to extract the edge contour of the feature region, the main edge direction is calculated using Hough transform, and the vector angle between the main edge and the standard direction of the PCB circuit is calculated. The shape matching degree σ is calculated using a contour matching algorithm, comparing the feature region contour with a standard circuit component template for similarity. The preset angle threshold is set to 15°-30°, and the preset matching degree threshold is set to 0.85-0.95. This joint judgment mechanism of angle and matching degree thresholds can simultaneously cover both directional and shape anomalies.

[0086] Specifically, in the feature filtering process, the initial feature map output from multi-scale feature fusion is first segmented into regions. For each feature region, the main edge direction is extracted using an edge direction detection algorithm, and the angle between this angle and the standard trace direction defined in the PCB circuit design file is calculated. When the edge direction of a feature region deviates from the standard direction by more than 15°, it is identified as a potential defect region. Simultaneously, the feature region contour is matched with a standard circuit component template to calculate the similarity σ. If the matching degree is less than 0.9, it is identified as an abnormal region. Through this dual judgment mechanism, regions that conform to normal circuit texture features can be effectively filtered out, while feature regions with abnormal orientation or shape distortion are retained. This filtering method transforms subjective experience judgment into calculable objective parameters by quantifying angle deviation and shape difference, avoiding the blindness of manually setting filtering rules in traditional methods.

[0087] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0088] The PCB circuit texture prior rules include the following steps:

[0089] Calculate the angle θ between the edge direction of the feature region and the standard direction of the PCB trace. Specifically, first, perform edge detection on the feature region and extract edge pixels. Then, calculate the gradient direction of the edge pixels and count the main edge directions. Compare the main edge directions with the standard direction of the PCB trace (usually horizontal, vertical, or a 45-degree angle) to obtain the angle θ.

[0090] The matching degree σ between the shape of a feature region and a normal circuit component can be calculated using template matching or shape descriptors (such as Hu moments). A shape template library of normal circuit components is pre-established, including standard contours of various common components such as resistors, capacitors, and chips. Shape analysis is performed on the feature region to extract contour features, which are then matched with standard contours in the template library to calculate the similarity score σ.

[0091] Set a preset angle threshold and a preset matching degree threshold. For example, you can set the preset angle threshold to 10 degrees and the preset matching degree threshold to 0.8.

[0092] For each feature region, determine whether it meets the condition that θ is greater than a preset angle threshold or σ is less than a preset matching degree threshold. If either condition is met, retain the feature region; otherwise, filter it out.

[0093] Furthermore, the threshold can be dynamically adjusted according to the actual application scenario. For example, for PCBs with high precision requirements, the preset angle threshold can be appropriately reduced and the preset matching degree threshold increased to enhance the filtering effect.

[0094] Through the above technical solution, this application can effectively filter out feature areas that match the texture of normal PCB circuits, while retaining potential defect areas. This improves the accuracy and efficiency of subsequent defect detection and reduces the false detection rate. Simultaneously, this method fully utilizes prior knowledge of PCB circuits, making feature filtering more targeted and interpretable. Furthermore, by setting an adjustable threshold, this method possesses good adaptability and flexibility, enabling it to adapt to PCB defect detection tasks of different types and accuracy requirements.

[0095] This application further proposes the calculation of the dynamic NWD loss function, which includes: constructing a predicted Gaussian distribution and a true Gaussian distribution based on the predicted bounding box data and the true bounding box data, respectively. The predicted bounding box data includes the predicted center coordinates, predicted width, and predicted height. The predicted Gaussian distribution is modeled using the predicted center coordinates, predicted half-width, and predicted half-height, while the true Gaussian distribution is modeled using the true center coordinates, true half-width, and true half-height. The squared Wasserstein distance between the predicted Gaussian distribution and the true Gaussian distribution is calculated using the following formula:

[0096]

[0097] in, For Wasserstein distance, To from the bounding box and The Gaussian distribution modeled To predict the center coordinates, To predict the width, To predict high, For the true center coordinates, For true width, For true high, The square of the L2 norm;

[0098]

[0099] in, Let C be the NWD distance between bounding boxes A and B, and C is a constant.

[0100] Specifically, the predicted and ground truth bounding boxes are converted to Gaussian distributions, and the difference in distribution is measured by calculating the squared Wasserstein distance. The NWD distance maps the distance to a similarity metric using an exponential function, and its scaling factor C controls the sensitivity of the loss function.

[0101] Through the above technical solutions, this application achieves an improvement in the accuracy of PCB defect detection. The dynamic NWD loss function models the bounding box using a Gaussian distribution, considering both the position and size information of the bounding box, and can more accurately measure the difference between the predicted and true bounding boxes. This method is particularly effective in detecting small defects, significantly reducing the false negative rate. Furthermore, because the NWD distance calculation considers the overall distribution characteristics of the bounding box, rather than just the overlapping area, it also exhibits high robustness when dealing with PCB defects with irregular shapes or large size variations. In addition, by introducing an adjustable constant C, this loss function can adapt to defects of different sizes, further improving the model's flexibility and generalization ability.

[0102] This application further proposes a method for dynamically adjusting the constant C in the dynamic NWD loss function, the adjustment formula being:

[0103]

[0104] in, is a preset coefficient, and S is the actual bounding box area.

[0105] Wherein, the preset coefficient α is used as a basic constant, β is an adjustment coefficient, and γ is the area offset. The square root of the actual bounding box area S. Used to quantify the actual size of the defect. When S is small, The value decreases, resulting in As the term increases, combined with the adjustment of the β coefficient, the C value dynamically increases as the defect size decreases. The preset coefficients α, β, and γ are determined in advance through experiments; for example, α is 0.5, β is 0.2, and γ is 10.

[0106] Specifically, during training, when the actual bounding box area S is small, for example, S = 4 mm², =2, at this time C=0.5+0.2*(10-2)=2.1. Compared with the case of a fixed constant C=1, increasing the value of C reduces the decay of the exponential term in the NWD distance calculation, thereby strengthening the loss gradient of small defects. When S is large, for example S=25mm², =5, C=0.5+0.2*(10-5)=1.5. At this point, the value of C is reduced to avoid oversensitivity to large-sized defects. By dynamically adjusting the value of C, the model can adaptively balance the optimization weights of defects of different scales according to the defect size, reducing the missed detection rate of small defects.

[0107] Through the above technical solution, this application achieves adaptive adjustment of the NWD loss function for defects of different sizes. For small defects, the sensitivity of the loss function is enhanced by dynamically adjusting the C value, thus improving the detection accuracy of small defects. Meanwhile, reasonable detection performance is maintained for large defects. This dynamic adjustment mechanism effectively solves the limitations of traditional loss functions in handling targets of different sizes, improving the overall performance of PCB defect detection.

[0108] This application further proposes an attention mechanism employing a Biformer module, the computation of which includes: generating a query matrix Q, a key matrix K, and a value matrix V from the filtered feature data, and calculating the feature weight matrix:

[0109]

[0110] in For feature dimensions.

[0111] The query matrix Q, key matrix K, and value matrix V are generated from the filtered feature data through convolution operations, with feature dimensions... Set to the same number of input feature channels. Attention weights are calculated using normalized dot products, with a scaling factor. This is used to control the range of dot product values ​​and avoid gradient vanishing. The enhanced feature data output by the attention mechanism highlights the features of the defect region through information in the weighted fusion value matrix V.

[0112] Specifically, after the filtered feature data is input into the Biformer module, it first generates Q, K, and V matrices through linear transformation. The dot product of Q and K captures the correlation between different spatial locations, and scaling operations ensure numerical stability. The Softmax function converts the correlation into a probability distribution, and weighted aggregates the feature information in the V matrix, allowing the model to focus on defect regions. For example, when the input feature dimension is 256, The value is set to 256, and the scaling factor is 16. By dynamically adjusting the attention weights, background interference features are suppressed, and the feature responses of small defect areas are enhanced, thereby improving detection accuracy.

[0113] Through the above technical solution, this application effectively solves the problem that the features of minute defects are easily interfered with by the background in complex industrial scenarios. By dynamically adjusting the attention mechanism through multi-branch pooling feature variance, the model's feature sensitivity to abnormal regions is enhanced, allowing the attention weights to be more accurately focused on the real defect areas. This technical solution significantly improves the feature differentiation between minute defects and normal circuit textures while maintaining computational efficiency, and reduces the probability of false detections caused by complex backgrounds.

[0114] This application further proposes a multi-scale feature fusion module using SPPF, which fuses feature maps from different receptive fields through multi-branch pooling operations. The Biformer attention mechanism module is directly connected to the output of the SPPF module. The multi-branch receptive field feature maps output by SPPF are extracted, and the feature variance ρ of each branch is calculated. ρ is then input into the attention module to dynamically adjust the generation of the query matrix Q.

[0115]

[0116] in, This is element-wise addition.

[0117] Furthermore, the SPPF module generates feature maps with receptive fields of three different sizes: 5×5, 9×9, and 13×13, through parallel pooling layers. The feature maps output by each branch are concatenated through channels to form a fused feature. The feature variance ρ is obtained by calculating the standard deviation of each branch's feature map in the channel dimension. Specifically, this is achieved by performing global average pooling on the feature maps output by each branch and then calculating the standard deviation. During dynamic adjustment, the key matrix K is multiplied element-wise with the feature variance ρ, and the resulting feature weight vector is superimposed on the original query matrix Q to form an enhanced query matrix Q' with multi-scale perception capabilities.

[0118] Specifically, after variance calculation, the variance value ρ of the multi-branch feature maps output by the SPPF module reflects the distribution differences of feature maps in different receptive fields. When the variance of a certain branch feature is large, it indicates that the feature at that scale contains more high-frequency information. In this case, the ρ*K operation will enhance the key weights corresponding to that branch. The superimposed Q' matrix prioritizes feature regions with large variance in the attention calculation, such as local texture abrupt changes corresponding to small defects. Experiments show that when the standard deviation algorithm is used for ρ calculation and the pooling kernel size is set to [5,9,13], the detection accuracy of the model in the 0402 package solder joint missing scenario is improved by 12.6%. This mechanism enables the attention module to dynamically adjust the region of interest according to the feature distribution, effectively improving the feature response intensity of small-sized defects.

[0119] Through the above technical solution, this application effectively solves the problem that traditional attention mechanisms have difficulty focusing on small defect areas in complex texture backgrounds. By dynamically fusing multi-scale feature variance information, attention weights are guided to areas with significant differences in feature distribution, enabling the model to enhance the feature response of defect areas when detecting low-pixel-ratio defects such as missing small solder joints and short circuits, while suppressing interference signals from normal circuit textures, thereby improving detection accuracy and robustness.

[0120] This application further proposes a performance evaluation method for the improved YOLOv8 model, wherein the performance evaluation metrics include precision, recall, mean precision (AP), mean mean precision (mAP), and F1 score.

[0121] The expression for the precision is:

[0122]

[0123] Where TP is the number of samples correctly identified as positive, and FP is the number of samples incorrectly identified as positive.

[0124] The expression for recall is:

[0125]

[0126] Where FN is the number of positive samples that are incorrectly identified as negative samples;

[0127] The The expression for the value is:

[0128]

[0129] Where P is precision and R is recall; setting B to 1 yields the F1 score.

[0130] The expression for the mean accuracy mAP value is:

[0131]

[0132]

[0133] Where AP is the area under the PR curve, and mAP is the average of AP calculated after AP is calculated for each category.

[0134] During implementation, model training was conducted on 4,783 annotated PCB images covering six types of defects, including open circuits, short circuits, and holes. The NWD+Biformer combination improved the small target F1 score by 21.8% and the mAP50 from 0.308 to 0.904.

[0135] Before-and-after performance tests were conducted to improve the short-circuit detection performance. The original model had an average confidence score of 0.72 for short-circuit detection, while the improved YOLOv8 model achieved a confidence score of 0.92 on the same dataset, with an accuracy improvement of 28%. Meanwhile, the improved YOLOv8 model also improved the confidence score for hole detection from an average of 0.83 to 0.92 on the same dataset, with an accuracy improvement of 11%.

[0136] This improvement significantly enhances the detection capability for minute defects such as short circuits and missing solder joints, especially in complex contexts, enhancing the model's ability to focus on critical areas. For example, when using the improved model for short circuit defect detection, the original model had a high false negative rate, particularly when the defect size was less than 0.5% of the total PCB image area, resulting in unstable detection results. The improved model, by enhancing the accuracy of feature extraction and bounding box regression, significantly reduces the false negative rate under the same conditions, demonstrating more reliable detection results.

[0137] For defects smaller than 10×10 pixels, the dynamic NWD loss function enhances the sensitivity of perception of small defect boundaries, enabling the model to more accurately locate defect edges. The accuracy of small defect detection increased from 72% to 92%, and the recall increased from 29.3% to 93.5%, substantially ensuring the reliability of the detection results.

[0138] Example 2:

[0139] like Figure 12 As shown, an artificial intelligence-based industrial PCB defect detection system includes:

[0140] The data acquisition module is used to acquire PCB image data and defect annotation data, wherein the defect annotation data includes actual bounding box data and category labels;

[0141] The data augmentation module is used to generate three types of augmented image sets through targeted augmentation based on the PCB image data and defect annotation data, and to extract small defect samples by weighting to construct a training dataset.

[0142] The model training module is used to input the training dataset into the improved YOLOv8 model for training, perform multi-scale feature fusion, and generate initial feature data.

[0143] The feature filtering module is used to filter the initial feature data based on the prior rules of PCB circuit texture and generate filtered feature data.

[0144] The feature enhancement module is used to enhance the defect region features in the filtered feature data through the Biformer attention mechanism to generate enhanced feature data;

[0145] The parameter optimization module is used to input the enhanced feature data into the detection head network to generate predicted bounding box data; and to calculate the distribution distance between the predicted bounding box and the ground truth bounding box using the dynamic NWD loss function to optimize the model parameters.

[0146] The target detection module is used to perform defect inference on the target PCB image based on the trained model, and output the defect location coordinates, category and confidence level.

[0147] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0148] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0149] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. An artificial intelligence-based industrial PCB defect detection method, characterized in that, Includes the following steps: Acquire PCB image data and defect annotation data, wherein the defect annotation data includes actual bounding box data and category labels; Based on the PCB image data and defect annotation data, three types of enhanced image sets are generated through targeted enhancement, and a training dataset is constructed by weighted extraction of small defect samples. The training dataset is input into the improved YOLOv8 model for training, and multi-scale feature fusion is performed on the input image to generate initial feature data; The initial feature data is filtered based on the prior rules of PCB circuit texture to generate filtered feature data; Enhanced feature data is generated by strengthening the defective region features in the filtered feature data using the Biformer attention mechanism. The enhanced feature data is input into the detection head network to generate predicted bounding box data; The dynamic NWD loss function is used to calculate the distribution distance between the predicted bounding box and the true bounding box, and the model parameters are optimized. Based on the trained model, perform defect inference on the target PCB image and output the defect location coordinates, category, and confidence level.

2. The artificial intelligence-based industrial PCB defect detection method according to claim 1, characterized in that: The PCB image data includes a matrix of printed circuit board images containing defects, and the defect annotation data includes the actual defect bounding box and category label. The process of generating three types of enhanced image sets through targeted enhancement includes: The CutMix data augmentation method is used to fuse defects and background regions from different images to generate the first augmented image set; The Mosaic data augmentation method is used to stitch the defect image and the background image at a first ratio to generate a second augmented image set; The Mosaic data augmentation method is used to stitch the defect image and the background image at a second ratio to generate a third augmented image set; The second proportion is higher than the first proportion.

3. The artificial intelligence-based industrial PCB defect detection method according to claim 2, characterized in that: The improved YOLOv8 model includes replacing the original YOLOv8 CIoU loss function with a dynamic NWD loss function and embedding a Biformer attention mechanism module after the SPPF multi-scale feature fusion module. The infrastructure of the improved YOLOv8 model also includes: Backbone network: includes Conv, C2f modules and SPPF module; C2f module extracts multi-level features through cross-stage partial connections, SPFF module extracts features through three pooling windows of different sizes to enhance the model's ability to detect multi-scale targets; Feature Pyramid Network: A bidirectional feature pyramid containing both top-down and bottom-up paths, fusing feature maps of different scales; Detection head: It adopts a decoupled head structure and outputs bounding box coordinates, class probability and confidence score respectively; The backbone network includes: Basic convolutional module: a convolutional unit composed of Conv-BN-SiLU; Cross-stage partial connection module: The feature map is divided into two parts, processed separately, and then concatenated; Spatial pyramid pooling module: It fuses features from different receptive fields through multi-branch pooling operations.

4. The artificial intelligence-based industrial PCB defect detection method according to claim 3, characterized in that: The specific steps for weighted extraction of small defect samples include: Mark small defect samples based on the bounding box dimensions in the defect annotation data; The first weight is assigned to samples with small defects, and the second weight is assigned to samples with non-small defects. Samples are randomly selected according to weights to form a training dataset, ensuring that the proportion of samples with small defects is not lower than a preset threshold.

5. The artificial intelligence-based industrial PCB defect detection method according to claim 4, characterized in that: The prior rules for PCB circuit texture are as follows: Calculate the angle θ between the edge direction of the feature region and the standard direction of the PCB circuit, as well as the matching degree σ between the shape and normal circuit components; Retain feature regions that satisfy θ > preset angle threshold or σ < preset matching degree threshold.

6. The artificial intelligence-based industrial PCB defect detection method according to claim 5, characterized in that: The calculation of the dynamic NWD loss function includes: A predicted Gaussian distribution and a true Gaussian distribution are constructed based on the predicted bounding box data and the true bounding box data, respectively. The predicted bounding box data includes the predicted center coordinates, predicted width, and predicted height. The predicted Gaussian distribution is modeled using the predicted center coordinates, predicted half-width, and predicted half-height. The true Gaussian distribution is modeled using the true center coordinates, true half-width, and true half-height. The squared Wasserstein distance between the predicted Gaussian distribution and the true Gaussian distribution is calculated using the following formula: in, For Wasserstein distance, To from the bounding box and The Gaussian distribution modeled To predict the center coordinates, To predict the width, To predict high, For the true center coordinates, For true width, For true high, The square of the L2 norm; in, Let C be the NWD distance between bounding boxes A and B, and C is a constant.

7. The artificial intelligence-based industrial PCB defect detection method according to claim 6, characterized in that: The constant C can be dynamically adjusted, and the adjustment formula is as follows: in, is a preset coefficient, and S is the actual bounding box area.

8. The artificial intelligence-based industrial PCB defect detection method according to claim 7, characterized in that: The attention mechanism employs a Biformer module, and its computation includes: Generate a query matrix Q, a key matrix K, and a value matrix V from the filtered feature data, and calculate the feature weight matrix: in For feature dimensions.

9. The artificial intelligence-based industrial PCB defect detection method according to claim 8, characterized in that: Also includes: The multi-scale feature fusion uses the SPPF module, which fuses feature maps from different receptive fields through multi-branch pooling operations. The Biformer attention mechanism module is directly connected to the output of the SPPF module. Extract the multi-branch receptive field feature map output by the SPPF module and calculate the feature variance ρ of each branch. Input ρ into the attention module to dynamically adjust the generation of the query matrix Q: in, This is element-wise addition.

10. An artificial intelligence-based industrial PCB defect detection system, characterized in that: Using an artificial intelligence-based industrial PCB defect detection method as described in any one of claims 1-9, comprising: The data acquisition module is used to acquire PCB image data and defect annotation data, wherein the defect annotation data includes actual bounding box data and category labels; The data augmentation module is used to generate three types of augmented image sets through targeted augmentation based on the PCB image data and defect annotation data, and to extract small defect samples by weighting to construct a training dataset. The model training module is used to input the training dataset into the improved YOLOv8 model for training, perform multi-scale feature fusion, and generate initial feature data. The feature filtering module is used to filter the initial feature data based on the prior rules of PCB circuit texture and generate filtered feature data. The feature enhancement module is used to enhance the defect region features in the filtered feature data through the Biformer attention mechanism to generate enhanced feature data; The parameter optimization module is used to input the enhanced feature data into the detection head network to generate predicted bounding box data; and to calculate the distribution distance between the predicted bounding box and the ground truth bounding box using the dynamic NWD loss function to optimize the model parameters. The target detection module is used to perform defect inference on the target PCB image based on the trained model, and output the defect location coordinates, category and confidence level.

Images