A PCB defect image detection method based on an improved deep learning algorithm

Abstract

The application discloses a PCB defect image detection method based on an improved deep learning algorithm. In view of the deficiencies of the YOLOv3 deep convolutional neural network model, such as imbalance of positive and negative samples, feature expression of small defects of the PCB, and extraction of important feature information of the image, the application proposes an improvement method of replacing the MSELoss function in the original YOLOv3 model with the FocalLoss function, so as to dynamically control the weights of positive and negative samples and difficult and easy classification samples. Meanwhile, the application embeds the SPP module in the branch small target detection head and parallelly integrates the CBAM module into the YOLOv3 model, so that the model can pay more attention to the defect information in the image, thereby improving the overall feature extraction capability of the model. The method can detect different types of PCB defect images, and compared with the original YOLOv3 model, the sensitivity and other indicators are improved, has good performance in the PCB defect detection task, can reduce the missed detection rate and the false detection rate, and better realizes the automatic and intelligent detection of the PCB defects.

Description

Technical Field

[0001] This invention relates to the field of printed circuit board (PCB) defect detection, and specifically to a PCB defect image detection method based on an improved deep learning algorithm. Background Technology

[0002] Printed circuit boards (PCBs) are core components of the underlying hardware in the information industry, widely used in industrial manufacturing, communications, medical, and aerospace fields. With continuous globalization, China, with its large market and low production costs, has attracted substantial investment, becoming the world's largest PCB producer with a complete range of products, holding a pivotal position in the global PCB industry. However, due to the complexity of the PCB manufacturing process, quality issues can arise at any stage. Therefore, to ensure the normal sale of PCB products, defect detection is necessary before shipment. Traditional manual and mechanical inspection methods suffer from numerous shortcomings, including susceptibility to misjudgment, missed detections, low accuracy, and damage to the product's surface structure. While subsequent traditional image processing algorithms have improved these aspects to some extent, threshold segmentation methods are ineffective in linearly processing defects with complex circuits, weak connectivity, and discrete features, resulting in low detection rates. Therefore, utilizing deep learning technology for PCB defect identification is particularly important.

[0003] In recent years, deep learning algorithms have made groundbreaking progress in fields such as machine vision, image classification, object detection, and intelligent robotics, which can effectively compensate for the shortcomings of traditional algorithms in PCB defect detection. Deep learning mainly uses neural networks to extract high-level features from input images, and then performs classification and detection operations on samples based on these features. This avoids manual feature extraction and has good robustness and generalization ability, making it well-suited for the field of PCB defect identification and detection. Moreover, deep learning is highly adaptable to different detection tasks; by learning from different datasets, it can potentially meet the needs of industrial production inspection. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a PCB defect image detection method based on an improved deep learning algorithm. This method is applied to various shaped defect images of PCBs and overcomes the numerous deficiencies of traditional manual and mechanical inspection methods, such as easy misjudgment, missed detection, low detection accuracy, and damage to the product surface structure. It also overcomes the drawback of traditional image processing algorithms, which, when using threshold segmentation, cannot effectively linearly process defects with complex circuits, weak connectivity, and discrete features, resulting in low detection rates. This significantly improves work efficiency, reduces manual inspection costs, increases production efficiency, and promotes the development of automated PCB defect detection.

[0005] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0006] A PCB defect image detection method based on an improved deep learning algorithm includes the following steps:

[0007] (1) Establish an image dataset: Establish a PCB image dataset, obtain images of various types of PCB defects, and annotate them;

[0008] (2) Replace the MSE Loss function in the original YOLOv3 model with the Focal Loss function.

[0009] (3) Embed the SPP module (Spatial Pyramid Pooling) into the branch small target detection head.

[0010] (4) The CBAM module (Convolutional Block Attention Module) is integrated into the YOLOv3 model in parallel.

[0011] (5) Divide the PCB defect image dataset into training set, validation set and test set, and train and test the improved YOLOv3 model in sequence to finally realize the automated and intelligent identification of PCB defect categories.

[0012] Preferably, step (1) specifically includes:

[0013] (1.1) Professionals collect and organize the PCB images output by the PCB appearance defect inspection machine to obtain a dataset of defective images.

[0014] (1.2) Based on the PCB image dataset prepared in step (1.1), the target detection open source software LalbelImg is used to annotate the images and generate corresponding XML format label files, and finally obtain PCB defect image label files.

[0015] Preferably, step (2) specifically includes:

[0016] To address the imbalance between positive and negative samples, an improvement method is proposed: replacing the MSE Loss function in the original YOLOv3 model with the Focal Loss function.

[0017] (2.1) Object detection algorithms can be mainly divided into two categories: one-stage detection and two-stage detection. In fact, the recognition accuracy of one-stage object detection algorithms is lower than that of two-stage algorithms. Two-stage detection algorithms, represented by Faster R-CNN, obtain candidate regions of interest (ROIs) through a region proposal network. These candidate regions are first processed by a binary classification neural network. This network can effectively control the ratio of positive to negative samples of bounding boxes to 1:3, thus dividing them into background and foreground containing the target object. In contrast, one-stage detection algorithms, represented by YOLO, do not perform binary classification of bounding boxes between background and target. As a result, an image may generate a large number of bounding boxes, but only a small portion of them contain the target object, meaning that a large number of bounding boxes are negative samples.

[0018] The main reason why single-stage algorithms are less accurate than two-stage algorithms is the imbalance of sample classes. Training samples contain both positive and negative samples. Positive samples generally refer to the object to be detected, while negative samples generally refer to irrelevant background. An imbalance occurs when the ratio of positive to negative samples differs. When training with multiple classes, the varying amounts of data in each class can also lead to imbalance among positive samples. Furthermore, different classes may contain easily classifiable samples. Easily classifiable samples have a large loss value and a significant impact on the network, and should be the focus of attention. Easily classifiable samples usually include background samples; these have a small loss value and a small impact on backpropagation. If the positive and negative samples are imbalanced, with too many negative samples (i.e., too many irrelevant background bounding boxes), this portion of the loss accounts for a large portion of the overall loss. This causes the model's optimization to focus on the larger number of easily classifiable samples, making it difficult for the network to learn effective target features, resulting in network degradation. A reasonable ratio of positive to negative samples can enhance the network's focus on effective target features, which is crucial for improving detection accuracy. A classic method to address the problem of imbalanced positive and negative samples is to use the Focal Loss loss function.

[0019] (2.2) Focal Loss is a modification of the Cross-Entropy Loss (CE Loss) function. Its basic idea is to dynamically control the weights of positive and negative samples and easy / difficult classification samples during training, making the network prioritize samples with target features when updating parameters. Essentially, it adjusts the training weights of misclassified samples through two hyperparameters, increasing or decreasing the loss value for some misclassified samples. This prevents the model training direction from being dominated by a large number of simple negative samples, thus alleviating some of the problem of imbalanced positive and negative samples.

[0020] For binary classification problems, the formula for the binary cross-entropy loss function is as follows:

[0021]

[0022] In the above formula, y′ represents the output of the activation function, with a value between 0 and 1, which is also the predicted value of the sample. y represents the true label value of the sample. For positive samples, i.e., when y = 1, the larger the predicted value, the smaller the loss value; for negative samples, i.e., when y = 0, the smaller the predicted value, the smaller the loss value. CE Loss is relatively slow during training with a large number of simple samples and may not converge to the optimal value.

[0023] To balance the ratio of positive to negative samples, a first hyperparameter α is introduced. The formula is as follows:

[0024]

[0025] In practical applications, α is obtained based on inverse class frequency or cross-validation. When the number of negative samples is less than the number of positive samples, the total loss of positive samples will overwhelm the total loss of a small number of negative samples, and the network will be unable to learn the features of negative samples. If α = 0.25 is taken in this case, it means that the network needs to balance the weight of positive samples, that is, the proportion of positive samples should be smaller than that of negative samples, so that the network can distinguish negative samples.

[0026] To address the imbalance between easy and difficult samples, a second hyperparameter (1-y′) is introduced. γ The final FocalLoss formula is obtained as follows:

[0027]

[0028] When γ > 0, the loss for easily classified samples will be smaller, while the loss for difficult samples will be larger. For positive class samples, the y′ of easily classified samples is greater than 0.5, in which case (1-y′) γ The value of y′ will decrease because the product of numbers between 0 and 1 results in a smaller loss function value. For difficult-to-classify samples, y′ is less than 0.5, leading to a relatively larger loss for that sample. For negative samples, the loss value for difficult-to-classify samples is much larger than that for easy-to-classify samples, causing the model to tend to predict negative samples more accurately. In summary, with a second hyperparameter, the model will pay more attention to difficult-to-classify samples with larger loss function values, reducing the influence of easy-to-classify samples.

[0029] (2.3) Due to YOLOv3's unique prediction mechanism, the network ultimately generates (13×13×3+26×26×3+52×52×3) predicted bounding boxes. Only when the intersection-union ratio (IUU) between the prior bounding box and the ground truth bounding box exceeds a set threshold (typically 0.5) will the prior bounding box adjust its size and position to become a positive sample based on the predicted value; the remaining predicted bounding boxes are negative samples. Therefore, the ratio of positive to negative samples in an image is severely imbalanced. In the YOLOv3 object detection network, the loss function consists of three parts: localization loss, confidence loss, and classification loss. The localization loss function is the MSE Loss function, which calculates the positional bias of the predicted bounding boxes. The classification loss function is the cross-entropy of binary classification, which reduces the classification problem to whether a bounding box belongs to a particular category, thus transforming a multi-class problem into a binary classification problem. Therefore, to address the imbalance between positive and negative samples, the weights of positive samples and the large number of negative samples should be adjusted. Ultimately, this invention decides to improve the MSE Loss function, which serves as the confidence loss, to Focal Loss.

[0030] Preferably, step (3) specifically includes:

[0031] An improvement was made to address the issue of information loss caused by cropping and stretching of input images: the SPP module was embedded in the branch small target detection head.

[0032] (3.1) In convolutional neural networks prior to YOLO, the input image size requirement was fixed. To meet this requirement, the input image needed to be cropped or stretched. These two methods may have different problems: (1) After cropping, some features related to the target will be cropped out, which will inevitably affect feature extraction; (2) After stretching, the target object will be deformed, causing the position information of the target object to be distorted, thus affecting feature extraction. If the size of the target object varies, and the detected PCB defects have different shapes due to their different types, it is impossible to define the image size in advance.

[0033] A typical convolutional neural network (CNN) consists of three main modules: convolution, pooling, and fully connected layers. Convolutional layers perform additive or multiplicative feature calculations on the input image using a sliding window operation, and they are not overly demanding on the size of the input image. They only need to ensure that the image size is larger than the convolution kernel size; that is, convolutional layers can process images of any size and output feature maps of any size. Pooling, since it performs linear operations directly on pixels in the image without any additional parameters, also has no requirements on the size of the input image. However, for fully connected layers, because the internal neurons are fixed, the resulting weight matrix is ​​also fixed after training. Matrix multiplication requires that the two matrices being multiplied have the same row and column dimensions; therefore, a fixed weight matrix requires that the size of the input image to the network also be fixed. In many practical applications, the size of the collected images is not necessarily fixed, which cannot meet the requirement of a fixed input image size for fully connected layers, thus affecting the forward propagation process of the network.

[0034] The SPP structure can improve upon the above situation. When applied to deep convolutional neural networks, the SPP structure has two significant advantages. First, when a fully connected layer exists in the deep convolutional neural network and the input image size is unlimited, applying the SPP structure before the fully connected layer allows the image to output a fixed-size feature map after pooling operations in the SPP structure during forward propagation. This satisfies the requirement of the fully connected layer for a fixed-size input. It also avoids information loss caused by cropping or stretching the input image. Second, the feature map contains information about both the response intensity of certain features and their corresponding spatial location. The SPP structure first extracts features from the feature map obtained from the previous convolutional layer using pooling kernels of different sizes, and then aggregates the extracted features. To a certain extent, multi-scale feature extraction can obtain different spatial location information of the input image, and the final aggregation operation can summarize the separated important contextual features, increasing the multi-scale feature reception capability of the backbone network and thus improving the accuracy of the algorithm's detection.

[0035] In deep neural networks, the last fully connected layer typically maps the learned object features to the sample label space (i.e., classifier or detector). SPP (Spatial Pooling Process) structures are usually applied before the last fully connected layer to extract multi-scale spatial location information, thereby increasing the receptive field of that branch. However, in the YOLOv3 network, the fully connected layer used for mapping is replaced by a 1×1 convolutional layer. The 1×1 convolutional layer has the function of cross-channel information interaction, which can change the number of channels in the feature map, effectively playing a similar role to the fully connected layer. Since the computational cost of convolutional layers is much less than that of fully connected layers, applying the SPP structure (built from pooling operations) before the 1×1 convolutional layer will not change the original classification characteristics of YOLOv3 due to a sharp increase in computational cost. Simultaneously, YOLOv3 models with the SPP module embedded in the main path large target detection head, YOLOv3 models with the SPP module embedded in the branch path small target detection head, and YOLOv3 models with the SPP module embedded in all three detection heads were tested. The results showed that the loss value of the YOLOv3 models with the SPP module embedded was lower than that of the original YOLOv3 models. Comparing the difference between the final training and validation set loss values ​​revealed that the YOLOv3 model with the SPP module embedded in the branch path small target detection head had the lowest loss value, and it was also the only model among the three with a validation set loss value lower than the training set loss value. This indicates that, for the same training set, the YOLOv3 model with the SPP module embedded in the branch path small target detection head has a stronger ability to learn PCB defect features.

[0036] Preferably, step (4) specifically includes:

[0037] To address the issue of insufficient feature extraction capability in the original YOLOv3 model, an improvement was made by integrating the CBAM module in parallel into the YOLOv3 model.

[0038] (4.1) The Channel Attention (CA) module utilizes the relationships between feature channels to generate channel attention feature maps. Since each channel of the feature map is treated as a feature detector, channel attention focuses on determining "what" the target is in the input image. To efficiently compute channel attention, the spatial dimension of the input feature map needs to be compressed. Average pooling is commonly used to aggregate spatial information and map it to the channel dimension, as this operation helps the network effectively learn the range and detail information of the target object. In addition, max pooling can also collect important clues about the target object in another feature space, and infer more refined channel attention through this unique object feature. Max pooling aims to preserve the edge information of the object, while average pooling aims to preserve the texture information of the object. Combining these two types of information helps the network more clearly recognize the detected target.

[0039] When the input feature map passes through the channel attention module, it first undergoes average pooling and max pooling operations, summing the information within the feature map into two distinct spatial vectors, representing the average pooling and max pooling features respectively. The length of these spatial vectors is the channel dimension of the original feature map. These two spatial vectors then flow through a shared network, which consists of a multilayer perceptron with hidden layers. The spatial vectors are first compressed to one-sixteenth of their original length and then expanded back to their original length. This method of extracting channel attention is inspired by SE-Net (Squeeze-and-excitation Networks). Since each channel contains different information and contributes differently to the image task during network training, channel importance needs to be compressed. To some extent, the compressed spatial vectors act as a global receptive field. Subsequently, based on the channel importance after training, the spatial vectors are expanded, which is the second feature labeling. Finally, the two output spatial vectors are summed pixel-by-pixel and then passed through a sigmoid activation function to generate the final channel attention vector.

[0040] (4.2) The Spatial Attention (SA) module utilizes the spatial relationships between features to generate a spatial attention feature map, primarily focusing on image features at the spatial level. Typically, the feature matrix generated by the spatial attention operation is the same size as the input feature map, and the number of channels is compressed to 1, thus focusing on more spatial information. Unlike the channel attention module, the spatial attention module focuses on "where" the target is in the given input image, which is complementary to the channel attention module. When calculating spatial attention, the same max pooling and average pooling operations are used as in the channel attention module. The difference lies in that in the channel attention module, pooling operations are calculated along the H (image height) × W (image width) dimension, while in the spatial attention module, pooling operations are calculated along the channel dimension, because applying pooling operations along the channel axis effectively highlights the region information of the object.

[0041] The feature maps after passing through the channel attention module are also fed into the spatial attention module. First, channel-level pooling is performed, aggregating the spatial information of the feature maps into a plane of size H×W. Two pooling operations are performed to obtain an average pooling plane and a max pooling plane, each with one channel. Next, a concat operation is used to connect the two planes along the channel dimension, generating a feature map with a width of 2. In the spatial attention module, max pooling encodes salient location information, while average pooling encodes global location information of surrounding objects. Connecting the two feature planes fuses these two different types of feature information, enhancing the network's feature representation capability. Then, a convolution operation is applied to the already connected spatial feature planes to reduce their width to 1 again. The feature map at this point now contains the location information that should be emphasized or suppressed. Finally, a sigmoid operation is performed to generate the final spatial attention layer.

[0042] (4.3) In the YOLOv3 object detection network, the backbone network framework is DarkNet53. As shown in the YOLOv3 model structure diagram, it consists of multiple residual units. Each residual block in the residual unit borrows from the original ResNet structure. The main path structure uses 1×1 convolution for dimensionality reduction, then uses 3×3 convolution for feature extraction, and finally adds the extracted feature map to the side path structure. Therefore, the residual block's image feature extraction mainly relies on 3×3 convolution kernels. One advantage of this single feature extraction method is its low computational cost. However, for convolution operations, the feature extraction method is a sliding window, which means that convolution operations treat most features equally. For the PCB defect detection task to be completed, NG involves different types of defects and also has requirements for defect-free images, i.e., background features. Therefore, a single extraction method is not suitable for PCB defect detection. The attention mechanism can focus on the location and feature information of the target in the image, enhancing the model's feature extraction capability. Furthermore, the spatial aggregation operation can strengthen the learned features, thereby helping the model focus on more feature information. To address this, the present invention improves the structure of the original residual block by embedding the CBAM module into the YOLOv3 network.

[0043] Compared with the prior art, the present invention has the following obvious and prominent substantive features and significant technical advancements:

[0044] 1. This invention provides a PCB defect image detection method based on an improved deep learning algorithm. This method addresses the problem of imbalanced positive and negative samples in the dataset, improves the model's detection accuracy for small PCB defects, and enhances the network's feature representation and extraction of important feature information from the detected images. As a result, the PCB defect detection algorithm of this invention has good robustness and generalization ability.

[0045] 2. This invention utilizes an improved model to detect PCB defect images. Compared to the original YOLOv3 model, the new model shows improved sensitivity, indicating that it has better PCB defect detection performance. In experimental comparisons under identical conditions, the model achieves a sensitivity of 97.26% at a threshold of 0.1, demonstrating good performance in PCB defect detection tasks. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a preferred embodiment of the PCB defect image detection method of the present invention.

[0047] Figure 2 This is a PCB defect sample type according to a preferred embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram of the YOLOv3 original target detection network according to a preferred embodiment of the present invention.

[0049] Figure 4 This is a network structure diagram of SPP according to a preferred embodiment of the present invention.

[0050] Figure 5 This is a structural diagram of the channel attention module according to a preferred embodiment of the present invention.

[0051] Figure 6 This is a structural diagram of the spatial attention module according to a preferred embodiment of the present invention.

[0052] Figure 7 This is a diagram of the final improved YOLOv3 model structure according to a preferred embodiment of the present invention.

[0053] Figure 8 These are the ROC curves of the YOLOv3 model and the improved YOLOv3 model in the PCB defect detection experiment of the preferred embodiment of the present invention. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0055] Example 1:

[0056] See Figure 1 A method for detecting PCB defects based on an improved deep learning algorithm is described below.

[0057] (1) Establish an image dataset: Establish a PCB image dataset, acquire various types of PCB defect images, and annotate them. Some PCB defect sample types, such as... Figure 2 As shown.

[0058] (2) Improvement to address the imbalance of positive and negative samples: It is proposed to replace the MSE Loss function in the original YOLOv3 model with the Focal Loss function.

[0059] (3) Improvement is made to address the problem of information loss caused by cropping, stretching and other operations on the input image: embed the SPP module into the branch small target detection head.

[0060] (4) Improvement is made to address the problem of insufficient feature extraction capability of the original YOLOv3 model: the CBAM module is integrated into the YOLOv3 model in parallel.

[0061] (5) PCB defect detection: The dataset of PCB defect images is divided into training set, validation set and test set. The improved YOLOv3 model is trained and tested in sequence to achieve automated and intelligent identification of PCB defect categories.

[0062] This embodiment presents a PCB defect image detection method based on an improved deep learning algorithm. It can be applied to PCB defect images of various shapes and sizes, and can overcome the shortcomings of traditional manual and mechanical inspection methods, such as easy misjudgment, missed detection, low detection accuracy, and damage to the surface structure of the product. It improves work efficiency, reduces manual inspection costs, increases production efficiency, and realizes automatic detection of PCB defects.

[0063] Example 2:

[0064] This embodiment is basically the same as Embodiment 1, with the following differences:

[0065] Step (2) addresses the imbalance between positive and negative samples by proposing an improvement that replaces the MSE Loss function in the original YOLOv3 model with the Focal Loss function. Object detection algorithms can be broadly categorized into two types: one-stage detection and two-stage detection. In fact, the accuracy of one-stage object detection algorithms is lower than that of two-stage algorithms. The main reason for the lower accuracy of one-stage algorithms compared to two-stage algorithms is the imbalance of sample categories. A classic method to address the imbalance between positive and negative samples is the use of the Focal Loss function. Due to the unique prediction mechanism of YOLOv3, the network ultimately generates (13×13×3+26×26×3+52×52×3) predicted boxes. Only when the intersection-union ratio (IUU) between the prior box and the ground truth box is greater than the set threshold (usually 0.5) will the prior box adjust its size and position to become a positive sample based on the predicted value; the remaining predicted boxes are negative samples. Therefore, the ratio of positive to negative samples in an image is severely imbalanced. In the YOLOv3 object detection network, the loss function consists of three parts: localization loss, confidence loss, and classification loss. The localization loss function is the MSE Loss function, which calculates the positional bias of the predicted bounding box. The classification loss function is the cross-entropy of binary classification, which reduces the classification problem to whether a data point belongs to a specific class, thus transforming a multi-class problem into a binary one. Therefore, to address the imbalance between positive and negative samples, the weights of positive samples and the large number of negative samples should be adjusted. Ultimately, this invention improves the MSE Loss function, which serves as the confidence loss, to FocalLoss.

[0066] Step (3) addresses the information loss caused by cropping and stretching of the input image by embedding the SPP module into the branch small target detection head. In deep neural networks, the last fully connected layer often maps the target object features learned by the model to the sample label space (i.e., classifier or detector). The SPP structure is usually applied before the last fully connected layer to extract multi-scale spatial location information, thereby increasing the receptive field of the branch. However, in the YOLOv3 network, the fully connected layer used for mapping is replaced by a 1×1 convolutional layer. The 1×1 convolutional layer has the function of cross-channel information interaction, which can change the number of channels in the feature map, and in fact plays a similar role to the fully connected layer. Since the computational cost of the convolutional layer is much smaller than that of the fully connected layer, applying the SPP structure built by pooling operations before the 1×1 convolutional layer will not change the original classification characteristics of YOLOv3 due to the sharp increase in computational cost. Simultaneously, YOLOv3 models with the SPP module embedded in the main path large target detection head, YOLOv3 models with the SPP module embedded in the branch path small target detection head, and YOLOv3 models with the SPP module embedded in all three detection heads were tested. The results showed that the loss value of the YOLOv3 models with the SPP module embedded was lower than that of the original YOLOv3 models. Comparing the difference between the final training and validation set loss values ​​revealed that the YOLOv3 model with the SPP module embedded in the branch path small target detection head had the lowest loss value, and it was also the only model among the three with a validation set loss value lower than the training set loss value. This indicates that, for the same training set, the YOLOv3 model with the SPP module embedded in the branch path small target detection head has a stronger ability to learn PCB defect features.

[0067] Step (4) addresses the insufficient feature extraction capability of the original YOLOv3 model by integrating the CBAM module in parallel into the YOLOv3 model. In the YOLOv3 object detection network, the backbone network framework is DarkNet53. For example... Figure 3As shown in the YOLOv3 model structure diagram, it consists of multiple residual units. Each residual block in the residual unit borrows from the original ResNet structure. The main path structure uses 1×1 convolution for dimensionality reduction, then uses 3×3 convolution for feature extraction, and finally adds the extracted feature map from the side path structure itself. It is evident that residual block feature extraction mainly relies on 3×3 convolution kernels, and one advantage of this single feature extraction method is its low computational cost. However, for convolution operations, the feature extraction method is a sliding window, indicating that convolution operations treat most features equally. For the PCB defect detection task to be completed, NG involves different types of defects and also has requirements for defect-free images (i.e., background features), so a single extraction method is not suitable for PCB defect detection. Attention mechanisms can focus on the location and feature information of targets in the image, enhancing the model's feature extraction capability. Furthermore, spatial aggregation operations can strengthen the learned features, thus helping the model focus on more feature information. Therefore, this invention improves the structure of the original residual block by embedding the CBAM module into the YOLOv3 network.

[0068] This embodiment utilizes an improved YOLOv3 network structure for PCB defect image detection. This convolutional neural network addresses the issue of imbalanced positive and negative samples in the dataset affecting model generalization and solves the problem of information loss due to cropping, stretching, and other operations on the input image. The model focuses more on the location and feature information of targets in the image, enhancing its feature extraction capabilities. Furthermore, the spatial aggregation operation strengthens the learned features, enabling the model to focus on more feature information and ultimately improving its PCB defect detection capabilities, meeting the requirements for PCB defect detection in industrial production.

[0069] Example 3:

[0070] In this embodiment, see Figure 1 A PCB defect image detection method based on an improved deep learning algorithm is described below:

[0071] (1) Obtain PCB defect images, label the images using LabelImg software, and establish PCB image training and testing datasets. The training dataset consists of 52,000 defect images, including 7,000 images of class NG_AU41, 25,000 images of class NG_GE22, 13,000 images of class NG_GE57, and 7,000 images of other defects such as scratches, voids, and residue. The images are randomly fed into the network for training, and divided into training and validation sets at a ratio of 9.5:0.5, i.e., 49,400 images for training and 2,600 images for validation during training. The testing set consists of two classes, totaling 280,000 images: 100,000 defective (NG) images and 180,000 defect-free (OK) images. The NG images cover all types of defects, and their proportions are roughly similar to those in the training set.

[0072] (2) Improvement to address the imbalance between positive and negative samples: An improvement method is proposed that replace the MSE Loss function in the original YOLOv3 model with the Focal Loss function. Object detection algorithms can be mainly divided into two categories: one-stage detection and two-stage detection. In fact, the recognition accuracy of one-stage object detection algorithms is lower than that of two-stage algorithms. The main reason why the accuracy of one-stage algorithms is lower than that of two-stage algorithms is the imbalance of sample categories. A classic method to solve the problem of imbalance between positive and negative samples is to use the Focal Loss loss function. Due to the special prediction mechanism of YOLOv3, the network will eventually generate (13×13×3+26×26×3+52×52×3) prediction boxes. Among them, only when the intersection-union ratio between the prior box and the ground truth box is greater than the set threshold (usually 0.5) will the prior box adjust its size and position according to the prediction value to become a positive sample. The rest of the prediction boxes are negative samples. Therefore, the ratio of positive samples to negative samples in an image is severely imbalanced. In the YOLOv3 object detection network, the loss function consists of three parts: localization loss, confidence loss, and classification loss. The localization loss function is the MSE Loss function, which calculates the positional bias of the predicted bounding box. The classification loss function is the cross-entropy of binary classification, which reduces the classification problem to whether a data point belongs to a specific class, thus transforming a multi-class problem into a binary one. Therefore, to address the imbalance between positive and negative samples, the weights of positive samples and the large number of negative samples should be adjusted. Ultimately, this invention proposes to improve the MSE Loss function used for confidence loss to Focal Loss.

[0073] (3) Improvements are made to address the information loss caused by cropping and stretching of input images: the SPP module is embedded in the branch small target detection head. In deep neural networks, the last fully connected layer often maps the target object features learned by the model to the sample label space (i.e., classifier or detector). The SPP structure is usually applied before the last fully connected layer to extract multi-scale spatial location information. An SPP structure is as follows: Figure 4 As shown, this increases the receptive field of the branch. However, in the YOLOv3 network, the fully connected layer used for mapping is replaced by a 1×1 convolutional layer. The 1×1 convolutional layer has the function of cross-channel information interaction, which can change the number of channels in the feature map, and in fact plays a similar role to the fully connected layer. Since the computational cost of the convolutional layer is much smaller than that of the fully connected layer, applying the SPP structure built by pooling operations before the 1×1 convolutional layer will not change the original classification characteristics of YOLOv3 due to the sharp increase in computational cost. At the same time, YOLOv3 models with SPP modules embedded in the main road large target detection head, YOLOv3 models with SPP modules embedded in the branch small target detection head, and YOLOv3 models with SPP modules embedded in three detection heads were tested. The results showed that the loss value of the YOLOv3 model with SPP module was smaller than that of the original YOLOv3 model. Comparing the difference in loss values ​​between the training and validation sets reveals that the YOLOv3 model, which embeds the SPP module into the branch small target detection head, has the lowest loss value. Furthermore, it is the only model among the three with a lower validation set loss value than its training set loss value. This indicates that, for the same training set, the YOLOv3 model with the SPP module embedded in the branch small target detection head has a stronger ability to learn PCB defect features.

[0074] (4) To address the insufficient feature extraction capability of the original YOLOv3 model, an improvement was made: the CBAM module was integrated into the YOLOv3 model in parallel. In the YOLOv3 object detection network, the backbone network framework is DarkNet53. Figure 3As shown in the YOLOv3 model structure diagram, it consists of multiple residual units. Each residual block in the residual unit borrows from the original ResNet structure. The main path structure uses 1×1 convolution for dimensionality reduction, then uses 3×3 convolution for feature extraction, and finally adds the extracted feature map to itself in the side path structure. It is evident that residual block feature extraction mainly relies on 3×3 convolution kernels, and one advantage of this single feature extraction method is its low computational cost. However, for convolution operations, the feature extraction method is a sliding window, meaning that convolution operations treat most features equally. For the PCB defect detection task to be completed, NG involves different types of defects and also has requirements for defect-free images (i.e., background features), so a single extraction method is not suitable for PCB defect detection. Attention mechanisms can focus on the location and feature information of targets in the image, enhancing the model's feature extraction capability. Furthermore, spatial aggregation operations can strengthen the learned features, thus helping the model focus on more feature information. Therefore, this invention improves the structure of the original residual block by embedding the CBAM module into the YOLOv3 network.

[0075] This paper proposes an improvement by replacing the MSE Loss function in the original YOLOv3 model with the Focal Loss function to address the bias in prediction results caused by the imbalance of positive and negative samples in single-stage object detection algorithms. It also proposes an improvement by embedding a spatial pyramid pooling structure before the 1×1 convolutional layer of the YOLOv3 detection head to extract multi-scale features of PCB defects using multiple pooling kernels. Finally, it proposes an improvement by applying channel attention and spatial attention modules to the YOLOv3 backbone network, with the structures of the channel attention and spatial attention modules as follows: Figure 5 and Figure 6 As shown, this allows the model to focus more on defect information in the image, thereby improving the overall feature extraction capability of the model. Based on the above improvements, this invention integrates the above methods, embedding the SPP module into the branch small target detection head based on the Focal Loss loss function, and incorporating the CBAM module in parallel into the YOLOv3 model, thus proposing an improved YOLOv3 model. The improved model structure is as follows. Figure 7 As shown.

[0076] (5) The PCB image set was divided into training, validation, and test sets, and trained and tested using the improved YOLOv3 model. The performance of the model, i.e., its generalization ability, needs to be measured by a specific metric. In deep learning, precision and recall are commonly used to measure a model's classification ability. However, in this invention, to mitigate the problem of imbalanced samples, specificity and sensitivity are used as evaluation criteria. These metrics are calculated based on the confusion matrix in deep learning, as shown in Table 1.

[0077] Table 1 Confusion Matrix

[0078]

[0079] The terms of the confusion matrix are interpreted as follows:

[0080] (1) TP (True Instance): The number of cases that are actually positive and predicted to be positive.

[0081] (2) FP (False Positive): The number of cases that are actually negative but are predicted to be positive;

[0082] (3) FN (False Negative): The number of cases that are actually positive but are predicted to be negative;

[0083] (4) TN (True Negative Examples): The number of instances that are actually negative but are predicted to be negative.

[0084] Specificity refers to negative samples; it represents the probability of a sample that is actually negative being predicted as negative. In this invention, it represents the proportion of correctly predicted OK images out of all OK images. The formula is as follows:

[0085]

[0086] Sensitivity is defined the same way as recall; it is the probability of a sample being predicted as positive among samples that are actually positive. The formula is as follows:

[0087]

[0088] To facilitate statistical data collection and comparison of experimental results, during the prediction process, if the detection result is OK, it is classified as a negative class; if the detection result is not OK, the other four categories are classified as NG, i.e., positive classes. Therefore, the detection requirement simply involves classifying the PCB image into OK and NG categories.

[0089] The improved YOLOv3 network was trained on the collected image dataset and then tested. Table 2 shows the detection results of the improved YOLOv3 model and the original YOLOv3 model when the classification threshold was set to 0.5. Table 2 shows that the specificity of the improved YOLOv3 model decreased slightly, but remained within an acceptable range. Its sensitivity improved significantly, reaching 95.78%. The inference speed of the improved YOLOv3 model decreased compared to the original YOLOv3 model, which is due to the embedding of the SPP and CBAM modules. Table 3 shows the sensitivity of the improved YOLOv3 model for NG images and the specificity for OK images when the confidence threshold was set from 0.1 to 0.6.

[0090] Table 2. Results of each indicator for the two models.

[0091]

[0092] Table 3. Performance of the improved YOLOv3 model in PCB defect detection.

[0093]

[0094] The ROC curves of the YOLOv3 model and the improved YOLOv3 model are as follows: Figure 8 As shown in Table 3, when the confidence threshold is 0.1, the true probability of the improved YOLOv3 model curve reaches 97.26%, and... Figure 8 It can be seen that the false positive rate of the YOLOv3 model is lower than that of the improved YOLOv3 model at this point. When the true positive rate of the improved YOLOv3 model is between 75% and 87.5%, the corresponding confidence threshold is between 0.5 and 0.6, and the false positive rate of the YOLOv3 model is higher than that of the improved YOLOv3 model. When the confidence threshold is low, the improved YOLOv3 model shows a slight decrease in specificity for OK images compared to the YOLOv3 model, but it is still within an acceptable range. For NG images, especially when the threshold is 0.5, the improved YOLOv3 model shows a significant improvement. Considering the low applicability of confidence thresholds of 0.6 and above, the improved YOLOv3 model performs better than the YOLOv3 model in terms of sensitivity to NG images when the threshold is 0.5 or below. The figure also shows that the curve of the improved YOLOv3 model basically surrounds the YOLOv3 model, indicating that the improved YOLOv3 model is better than the YOLOv3 model in terms of PCB defect detection capability.

[0095] The above embodiments are based on an improved deep learning algorithm for PCB defect image detection. Addressing the shortcomings of the YOLOv3 deep convolutional neural network model, such as imbalanced positive and negative samples, influencing the network's feature representation of small PCB defects, and poor extraction of important image features, this invention proposes an improvement by replacing the MSE Loss function in the original YOLOv3 model with the Focal Loss function. This dynamically controls the weights of positive and negative samples and samples with varying difficulty levels. Simultaneously, this invention embeds the SPP module into the branch small target detection head and integrates the CBAM module in parallel into the YOLOv3 model, allowing the model to focus more on defect information in the image, thereby improving the overall feature extraction capability. This method can detect different types of PCB defect images. Compared to the original YOLOv3 model, it shows improved sensitivity and other indicators, demonstrating better performance in PCB defect detection tasks, reducing false negative and false positive rates, and better achieving automated and intelligent PCB defect detection.

[0096] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent procedural transformations made based on the content of the present invention specification and drawings, or applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A PCB defect image detection method based on an improved deep learning algorithm, characterized in that, The specific steps are as follows: (1) Establish image dataset: Establish PCB image dataset, obtain images of various types of PCB defects, and annotate them; (2) In the YOLOv3 object detection network, the loss function is divided into three parts: localization loss, confidence loss and classification loss; the MSE Loss function, which is used as the confidence loss, is improved to the Focal Loss function; (3) Embed the SPP module into the branch small target detection head; The SPP module is embedded only in the branch small target detection head of the YOLOv3 model; in the YOLOv3 network, the fully connected layer used for mapping is replaced by a 1×1 convolutional layer; the SPP structure built by pooling operations is applied before the 1×1 convolutional layer; (4) Integrate the CBAM module in parallel into the YOLOv3 model; In the YOLOv3 object detection network, the backbone network framework is DarkNet53, and the YOLOv3 model is composed of multiple residual units; Embed the CBAM module into the residual block of the residual unit; (5) Divide the PCB defect image dataset into training set, validation set and test set, and train and test the improved YOLOv3 model in sequence to finally realize the automated and intelligent identification of PCB defect categories. 2.The PCB defect image detection method based on improved deep learning algorithm according to claim 1, characterized in that, Step (1) specifically includes: (1.1) Professionals collect and organize the PCB images produced, and finally obtain a defective image dataset; then, the organized PCB image dataset is expanded using data augmentation to improve the generalization ability of the deep neural network model. (1.2) Based on the PCB image dataset prepared in step (1.1), the target detection open source software LalbelImg is used to annotate the data and generate the corresponding XML format label file. 3.The PCB defect image detection method based on improved deep learning algorithm according to claim 1, characterized in that, Step (2) specifically includes: Due to YOLOv3's unique prediction mechanism, the network will eventually generate (13×13×3+26×26×3+52×52×3) predicted boxes. Among them, only when the intersection-union ratio between the prior box and the ground truth box is greater than the set threshold of 0.5 will the prior box adjust its size and position according to the predicted value to become a positive sample, and the rest of the predicted boxes are negative samples. 4.The PCB defect image detection method based on improved deep learning algorithm according to claim 1, characterized in that, Step (3) specifically includes: In deep neural networks, the last fully connected layer maps the target object features learned by the model to the sample label space, i.e., the classifier or detector; the SPP structure is applied before the last fully connected layer to extract multi-scale spatial location information, thereby increasing the receptive field of this branch. 5.The PCB defect image detection method based on improved deep learning algorithm according to claim 1, characterized in that, Step (4) specifically includes: For each residual block in the residual unit, the main path structure uses 1×1 convolution for dimensionality reduction, then uses 3×3 convolution to extract features, and finally adds the extracted feature map to itself in the bypass structure.

Images