BGA defect detection method based on multi-scale feature fusion attention mechanism

Abstract

The invention discloses a BGA defect detection method based on a multi-scale feature fusion attention mechanism, and the method specifically comprises the steps: obtaining a BGA defect image, and marking the defect image; designing a multi-scale feature fusion backbone network MFFBN, introducing an attention mechanism and a more efficient detection head EGDH at the same time, and proposing a better model YOLO-MCE based on YOLO11; training the improved algorithm model by adopting a BGA defect image training set; and transmitting a defect image test set into the trained model, recording a detection result and evaluating the performance of the model. According to the method, the effectiveness of BGA surface solder ball defect detection under a BGA solder ball defect scene and the superiority in methods in the same field are ensured.

Description

Technical Field

[0001] The present invention belongs to the field of quality inspection in the electronic information industry, and particularly relates to a BGA defect detection method based on a multi-scale feature fusion attention mechanism. Background Art

[0002] With the continuous development of the electronic information manufacturing industry, the Ball Grid Array (BGA) packaging technology has been widely used in integrated circuits (ICs) and plays an important role in the electronics industry. BGA is an efficient chip packaging technology that realizes electrical connection with the circuit board by arranging an array of solder balls at the bottom of the chip, and has the advantages of high pin density, good electrothermal performance, and miniaturization. The quality of BGA directly affects the performance and reliability of electronic products. However, during the manufacturing process, due to the influence of many factors such as manufacturing processes and environment, minute defects will occur on the surface solder balls. Moreover, the diameter range of the surface solder balls of BGA is usually 0.1 - 1.5 mm, and the solder ball pitch is usually 0.25 - 1.5 mm, making detection very difficult. Therefore, the quality inspection of BGA is a major link in ensuring product quality.

[0003] The existing BGA solder ball defect detection methods mainly include manual visual inspection, Automatic Optical Inspection (AOI), and X-ray inspection. Manual visual inspection is the most common detection method, but this method has low efficiency by observing with the naked eye, and the results are easily affected subjectively, making it difficult to meet production requirements; Automatic Optical Inspection has a higher degree of automation. It collects surface images through a high-resolution camera, uses image processing algorithms to extract various features, and realizes defect detection, effectively improving the detection accuracy and efficiency. However, the equipment is expensive, and it is difficult to effectively capture minute defects; X-ray inspection is to pass a high-energy electromagnetic wave through an object. Different parts of the object will absorb or scatter rays of different intensities to form a transmission image, and defects are analyzed according to the image. Different from other traditional detection methods, X-ray inspection can detect surface and internal defects. However, the purchase and maintenance costs of X-ray equipment are high, and it is greatly affected by the surface smoothness and reflectivity effect when detecting surface defects, and the imaging process is complex.

[0004] Benefiting from the rapid development of artificial intelligence, deep learning is increasingly widely used in the field of defect detection, and the machine vision detection method has become an important detection means at present. Different from the traditional defect detection methods, using deep learning for defect inspection can automatically process a large amount of data, be more efficient, and reduce labor costs. Moreover, the deep learning model has strong adaptability and can quickly learn new defect patterns or changes by training different data sets, and can automatically detect tiny defects such as scratches, cracks, and poor soldering, thus effectively improving the detection accuracy and production efficiency, while avoiding the disadvantages of manual adjustment or re-design required by traditional detection methods. At present, object detection algorithms are mainly divided into single-stage and two-stage algorithms. The single-stage object detection algorithm directly realizes object classification and bounding box regression through a single neural network, and its characteristics are fast speed but low accuracy. Classic single-stage algorithms include the YOLO series and (Single Shot Detector, SSD). The two-stage object detection algorithm is usually divided into two steps: First, generate candidate regions; Second, classify and perform bounding box regression on the candidate regions generated in the first stage. This method has high accuracy, but the detection speed is slow. For example, the Fast R-CNN, SPPNet, and Faster R-CNN algorithms are all classic two-stage object detection algorithms. Although deep learning methods have made remarkable progress in the detection of surface defects of electronic products, there is still a lack of effective deep learning methods for the detection of BGA surface defects. The BGA surface defect features are very similar, difficult to distinguish, and the number is huge, up to thousands at most. The traditional detection algorithms are inefficient and have a high false detection rate. Therefore, in view of this gap, it is necessary to propose a deep learning defect detection method for BGA defect features to meet the detection requirements of high precision and high efficiency. Summary of the Invention

[0005] To solve the problems of weak recognition ability of tiny surface solder ball defects in the existing BGA solder ball defect detection method, such as false detection and missed detection, the present invention provides a BGA defect detection method based on a multi-scale feature fusion attention mechanism.

[0006] A BGA defect detection method based on a multi-scale feature fusion attention mechanism of the present invention includes the following steps:

[0007] Step 1: Obtain a BGA defect image and annotate the defect image.

[0008] Step 2: Design a multi-scale feature fusion backbone network MFFBN, introduce the Faster Block and the attention mechanism to improve the C3k2 module, and at the same time propose a more efficient detection head EGDH, and propose a better model YOLO-MCE based on YOLO11.

[0009] Step 3: Use the BGA defect image training set to train the improved algorithm model.

[0010] Step 4: Input the defect image test set into the trained model, record the detection results and evaluate the model performance.

[0011] Further, Step 1 is specifically as follows: Obtain the BGA solder ball defect dataset, use the Labelimg tool to annotate the dataset images, and generate label files in TXT format.

[0012] Further, the YOLO-MCE model is specifically as follows:

[0013] The dataset is input into the backbone layer structure. By designing a Multi-scale Feature Fusion Backbone Network (MFFBN), a multi-scale feature fusion module MFFM in the backbone network combines the C3k2 module to extract the feature information of the defect image. At the same time, the Spatial Pyramid Pooling Fast (SPFF) and Cross-Stage Partial Spatial Attention (C2PSA) modules in the original structure are introduced to improve the detection performance of multi-scale targets, and the results are transmitted to the neck structure; in the neck structure, the Faster Block and eSE attention mechanism are introduced to improve C3k2. Different levels of features are aggregated through bottom-up and top-down paths to enhance the model's ability to capture local features in complex scenarios; finally, in the detection head, aiming at the problems of high false detection rate and missed detection rate, an Effective Detection Head (EGDH) is designed by combining parameter sharing and Group Convolution (Gconv), and a decoupled design is adopted to output the defect classification and location information in the detection image.

[0014] Further, Step 3 is specifically as follows: After improving the YOLO11 model, train the YOLO-MCE model: Set the size of the input image to 640×640 pixels, set both the initial learning rate and the final learning rate to 0.01, use the Mosaic method for image data augmentation, and in the feature extraction stage, the number of iterations is 300 epochs, and the batch size is set to 32.

[0015] Further, in Step 4, the model performance metric is the Mean Average Precision (mAP), which comprehensively evaluates the detection ability of the model in multi-classification and can comprehensively and accurately evaluate the overall performance of the object detection model. Specifically:

[0016]

[0017] AP = ∫ 0 1 P(r)dr (3)

[0018]

[0019] where Percision refers to the proportion of actual samples among all the targets detected as positive samples, Recall refers to the proportion of targets detected as positive samples among all the actual positive samples, where T P represents the number of true positive samples predicted as positive samples, F P represents the number of negative samples misjudged as positive samples, F N represents the number of negative samples predicted as negative samples; AP represents the average precision of a single classification, P(r) represents the function image enclosed by Recall and Percision, n represents the number of defect categories; AP i represents the average classification precision of the i-th category.

[0020] The beneficial technical effects of the present invention are as follows:

[0021] The YOLO-MCE algorithm proposed by the present invention for solving the problem of detecting solder ball defects on the surface of BGA in complex scenarios effectively solves the problem of insufficient ability of the original feature network to extract features of tiny solder ball defects on BGA by designing a multi-scale feature fusion attention backbone network. At the same time, the neck network C3k2 module is improved, and the Faster Block and eSE attention mechanisms are introduced therein, effectively improving the ability to capture local features in similar scenarios. In addition, to reduce the overlapping occlusion phenomenon between prediction boxes, parameter sharing and GConv are introduced, reducing the false detection and missed detection rates. Compared with mainstream object detection algorithms on the self-built dataset BGA Defect dataset, the superiority and effectiveness of the YOLO-MCE model in the process of BGA defect detection are verified. The present invention ensures the effectiveness of the problem of detecting solder ball defects on BGA in complex scenarios and the superiority among methods in the same field. Description of the Drawings

[0022] Figure 1 is the framework diagram of the YOLO-MCE model proposed by the present invention.

[0023] Figure 2 is the structural diagram of the multi-scale feature fusion module MFFM.

[0024] Figure 3 is the core EC3k structural diagram in the improved CFeSE module.

[0025] Figure 4 It is the structure diagram of EDFH.

[0026] Figure 5 It is the detection effect diagram of the experimental BGA Defect dataset ((a) damage; (b) oxidation; (c) burr; (d) bridging; (e) excessive solder; (f) insufficient solder). Specific implementation manners

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] A BGA defect detection method based on a multi-scale feature fusion attention mechanism of the present invention includes the following steps:

[0029] Step 1: Obtain a BGA defect image and label the defect image.

[0030] Obtain a BGA solder ball defect dataset, use the Labelimg tool to label the dataset images, and generate label files in TXT format.

[0031] Step 2: Design a multi-scale feature fusion backbone network MFFBN, introduce the Faster Block and the attention mechanism to improve the C3k2 module, and at the same time propose a more efficient detection head EGDH, and propose a better YOLO-MCE model based on YOLO11.

[0032] The framework of the YOLO-MCE model is as Figure 1 shown, specifically:

[0033] The dataset is input into the backbone layer structure. By designing a multi-scale feature fusion backbone MFFBN, a multi-scale feature fusion module MFFM in the backbone network combines the C3k2 module to extract the feature information of the defect image. At the same time, the SPFF layer and the C2PSA module in the original structure are introduced to improve the detection performance of multi-scale targets, and the results are transmitted to the neck structure; in the neck structure, the Faster Block and the eSE attention mechanism are introduced to improve the C3k2. Different levels of features are aggregated through the bottom-up and top-down paths to improve the model's ability to capture local features of complex scenes; finally, in the detection head, in view of the high false detection rate and missed detection rate, a more efficient detection head EGDH is designed by combining parameter sharing and GConv, and a decoupled design is adopted to output the defect classification and position information in the detection image.

[0034] The optimization is specifically:

[0035] (1) A multi-scale feature fusion backbone network MFFBN with the multi-scale feature fusion module (MFFM) as the core is designed. The structure of the MFFM module is as shown in Figure 2 . First, multi-scale cross-layer fusion is achieved. Context information is captured through convolution operations, and detailed features and semantic information are integrated through connection operations. Then, the concept of the ECA attention mechanism is combined to achieve dynamic channel weight adjustment. Specifically, multi-scale cross-layer fusion is realized through 1×1 convolution, 3×3 convolution, and connection operations. The main function of the convolution operation is to extract local features and compress redundant information. At the same time, convolutions of different scales enable the module to effectively model context dependencies. After multiple convolution operations, the connection operation is used to connect the three feature maps along the channel dimension, and finally, a 1×1 convolution operation with batch normalization and the SiLU activation layer is performed. Multi-scale cross-layer fusion effectively enhances the model's ability to express features of different scales, making it more adaptable to the detection of small target defects that are easily overlooked in complex scenarios. To ensure the model efficiency, the ECA attention mechanism is introduced into the module, and the computational cost is reduced through one-dimensional convolution. The concept of the ECA attention mechanism is to dynamically adjust the channel weights, which can effectively suppress background noise. This module can effectively improve the problem of insufficient sensitivity of the original model to small targets.

[0036] (2) A more effective EC3k module is designed based on C3k and introduced into the C3k2 module to construct a new module CFeSE. The structure is as shown in Figure 3 . To solve the problem of easy confusion of similar defect features in detection, the core module FasterNet Block in FasterNet is introduced to enhance the ability to extract local features using its grouped convolution and multi-layer perceptron. At the same time, the eSE attention mechanism is introduced to improve the model's sensitivity and focusing ability to key features. Specifically, the core module of EC3k consists of multiple convolutional layers, skip connections, and the eSE attention mechanism. First, grouped convolution divides the ordinary convolution into multiple groups to calculate and reweight the effective regions, reducing the computational amount and improving the efficiency. Second, the channel attention mechanism is introduced to dynamically adjust the weights of each channel, enabling the network to focus more on important features. The use of residual connections further optimizes the information flow, avoiding the problem of gradient disappearance and accelerating the training process.

[0037] (3) A more effective detection head EGDH is designed to enhance the local optimization ability, reduce the overlapping occlusion phenomenon between prediction boxes, and reduce the false detection and missed detection rates. The structure is as shown in Figure 4As shown. Since the BGA solder balls are densely distributed and the spacing between the solder balls is narrow, there are prone to prediction box overlap and occlusion phenomena when predicting defect types. To solve this problem, parameter sharing technology and group convolution are introduced in EGDH. Specifically, after the feature map is input, it undergoes two group convolutions. The input channels are divided into multiple groups, and each group is calculated independently to reduce the computational burden while retaining the correlation between the targets. At the same time, in this part, when dealing with multiple similar or overlapping targets, the same convolutional kernel is reused to achieve parameter sharing, effectively reducing redundant features, thereby improving the efficiency and accuracy of target recognition. Finally, classification and regression tasks are achieved through two independent branches: the binary cross-entropy (BCE) loss is used to achieve the classification task to improve the accuracy of class judgment; the distribution focal loss (DFL) is combined with the complete intersection over union (CIOU) loss to achieve the regression task, optimizing the prediction box localization accuracy and the model robustness.

[0038] Step 3: Use the BGA defect image training set to train the improved algorithm model.

[0039] After improving the YOLO11 model, train the YOLO-MCE model: the size of the input image is set to 640×640 pixels, the initial learning rate and the final learning rate are both set to 0.01, the Mosaic method is used for image data augmentation, and in the feature extraction stage, the number of iterations is 300 epochs, and the batch size is set to 32.

[0040] Step 4: Pass the defect image test set into the trained model, record the detection results and evaluate the model performance.

[0041] The model performance metric is the mean average precision (mAP), which comprehensively evaluates the detection ability of the model in multi-classification and can comprehensively and accurately evaluate the overall performance of the object detection model. Specifically:

[0042]

[0043] AP = ∫ 0 1 P(r)dr (3)

[0044]

[0045] In the formula: Percision refers to the proportion of actual samples among all the targets detected as positive samples, and Recall refers to the proportion of the targets detected as positive samples among all the actual positive samples, where T P represents the number of true positive samples predicted as positive samples, and F P represents the number of negative samples misjudged as positive samples, and F NIndicates the number of negative samples predicted as negative samples; AP represents the average precision of a single classification, P(r) represents the function image enclosed by Recall and Percision, n represents the number of defect categories; AP i Indicates the average classification precision of the i-th category.

[0046] Through testing on the self-built BGA solder ball dataset BGA Defect dataset and the publicly available PCB dataset PCB Defectdataset, the method proposed in the present invention is compared with the fully supervised object detection algorithm. The comparison results of this method on the BGA dataset are shown in Table 1, and the comparison results on the PCB dataset are shown in Table 2. The detection effect of this method is as Figure 5 shown.

[0047] Table 1 Performance comparison experiment on BGA dataset

[0048]

[0049] Table 2 Performance comparison experiment on BGA dataset

[0050]

[0051] It can be seen that the method of the present invention is more suitable for the BGA solder ball defect detection scenario.

Claims

1. A BGA defect detection method based on multi-scale feature fusion attention mechanism, characterized in that: The following steps are involved: Step 1: Obtain BGA defect image and mark the defect image; Step 2: Optimize the BGA defect detection model based on the YOLO11 model and improve it to the YOLO-MCE model; Step 3: Use the BGA defect image training set to train the improved algorithm model; Step 4: Pass the defect image test set into the trained model, record the detection results and evaluate the model performance.

2. According to claim 1, a BGA defect detection method based on multi-scale feature fusion attention mechanism is characterized in that: The step 1 specifically includes: obtaining a BGA solder ball defect dataset, using the Labelimg tool to annotate the dataset image, and generating a label file in TXT format.

3. According to claim 1, a BGA defect detection method based on multi-scale feature fusion attention mechanism is characterized in that: The YOLO-MCE model is as follows: The dataset is input into the backbone layer structure. A multi-scale feature fusion backbone MFFBN is designed. A multi-scale feature fusion module MFFM in the backbone network is combined with the C3k2 module to extract the feature information of the defect image. At the same time, the SPFF layer and C2PSA module in the original structure are introduced to improve the detection performance of multi-scale targets, and the results are transmitted to the neck structure. In the neck structure, Faster Block and eSE attention mechanisms are introduced to improve C3k2. Features at different levels are aggregated through bottom-up and top-down paths to enhance the model's ability to capture local features in complex scenes. Finally, in the detection head, a more efficient detection head EGDH is designed to address the problems of high false detection rate and missed detection rate by combining parameter sharing and group convolution. A decoupled design is adopted to output the defect classification and location information in the detection image.

4. A BGA defect detection method based on multi-scale feature fusion attention mechanism according to claim 1, characterized in that: The step 3 is specifically as follows: after improving the YOLO11 model, the YOLO-MCE model is trained: the size of the input image is set to 640×640 pixels, the initial learning rate and the final learning rate are both set to 0.01, the Mosaic method is used for image data enhancement, and in the feature extraction stage, the number of iterations is 300 cycles and the batch size is set to 32.

5. A BGA defect detection method based on multi-scale feature fusion attention mechanism according to claim 1, characterized in that: In step 4, the model performance indicator is the mean average precision mAP, specifically: Where: Percision refers to the proportion of actual samples among all targets detected as positive samples, Recall refers to the proportion of targets detected as positive samples among all actual positive samples, where T P Indicates the number of true positive samples predicted as positive samples, F P Indicates the number of negative samples misclassified as positive samples, F N It indicates the number of negative samples predicted as negative samples; AP indicates the average precision of a single classification, P(r) indicates the function image surrounded by Recall and Percision, and n indicates the number of defect categories; AP i Represents the average classification accuracy of the i-th category.

Images