A lightweight chip pad alignment detection method based on YOLOv5

Abstract

This paper discloses a lightweight chip pad alignment detection method based on YOLOv5, comprising: 1) acquisition: using an industrial camera to capture sample chip pad images; 2) preprocessing; 3) segmentation; 4) model construction; and 5) detection. This method not only improves the network's detection accuracy for chip pads but also enables the detection of more chip pads. With fewer network parameters, a smaller model size, and high detection accuracy, this method is particularly suitable for deploying inference in actual production of embedded devices.

Description

Technical Field

[0001] The present invention relates to computer vision target detection technology, and in particular to a lightweight chip pad alignment detection method based on YOLOv5. Background Art

[0002] With the advancement of semiconductor technology and integrated circuits, chips are becoming increasingly complex and smaller in size. Chip pads are becoming more numerous and smaller in size, further increasing the difficulty of chip alignment testing. Power-on testing is a crucial method for determining chip yield. By conducting power-on testing on chip pads, it is possible to determine whether the chip meets product standards and classify the chip for subsequent packaging and other processes. Therefore, chip alignment testing is crucial to the entire chip manufacturing process.

[0003] Probes are used to test contact with the solder joints on the chip's pads to measure the chip's capacity and other performance. This operation typically requires manual alignment by workers under a microscope, which is difficult to control and slow. During the alignment process, the force of the probe contact is crucial. If the probe contact is too light, the test data may be inaccurate, making it impossible to obtain precise results. However, if the contact is too strong, it may damage the circuitry on the chip and may even cause the solder joint probe to deform, affecting the subsequent progress of the entire test. Due to these problems, traditional manual alignment inspection methods can no longer meet the high-precision and high-efficiency requirements of today's industrial assembly lines.

[0004] The development of computer vision has provided two mainstream target detection algorithms for industrial inspection: one is a single-stage detection algorithm represented by the SSD and YOLO series, which extracts the features of the target to be detected through a convolutional neural network and then predicts the classification and positioning of the target; the other is a two-stage detection algorithm represented by Faster R-CNN, which first generates candidate regions and then uses a convolutional neural network to classify samples to obtain the final result.

[0005] The alignment detection algorithm in the industrial assembly line testing process must ensure both accuracy and speed. Although the two-stage detection algorithm has good detection accuracy, it cannot meet the requirements of real-time detection due to its large number of network parameters and slow detection speed. The single-stage detection algorithm has the advantages of fast detection speed and high real-time performance, but its detection accuracy is slightly lower than that of the two-stage algorithm. Among them, the YOLO series of algorithms in the single-stage target detection algorithm currently achieves a good balance between detection accuracy and network model size. The YOLOv5 algorithm proposed in 2020 reduces some network parameters through compression processing techniques on the number of network channels, while keeping the network's detection accuracy at a high level. However, since the detection targets of different detection tasks will be different during use, the YOLOv5 algorithm has room for optimization in the detection task of chip pads, and the above algorithms still have the problem of too large models and difficulty in deployment and reasoning on the embedded side. Summary of the Invention

[0006] The present invention addresses the shortcomings of existing technologies by providing a lightweight chip pad alignment detection method based on YOLOv5. This method not only improves the network's detection accuracy for chip pads but also enables the detection of more chip pads. With fewer network parameters, a smaller model size, and high detection accuracy, this method is particularly suitable for deploying inference in actual production on embedded devices.

[0007] The technical solution for achieving the purpose of the present invention is:

[0008] A lightweight chip pad alignment detection method based on YOLOv5 includes the following steps:

[0009] 1) Acquisition: Use industrial cameras to capture chip pad sample images;

[0010] 2) Preprocessing: Data augmentation is performed on the acquired images. LabelImg is used to label the augmented images. The wafer solder joints that require probe alignment during inspection are named rig, the wafer solder joints that do not require probe alignment are named wro, and the probes for contact testing are named Probe. This ultimately yields a dataset. Data augmentation involves cropping the original 6112*3440 images to 2030*1500 pixels. This reduces redundant background and shortens network training time.

[0011] 3) Partition: Divide the dataset obtained in step 2) into a training set and a validation set in a ratio of 9:1;

[0012] 4) Model Construction: Build a chip pad alignment detection model based on lightweight YOLOv5, and load the dataset into the chip pad alignment detection model based on lightweight YOLOv5 for training to obtain target detection results, including:

[0013] 4-1) Based on the YOLOv5 chip pad alignment detection model, the multi-scale fusion and detection parts are optimized to reduce network parameters while enhancing the detection accuracy of the target. Specifically:

[0014] 4-1-1) Trimming the last layer C3 module of the YOLOv5 backbone network can effectively reduce network parameters and speed up the deployment and inference of the model on embedded devices;

[0015] 4-1-2) Using upsampling to double the resolution of multi-size detection heads can effectively enhance the detection of small targets;

[0016] 4-2) Replace Bottleneck with Ghostneck in the backbone network feature extraction module C3;

[0017] 4-3) Introducing the CBAM attention mechanism, which uses weighted processing of information in both channel and spatial dimensions to make the network more focused on useful information;

[0018] 4-4) Introducing parallel dilated convolution into the SPPF module to form a context space module;

[0019] 4-5) Replace the CIOU loss function with the SIOU loss function, including:

[0020] 4-5-1) Angle loss:

[0021]

[0022]

[0023]

[0024] Among them, σ is the distance between the center point of the real box and the predicted box, c h is the height difference between the center point of the real box and the predicted box, Represents the x and y coordinates of the center values of the real box and the predicted box respectively;

[0025] 4-5-2) Distance loss:

[0026]

[0027]

[0028]

[0029] γ=2-Λ,

[0030] Among them, (c w ,c h ) is the width and height of the minimum bounding rectangle of the real box and the predicted box;

[0031] 4-5-3) Shape loss:

[0032]

[0033]

[0034]

[0035] Among them, (w,h) and (w gt ,h gt ) are the width and height of the predicted box and the real box respectively, and θ represents the degree of attention to shape loss;

[0036] 4-5-4) IoU loss:

[0037]

[0038] Among them, IoU represents the intersection-over-union ratio between the real image frame and the predicted frame;

[0039] 5) The trained deep learning model, namely the chip pad alignment detection model based on lightweight YOLOv5, is applied to chip pad alignment detection in industrial assembly lines.

[0040] This technical solution significantly reduces the network parameters and computational complexity of the model while improving the model detection accuracy.

[0041] This method can not only improve the network's detection accuracy of chip pads, but also detect more chip pads. This method has fewer network parameters, a smaller model, and higher detection accuracy, making it more suitable for deploying reasoning in actual production of embedded devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a method flow chart of an embodiment;

[0043] Figure 2 Schematic diagram of multi-scale fusion and detection structure improvement in the embodiment;

[0044] Figure 3 Schematic diagram of Ghost convolution module in the embodiment;

[0045] Figure 4 Schematic diagram of the CBAM attention module in the embodiment;

[0046] Figure 5 Schematic diagram of the context space module structure of the embodiment;

[0047] Figure 6 A schematic diagram of the test results. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the present invention is not limited thereto.

[0049] Example:

[0050] Reference Figure 1 , a lightweight chip pad alignment detection method based on YOLOv5, comprising the following steps:

[0051] 1) Acquisition: Use industrial cameras to capture chip pad sample images;

[0052] 2) Preprocessing: Data enhancement is performed on the collected images, and LabelImg is used to label the samples of the data-enhanced images. The wafer solder joints that require solder joint probe alignment during detection are named rig, the wafer solder joints that do not require solder joint probe alignment during detection are named wro, and the solder joint probes that undergo contact testing during detection are named Probe. Finally, the data set is obtained. Data enhancement is to crop the original 6112*3440 image into a 2030*1500 image. This can reduce redundant background and reduce network training time. In this case, the actual solder joints in the 6112*3440 resolution wafer pad image usually only occupy a small part of the entire image. At this time, if the entire image is put into the network for training, the training time will increase and the network detection accuracy will not be good. Cropping the image to a 2030*1500 resolution image that only contains wafer solder joints and pad probes can reduce background redundant information while expanding the originally small-sized target to be inspected into a small target;

[0053] 3) Partition: Divide the dataset obtained in step 2) into a training set and a validation set in a ratio of 9:1;

[0054] 4) Model Construction: Build a chip pad alignment detection model based on lightweight YOLOv5, and load the dataset into the chip pad alignment detection model based on lightweight YOLOv5 for training to obtain target detection results, including:

[0055] 4-1) Based on the chip pad alignment detection model of YOLOv5, the multi-scale fusion and detection parts are optimized. The improved multi-scale fusion and detection modules are as follows: Figure 2 As shown in the figure, it can reduce network parameters while enhancing the detection accuracy of the target, specifically:

[0056] 4-1-1) Trimming the last layer C3 module of the YOLOv5 backbone network can effectively reduce network parameters and speed up the deployment and inference of the model on embedded devices;

[0057] 4-1-2) Using upsampling to double the resolution of multi-size detection heads can effectively enhance the detection of small targets;

[0058] 4-2) Replace Bottleneck with Ghostneck in the backbone network feature extraction module C3;

[0059] like Figure 3 As shown in the figure, Ghost convolution first uses ordinary convolution to generate part of the feature map, then uses depth-wise separable convolution to process the generated feature map to obtain redundant feature maps, and finally splices these two different feature maps. In this way, more image feature maps can be obtained at a lower computational cost;

[0060] 4-3) Introducing the CBAM attention mechanism, which uses weighted processing of information in both channel and spatial dimensions to make the network more focused on useful information;

[0061] CBAM attention module such as Figure 4 As shown in the figure, the module consists of two independent sub-modules: the channel attention module and the spatial attention module. First, the channel feature map is generated by the channel attention module, and then it is multiplied with the residual input feature to weightedly refine and strengthen the useful channel information to solve the problem of "what" the target is. Similarly, the output result is processed by the spatial attention module to focus on "where" of the information part, and finally the final result is obtained. Adding the CBAM attention module after feature extraction can effectively aggregate the network's attention to the target and improve the detection ability of small targets.

[0062] 4-4) Introducing parallel dilated convolution into the SPPF module to form a context space module;

[0063] The context space module structure is as follows Figure 5 As shown in the figure, the dilated convolution expands the ordinary convolution to increase the receptive field without losing resolution, thereby extracting more contextual information and improving detection capabilities;

[0064] 4-5) Replace the CIOU loss function with the SIOU loss function, including:

[0065] 4-5-1) Angle loss:

[0066]

[0067]

[0068]

[0069] Among them, σ is the distance between the center point of the real box and the predicted box, c h is the height difference between the center point of the real box and the predicted box, Represents the x and y coordinates of the center values of the real box and the predicted box respectively;

[0070] 4-5-2) Distance loss:

[0071]

[0072]

[0073]

[0074] γ=2-Λ,

[0075] Among them, (c w ,c h ) is the width and height of the minimum bounding rectangle of the real box and the predicted box;

[0076] 4-5-3) Shape loss:

[0077]

[0078]

[0079]

[0080] Among them, (w,h) and (w gt ,h gt ) are the width and height of the predicted box and the real box respectively, and θ represents the degree of attention to shape loss;

[0081] 4-5-4) IoU loss:

[0082]

[0083] Among them, IoU represents the intersection-over-union ratio between the real image frame and the predicted frame;

[0084] 5) The trained deep learning model, namely the chip pad alignment detection model based on lightweight YOLOv5, is applied to chip pad alignment detection in industrial assembly lines.

[0085] To better test the performance of this method in detecting chip pads, we conducted an ablation experiment using model size, network parameters, and mean average precision (mAP) as evaluation indicators. The results of the ablation experiment are shown in Table 1:

[0086] Table 1:

[0087]

[0088] It can be seen from the experimental data given in Table 1 that after the multi-scale fusion improvement, the model size, network parameters and average accuracy are optimized compared with the original YOLOv5. The model size is reduced to 4.69M, the network parameters are reduced by 5M, and the mAP is improved by 0.8%; after the feature extraction part is improved, the model size is further reduced to 3.25M, the network parameters are reduced to 1.27M, and the mAP is improved by 1.7%; after the CSPP module is adopted, the model size and network parameters are slightly increased compared with those before using this method, but the detection accuracy is improved by 2.1% compared with the original YOLOv5. Overall, this method has significantly reduced the model size and network parameters, and significantly improved the average accuracy, achieving a good balance between model parameters and detection accuracy.

[0089] Table 2 shows the comparative experimental results of the improved YOLOv5 algorithm and other mainstream network algorithms. On the basis of the above evaluation indicators, the detection accuracy (AP) of rig, wro and Probe is used as the evaluation indicator. It can be seen from Table 2 that the detection accuracy of the original YOLOv5s algorithm for rig, wro and Probe is 90.6%, 85.3% and 84% respectively; the detection accuracy of the YOLOR algorithm for rig, wro and Probe is 90.3%, 85.1% and 79.3% respectively, and the mAP is 84.9%. The network parameters are 9M and the model is 10M. The model size is 17.46M; the YOLOv7-tiny algorithm has a detection accuracy of 91.1%, 87%, and 85.5% for rig, wro, and Probe, respectively, and a mAP of 87.9%. The network parameters are 6M and the model size is 11.72M. This example method improves the YOLOv5 algorithm to achieve a detection accuracy of 92%, 90%, and 85.1% for rig, wro, and Probe, respectively. At the same time, the average detection accuracy and the number of model parameters are better than the current mainstream detection algorithms. While ensuring accuracy, it reduces the number of network parameters and is suitable for deployment on industrial assembly lines.

[0090] Table 2 Comparative experiments of mainstream algorithms

[0091]

[0092]

[0093] The improved detection effect of this method is more intuitively shown as follows Figure 6 As shown, it has good robustness in detecting chip pads, and has the advantages of smaller network model and higher detection accuracy, which can better meet the requirements of real-time and accuracy in industrial assembly lines.

Claims

1. A lightweight chip pad alignment detection method based on YOLOv5, characterized in that: The steps include: 1) Acquisition: Use industrial cameras to capture chip pad sample images; 2) Preprocessing: Data augmentation is performed on the acquired images. LabelImg is used to label the augmented images. The wafer solder joints that require probe alignment during inspection are named rig, the wafer solder joints that do not require probe alignment are named wro, and the probes for contact testing are named Probe. This ultimately yields the dataset. Data augmentation involves cropping the original 6112*3440 images to 2030*1500 pixels. 3) Partition: Divide the dataset obtained in step 2) into a training set and a validation set in a ratio of 9:1; 4) Model Construction: Build a chip pad alignment detection model based on lightweight YOLOv5, and load the dataset into the chip pad alignment detection model based on lightweight YOLOv5 for training to obtain target detection results, including: 4-1) Based on the YOLOv5 chip pad alignment detection model, the multi-scale fusion and detection parts are optimized. Specifically: 4-1-1) Cut out the last layer C3 module of the YOLOv5 backbone network; 4-1-2) Using upsampling to double the resolution of multi-size detection heads; 4-2) Replace Bottleneck with Ghostneck in the backbone network feature extraction module C3; 4-3) Introducing the CBAM attention mechanism, which uses weighted processing of information in both channel and spatial dimensions; 4-4) Introducing parallel dilated convolution into the SPPF module to form a context space module; 4-5) Replace the CIOU loss function with the SIOU loss function, including: 4-5-1) Angle loss: Among them, σ is the distance between the center point of the real box and the predicted box, c h is the height difference between the center point of the real box and the predicted box, Represents the x and y coordinates of the center values of the real box and the predicted box respectively; 4-5-2) Distance loss: γ=2-Λ, Among them, (c w ,c h ) is the width and height of the minimum bounding rectangle of the real box and the predicted box; 4-5-3) Shape loss: Among them, (w,h) and (w gt ,h gt ) are the width and height of the predicted box and the real box respectively, and θ represents the degree of attention to shape loss; 4-5-4) IoU loss: Among them, IoU represents the intersection-over-union ratio between the real image frame and the predicted frame; 5) The trained deep learning model, namely the chip pad alignment detection model based on lightweight YOLOv5, is applied to chip pad alignment detection in industrial assembly lines.

Images