A deep learning wafer solder joint detection method based on improved YOLOV5

Abstract

The present invention discloses a deep learning wafer solder joint detection method based on improved YOLOV5, the method comprising: 1) preparing a wafer solder joint data set; 2) constructing an attention mechanism module CCANET; 3) constructing a YOLOV5 network incorporating the attention mechanism; 4) introducing a Ghost module; 5) training the improved YOLOV5 network. The present invention improves the accuracy of wafer solder joint detection while using fewer network parameters. Under the same conditions, not only more wafer solder joints can be detected, but also blocked wafer solder joints can be detected.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a deep learning wafer solder joint detection method based on improved YOLOV5. Background Art

[0002] As chips become more complex, there are more and more modules and functions inside the chip, and how to effectively test the wafer is increasingly considered in the entire chip design. At the same time, wafer testing is one of the most important statistical methods for chip yield. Improving chip yield can greatly reduce the loss of chip production in industrial production and improve chip production efficiency. Therefore, wafer testing has a very important strategic significance in the entire chip production.

[0003] Wafer testing is to perform needle testing on each die on the wafer. The probe on the detection head is used to contact the solder joint, and then the capacity and other properties of the wafer are measured through electrical testing. The contact between the probe and the solder joint is mostly completed manually. Since the solder joint area is 40x50 microns or less, the solder joint needs to be touched during the manual alignment process. If the probe head contacts the surface of the wafer solder joint too lightly, the test data will be inaccurate; if the contact is too heavy, it will cause damage to the tiny circuit on the wafer. At the same time, the existing wafer tester is not stable and has a high misjudgment rate, which is also one of the main contradictions that has been plaguing wafer production plants. In actual industrial production, the detection rate of wafer testers is about 40%, and the manual test rate is about 30%. One of the main reasons for the low detection efficiency of wafer testers is the misalignment of the probe and the wafer solder joint. The traditional method of manually detecting the alignment situation is difficult to meet the requirements of high precision and high real-time performance in industry. With the rapid development of machine vision, the change of industrial detection methods has been promoted. People have begun to use machine vision methods to identify and locate wafer solder joints. By identifying the wafer solder joints, the alignment status can be detected in real time, and calibration can be performed when alignment anomalies occur.

[0004] The wide application of deep learning in object detection provides new technical support for the detection of wafer solder joints. Currently, the two most mainstream directions of object detection algorithms are two-stage detection algorithms and one-stage detection algorithms: the two-stage detection algorithm treats object detection as a classification problem, first generating the area containing the object, then classifying and calibrating the candidate area to obtain the final detection result; the one-stage detection algorithm directly gives the final detection result without an explicit step of generating candidate boxes.

[0005] The most classic two-stage target detection method is the R-CNN series of algorithms. Although the R-CNN series of algorithms have high accuracy, they cannot meet the requirements of real-time detection due to the large number of network parameters and slow detection speed. However, the YOLO series of single-stage target detection algorithms is the algorithm with the best accuracy and speed so far. It successfully achieves a good balance between detection accuracy and detection speed. Among them, the YOLOV5 algorithm proposed in 2020 uses many network processing techniques to reduce the number of network parameters while maintaining high accuracy. However, due to the particularity of the detection target, the YOLOV5 algorithm still has room for optimization for wafer solder joint detection tasks. Summary of the invention

[0006] The purpose of the present invention is to provide a deep learning wafer solder joint detection method based on improved YOLOV5 to address the deficiencies of the prior art. This method can not only detect more wafer solder joints, but also detect obstructed wafer solder joints. This method has fewer network parameters and higher accuracy, and is more suitable for deployment in practical applications.

[0007] The technical solution for achieving the purpose of the present invention is:

[0008] A deep learning wafer solder joint detection method based on improved YOLOV5 includes the following steps:

[0009] 1) Create a wafer solder joint dataset: Collect wafer solder joint images, pre-process the collected wafer solder joint images to improve the number of samples and image quality of the dataset, and then annotate the dataset. Rectangular annotation is used for solder joints that need to be aligned and solder joints that do not need to be aligned, that is, solder joints that need to be aligned are named Rig holes, and solder joints that do not need to be aligned are named Wro holes. Finally, a wafer solder joint dataset containing 1464 wafer solder joint images is created, and the dataset is divided into a training set and a validation set at a ratio of 9:1;

[0010] 2) Construct the attention mechanism module CCANET: including:

[0011] 2.1) First, a filter module consisting of 1*1 convolution, Bathnorm module and Sigmoid function is used to obtain the feature map X. By reducing the dimension of the data, the cross-channel interaction of information is realized while reducing the amount of calculation, and the coherence of information is enhanced;

[0012] 2.2) Then adopt a three-branch structure:

[0013] 2.2.1) The first branch feature map X is connected to the color attention mechanism module, and the output feature map X 1 As shown in formula (1):

[0014] X 1=X*(σ(MLP(Resize(AvgPool(X)))+σ(MLP(Resize(AvgPool7(X)))+σ(MLP(Resize(MaxPool(X))))) (1),

[0015] Where MLP is a multi-layer perceptron, Resize is feature map size optimization, AvgPool is global average pooling of feature map size 1*1, AvgPool7 is global average pooling of feature map size 7*7, MaxPool is global maximum pooling of feature map size 1*1, and σ is the Sigmoid activation function;

[0016] 2.2.2) The second branch feature map X and the output feature map X of the first branch 1 Add channels and output feature map X 2 , as shown in formula (2):

[0017]

[0018] 2.2.3) The feature map X output in step 2.2.2) 2 Input position attention mechanism module, output feature map X 3 , as shown in formula (3):

[0019] X 3 =M s (X 2 ) (3),

[0020] Among them, M s The operation of the position attention mechanism;

[0021] 2.2.4) The third branch feature map X and feature map X 3 Multiply to get feature map X 4 , as shown in formula (4):

[0022]

[0023] 2.3) Finally output feature map X 4 , complete the attention mechanism module CCANET;

[0024] 3) Construct a YOLOV5 network with attention mechanism: including:

[0025] 3.1) Introducing the attention mechanism, the original YOLOV5 network consists of three parts: feature extraction layer, backbone layer and detection layer. The feature extraction layer and backbone layer of the original network are optimized: first, after the first, second and third C3 modules and SPP modules in the feature extraction layer of the YOLOV5 network, a CCANET module constructed in step 2) is added respectively. Then, the feature map output by each CCANET module is channel-added with the feature map output by the backbone layer of the original YOLOV5 network to obtain a new feature extraction layer;

[0026] 3.2) The number of output channels of each CCANET module is adaptively scaled, and the adaptive scaling ratio σ of the number of channels is 1 / 2. The channel number scaling expression is shown in formula (5):

[0027] X C =Y*σ (5),

[0028] Among them, X C is the number of output channels, Y is the number of input channels, and by scaling the output feature map, the network efficiency is enhanced, and the network accuracy is improved while reducing the parameters;

[0029] 4) Introducing the Ghost module: Further optimize the YOLOV5 network with attention mechanism constructed in step 3) by introducing the Ghost module: replace the first basic convolution module Conv after the three feature map concatenation modules Concat in the network backbone layer with the Ghost module to obtain the final improved YOLOV5 convolutional neural network;

[0030] 5) Training the improved YOLOV5 network: First, the improved YOLOV5 network obtained in step 4) is used to train the wafer solder joint data set produced in step 1) to obtain a wafer solder joint detection network. Then, the wafer solder joint image or wafer solder joint video to be tested is input into the wafer solder joint detection network to obtain the position and number of the wafer solder joints. Then, the accuracy and network model parameters of the wafer solder joint detection network are evaluated.

[0031] The preprocessing described in step 1) is image cropping and image enhancement, wherein image cropping fully takes into account the situation that the actual solder joints in the wafer solder joint picture only occupy a small part of the entire picture. If the entire picture is trained, the training time will be increased and the network accuracy will be reduced. The original 6112*3440 picture is cropped to a 2030*1500 picture containing only wafer solder joints, which reduces background redundant information and expands the small target wafer solder joints to medium targets, thereby improving the accuracy of network training and reducing training time; image enhancement is to enhance the wafer solder joint pictures under different light intensities, that is, to enhance the brightness of the solder joint pictures with darker brightness.

[0032] The color attention mechanism module described in step 2.2.1) includes the following process:

[0033] 2.2.1.1): Input the training set images into the network in the form of RGB three channels. In view of the particularity that the target colors of wafer solder joints are obviously different, the channel feature information is enhanced to enhance the color feature information and focus the network's attention more on the color features;

[0034] 2.2.1.2): First, three scales of pooling methods, 1*1 global average pooling, 7*7 global average pooling and 1*1 global maximum pooling, are used to extract channel features from feature maps of different structures, which improves the structural information of feature maps and reduces information loss; then the three pooled feature maps are normalized by the Resize module to keep the feature map size at the same scale, which facilitates further fusion of feature maps. The Resize module uses adaptive scaling of feature map size, and the scaled size is 1*1; finally, the three scaled feature maps are respectively input into a shared network composed of a multi-layer perceptron, and the feature vector is output after addition.

[0035] In step 3), a CCANET module constructed in step 2) is added respectively. In this way, the number of output channels of each module is adaptively scaled to half of the number of input channels while deepening the network. By scaling the attention mechanism module, the efficiency of network feature extraction is improved to a certain extent, and the network accuracy is improved while reducing parameters.

[0036] The color attention module described in step 2) is embedded in the position attention module. Due to the particularity of the target detection task, the color attention module and the position attention module are used to describe the questions of "what" and "where" the task target is in a picture, and an attention mechanism module that is more suitable for the target detection subdivision task is generated. The color attention module is embedded in the position attention module so that the two modules are connected through the embedded structure, which reduces redundant calculations and strengthens the correlation and subordination between the two features.

[0037] This technical solution proposes a color and position attention mechanism module CCANET for the specific color and position features of wafer inspection tasks. By strengthening the channel information and position feature information, the network feature extraction capability is improved, and the accuracy of wafer solder joint inspection tasks is improved;

[0038] This technical solution contains different semantic features according to feature maps of different depths. First, the present invention embeds the color attention mechanism into the position attention mechanism through a nested structure, thereby strengthening the correlation between different feature information, and reducing information loss while strengthening the two feature information; then, through the internal residual structure, the enhanced color feature information is channel-added with the shallow information, ensuring that the global information added to the attention mechanism is not lost; finally, through the feature information, the network attention is better focused on the place of interest, thereby improving the accuracy of the target task;

[0039] This technical solution designs the attention mechanism layer from three dimensions: number, position, and number of channels. By adding four CCANET modules to the feature extraction layer of the YOLOV5 network, and then adaptively scaling the CCANET modules, the output channels of the attention mechanism module CCANET are adaptively scaled to 1 / 2 of the number of input channels. From the perspective of network scaling, the relationship between the number of attention mechanism layers and positions, as well as the relationship between the number of attention mechanism layers and the number of channels, is found when designing the attention mechanism layer. Under limited network parameters, the target detection accuracy is greatly improved;

[0040] This technical solution introduces the Ghost module, replaces the ordinary convolution layer conv in the Neck layer in the original network with the Ghost module, generates more redundant features with less calculation, and reduces network parameters.

[0041] The present invention can improve the efficiency of accurate alignment and detection of wafer chips in actual industrial production, and can also assist in determining the alignment between the wafer solder joints and the probes by detecting the wafer solder joints.

[0042] This method can not only detect more wafer solder joints, but also detect obstructed wafer solder joints. This method has fewer network parameters and higher accuracy, and is more suitable for deployment in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a method flow chart of an embodiment;

[0044] Figure 2 Schematic diagram of the structure of the attention mechanism module CCANET in the embodiment;

[0045] Figure 3 Schematic diagram of the color attention mechanism module structure in the embodiment;

[0046] Figure 4 It is a schematic diagram of the network structure of the improved YOLOV5 network feature extraction layer in the embodiment;

[0047] Figure 5It is a schematic diagram of the network structure of the Neck layer of the improved YOLOV5 network in the embodiment;

[0048] Figure 6 It is a schematic diagram of the detection results in the embodiment;

[0049] Figure 7 It is a schematic diagram of the detection results in the embodiment;

[0050] Figure 8 It is a schematic diagram of the detection results in the embodiment. DETAILED DESCRIPTION

[0051] The content of the present invention is further described below in conjunction with the drawings and embodiments, but the present invention is not limited thereto.

[0052] Example:

[0053] Reference Figure 1 , a deep learning wafer solder joint detection method based on improved YOLOV5, comprising the following steps:

[0054] 1) Create a wafer solder joint dataset: Collect wafer solder joint images, pre-process the collected wafer solder joint images to improve the number of samples and image quality of the dataset, and then annotate the dataset. Rectangular annotation is used for solder joints that need to be aligned and solder joints that do not need to be aligned, that is, solder joints that need to be aligned are named Rig holes, and solder joints that do not need to be aligned are named Wro holes. Finally, a wafer solder joint dataset containing 1464 wafer solder joint images is created, and the dataset is divided into a training set and a validation set at a ratio of 9:1;

[0055] 2) Construct the attention mechanism module CCANET: The attention mechanism module CCANET is as follows Figure 2 As shown, including:

[0056] 2.1) First, a filter module consisting of 1*1 convolution, Bathnorm module and Sigmoid function is used to obtain the feature map X. By reducing the dimension of the data, the cross-channel interaction of information is realized while reducing the amount of calculation, and the coherence of information is enhanced;

[0057] 2.2) Then adopt a three-branch structure:

[0058] 2.2.1) The first branch feature map X is connected to the color attention mechanism module, and the color attention mechanism module is as follows Figure 3 As shown, the output feature map X 1 As shown in formula (1):

[0059] X 1=X*(σ(MLP(Resize(AvgPool(X)))+σ(MLP(Resize(AvgPool7(X)))+σ(MLP(Resize(MaxPool(X))))) (1),

[0060] Where MLP is a multi-layer perceptron, Resize is feature map size optimization, AvgPool is global average pooling of feature map size 1*1, AvgPool7 is global average pooling of feature map size 7*7, MaxPool is global maximum pooling of feature map size 1*1, and σ is the Sigmoid activation function;

[0061] 2.2.2) The second branch feature map X and the output feature map X of the first branch 1 Add channels and output feature map X 2 , as shown in formula (2):

[0062]

[0063] 2.2.3) The feature map X output in step 2.2.2) 2 Input position attention mechanism module, output feature map X 3 , as shown in formula (3):

[0064] X 3 =M s (X 2 ) (3),

[0065] Among them, M s The operation of the position attention mechanism;

[0066] 2.2.4) The third branch feature map X and feature map X 3 Multiply to get feature map X 4 , as shown in formula (4):

[0067]

[0068] 2.3) Finally output feature map X 4 , complete the attention mechanism module CCANET;

[0069] 3) Construct a YOLOV5 network with attention mechanism: including:

[0070] 3.1) Figure 4As shown in the figure, the attention mechanism is introduced. The original YOLOV5 network consists of three parts: feature extraction layer, backbone layer and detection layer. The feature extraction layer and backbone layer of the original network are optimized: first, a CCANET module constructed in step 2) is added after the first, second and third C3 modules and SPP modules in the feature extraction layer of the YOLOV5 network, and then the feature map output by each CCANET module is channel-added with the feature map output by the backbone layer of the original YOLOV5 network to obtain a new feature extraction layer;

[0071] 3.2) The number of output channels of each CCANET module is adaptively scaled, and the adaptive scaling ratio σ of the number of channels is 1 / 2. The channel number scaling expression is shown in formula (5):

[0072] X C =Y*σ (5),

[0073] Among them, X C is the number of output channels, Y is the number of input channels, and by scaling the output feature map, the network efficiency is enhanced, and the network accuracy is improved while reducing the parameters;

[0074] 4) Introducing the Ghost module: Further optimize the YOLOV5 network with attention mechanism constructed in step 3) by introducing the Ghost module: replace the first basic convolution module Conv after the three feature map concatenation modules Concat in the network backbone layer with the Ghost module to obtain the final improved YOLOV5 convolutional neural network;

[0075] 5) Train and improve the YOLOV5 network: Figure 4 , Figure 5 As shown, first, the improved YOLOV5 network obtained in step 4) is used to train the wafer solder joint data set produced in step 1) to obtain a wafer solder joint detection network, and then the wafer solder joint image or wafer solder joint video to be tested is input into the wafer solder joint detection network to obtain the position and number of the wafer solder joints, and then the accuracy of the wafer solder joint detection network and the network model parameters are evaluated.

[0076] The preprocessing described in step 1) is image cropping and image enhancement, wherein image cropping fully takes into account the situation that the actual solder joints in the wafer solder joint picture only occupy a small part of the entire picture. If the entire picture is trained, the training time will be increased and the network accuracy will be reduced. The original 6112*3440 picture is cropped to a 2030*1500 picture containing only wafer solder joints, which reduces background redundant information and expands the small target wafer solder joints to medium targets, thereby improving the accuracy of network training and reducing training time; image enhancement is to enhance the wafer solder joint pictures under different light intensities, that is, to enhance the brightness of the solder joint pictures with darker brightness.

[0077] The color attention mechanism module described in step 2.2.1) includes the following process:

[0078] 2.2.1.1): Input the training set images into the network in the form of RGB three channels. In view of the particularity that the target colors of wafer solder joints are obviously different, the channel feature information is enhanced to enhance the color feature information and focus the network's attention more on the color features;

[0079] 2.2.1.2): First, three scales of pooling methods, 1*1 global average pooling, 7*7 global average pooling and 1*1 global maximum pooling, are used to extract channel features from feature maps of different structures, which improves the structural information of feature maps and reduces information loss; then the three pooled feature maps are normalized by the Resize module to keep the feature map size at the same scale, which facilitates further fusion of feature maps. The Resize module uses adaptive scaling of feature map size, and the scaled size is 1*1; finally, the three scaled feature maps are respectively input into a shared network composed of a multi-layer perceptron, and the feature vector is output after addition.

[0080] In step 3), a CCANET module constructed in step 2) is added respectively. In this way, the number of output channels of each module is adaptively scaled to half of the number of input channels while deepening the network. By scaling the attention mechanism module, the efficiency of network feature extraction is improved to a certain extent, and the network accuracy is improved while reducing parameters.

[0081] The color attention module described in step 2) is embedded in the position attention module. Due to the particularity of the target detection task, the color attention module and the position attention module are used to describe the questions of "what" and "where" the task target is in a picture, and an attention mechanism module that is more suitable for the target detection subdivision task is generated. The color attention module is embedded in the position attention module so that the two modules are connected through the embedded structure, which reduces redundant calculations and strengthens the correlation and subordination between the two features.

[0082] In this example, the experimental environment configuration is shown in Table 1. The experiment is written in Python 3.8, the deep learning framework is Pytorch1.9, the training batch is set to 500, the mini-batch is set to 32, the initial learning rate is 0.01, the weight decay coefficient is 0.0005, and the momentum coefficient is 0.9. The experimental data set uses the wafer solder joint data set that has been collected and produced. The solder joints are divided into two categories, Rig solder joints and Wro solder joints. The data set has a total of 1464 images, the ratio of training set to validation set is 9:1, and the number of training epochs is 500. The pre-selected boxes used in the training process are clustered using the k-means clustering algorithm to obtain nine pre-selected boxes, namely (10,13), (16,30), (33,23), (30,61), (62,45), (59,119), (116,90), (156,198), (373,326) to improve the generalization ability of the model.

[0083] Table 1

[0084] name Configuration operating system Windows Server 2016 Standard CPU Intel(R) Xeon(R) Gold 6152 CPU@2.10GHz 2.10 GHz GPU NVIDIA Tesla P40 24GB GPU Accelerated Libraries CUDA Version:10.2

[0085] Experimental results and analysis:

[0086] The test evaluation indicators use Rig (AP), Wro (AP), F1 and the comprehensive accuracy mean mAP@0.5 to measure the accuracy of the detection algorithm, and then use network parameters and model size to comprehensively evaluate the network model size.

[0087] Table 2 gives the ablation experiments of the YOLOV5 algorithm, CCANET module and Ghost module combined with the above evaluation indicators. It can be seen from Table 2 that the original YOLOV5 algorithm Rig (AP) is 75.4%, Wro (AP) is 89.2%, the average accuracy Map@0.5 is 82.3%, parameters are 7.3M, F1 is 78%, and the model size is 14.4M; when the Ghost module is added, Rig (AP) decreases by 6.8%, Wro (AP) increases by 0.6%, the average accuracy Map@0.5 decreases by 3.1%, parameters are reduced by 1.05M, F1 decreases by 1%, and the model size is reduced by 2.3M; when the Ghost module and CCANET module are added, Rig (AP) increases by 7.9%, Wro (AP) increases by 5%, the average accuracy Map@0.5 increases by 6.5%, parameters are reduced by 0.6M, F1 increases by 7%, and the model size is reduced by 1.6M. Through the above experiments, it is found that when the Ghost module is added, the parameters and model size of the model will decrease, but the average accuracy and the single accuracy of each solder joint will also decrease; when the two modules are combined, it can be found that the improved algorithm after the combination has the best effect, and all its accuracy indicators have been significantly improved. At the same time, the number of parameters and the model size are reduced. The improved algorithm after the combination has achieved a better balance between model parameters and detection accuracy.

[0088] Table 2: Ablation experiments of different modules

[0089]

[0090] Table 3 gives the comparative experiments of the improved YOLOV5 algorithm and other network algorithms combined with the above evaluation indicators. From Table 3, we can see that the original YOLOV5 algorithm Rig (AP) is 75.4%, Wro (AP) is 89.2%, the average accuracy Map @ 0.5 is 82.3%, parameters are 7.3M, F1 is 78%, and the model size is 14.4M; the Yolov3-spp algorithm Rig (AP) is 61.3%, Wro (AP) is 92.4%, the average accuracy Map @ 0.5 is 76.8%, parameters are 62.5M, F1 is 78%, and the model size is 119M; the improved YOLOV5 algorithm Rig (AP) is 83.3%, Wro (AP) is 94.2%, the average accuracy Map @ 0.5 is 88.8%, parameters are 6.7M, F1 is 85%, and the model size is 12.8M. Through algorithm comparison experiments, the improved YOLOV5 algorithm is superior to other algorithms in both accuracy and model parameter quantity. It reduces the number of model parameters while ensuring accuracy, and is more suitable for deployment in industrial applications.

[0091] Table 3 Comparison test with other algorithms

[0092]

[0093] like Figure 6 , Figure 7 , Figure 8 As shown in the figure, this method has good robustness in the detection of wafer solder joints. It has the advantage of a smaller model while maintaining a high detection accuracy, and is more suitable for deployment in actual industrial applications to assist in the real-time detection of wafer alignment. However, when the light intensity on the surface of the wafer solder joint is too low, there is still room for improvement in accuracy. In the next work, we will consider further improving the detection accuracy under different light intensities.

Claims

1. A deep learning wafer solder joint detection method based on improved YOLOV5, characterized in that: The steps include: 1) Create a wafer solder joint dataset: Collect wafer solder joint images, pre-process the collected wafer solder joint images, and then annotate the dataset. Rectangular annotation is used for solder joints that need to be aligned and solder joints that do not need to be aligned, that is, solder joints that need to be aligned are named Rig holes, and solder joints that do not need to be aligned are named Wro holes. Finally, a wafer solder joint dataset containing 1464 wafer solder joint images is created, and the dataset is divided into a training set and a validation set at a ratio of 9:1; 2) Construct the attention mechanism module CCANET: including: 2.1) First, a filter module consisting of 1*1 convolution, Bathnorm module and Sigmoid function is used to obtain the feature map X; 2.2) Then adopt a three-branch structure: 2.2.1) The first branch feature map X is connected to the color attention mechanism module, and the output feature map X1 is shown in formula (1): X1=X*(σ(MLP(Resize(AvgPool(X)))+σ(MLP(Resize(AvgPool7(X)))+σ(MLP(Resize(MaxPool(X))))) (1), Where MLP is a multi-layer perceptron, Resize is feature map size optimization, AvgPool is global average pooling of feature map size 1*1, AvgPool7 is global average pooling of feature map size 7*7, MaxPool is global maximum pooling of feature map size 1*1, and σ is the Sigmoid activation function; 2.2.2) The second branch feature map X is channel-wise added to the output feature map X1 of the first branch to output the feature map X2, as shown in formula (2): 2.2.3) Input the feature map X2 output in step 2.2.2) into the position attention mechanism module and output the feature map X3, as shown in formula (3): X3=M s (X2) (3), Among them, M s The operation of the position attention mechanism; 2.2.4) The third branch feature map X is multiplied by the feature map X3 to obtain the feature map X4, as shown in formula (4): 2.3) Finally, the feature map X4 is output to complete the attention mechanism module CCANET; 3) Construct a YOLOV5 network with attention mechanism: including: 3.1) Introducing the attention mechanism, the original YOLOV5 network consists of three parts: feature extraction layer, backbone layer and detection layer. The feature extraction layer and backbone layer of the original network are optimized: first, after the first, second and third C3 modules and SPP modules in the feature extraction layer of the YOLOV5 network, a CCANET module constructed in step 2) is added respectively. Then, the feature map output by each CCANET module is channel-added with the feature map output by the backbone layer of the original YOLOV5 network to obtain a new feature extraction layer; 3.2) The number of output channels of each CCANET module is adaptively scaled, and the adaptive scaling ratio σ of the number of channels is 1 / 2. The channel number scaling expression is shown in formula (5): X C =Y*σ (5), Among them, X C is the number of output channels, Y is the number of input channels; 4) Introducing the Ghost module: Further optimize the YOLOV5 network with attention mechanism constructed in step 3) by introducing the Ghost module: replace the first basic convolution module Conv after the three feature map concatenation modules Concat in the network backbone layer with the Ghost module to obtain the final improved YOLOV5 convolutional neural network; 5) Training the improved YOLOV5 network: First, the improved YOLOV5 network obtained in step 4) is used to train the wafer solder joint data set produced in step 1) to obtain a wafer solder joint detection network. Then, the wafer solder joint image or wafer solder joint video to be tested is input into the wafer solder joint detection network to obtain the position and number of the wafer solder joints. Then, the accuracy and network model parameters of the wafer solder joint detection network are evaluated.

2. According to the deep learning wafer solder joint detection method based on improved YOLOV5 according to claim 1, it is characterized in that the preprocessing described in step 1) is image cropping and image enhancement, wherein the image cropping crops the original image of 6112*3440 into a 2030*1500 image containing only wafer solder joints, and expands the wafer solder joints of small targets into medium targets; Image enhancement is to enhance the images of wafer solder joints under different light intensities, that is, to enhance the brightness of solder joints with darker brightness.

3. The deep learning wafer solder joint detection method based on improved YOLOV5 according to claim 1, characterized in that, The color attention mechanism module described in step 2.2.1) includes the following process: 2.2.1.1): Input the training set images into the network in the form of RGB three channels. In view of the particularity that the target colors of wafer solder joints are obviously different, the channel feature information is enhanced to enhance the color feature information and focus the network's attention more on the color features; 2.2.1.2): First, three scales of pooling methods, namely 1*1 global average pooling, 7*7 global average pooling and 1*1 global maximum pooling, are used to extract channel features from feature maps of different structures. Then, the three pooled feature maps are normalized by the Resize module to keep the feature map size at the same scale. The Resize module adaptively scales the feature map size, and the scaled size is 1*1. Finally, the three scaled feature maps are respectively input into a shared network composed of a multi-layer perceptron, and the feature vector is output after addition.

4. According to the deep learning wafer solder joint detection method based on improved YOLOV5 according to claim 1, it is characterized in that, In step 3), while adding a CCANET module constructed in step 2), the number of output channels of each module is adaptively scaled to half of the number of input channels.

5. According to the deep learning wafer solder joint detection method based on improved YOLOV5 according to claim 1, it is characterized in that, The color attention module described in step 2) is embedded into the position attention module.

Images