Defective solder ball detection method based on improved YOLO v8 network

Abstract

The present invention discloses a defective solder ball detection method based on an improved YOLO v8 network, the method comprising: collecting grayscale images of chip BGA packages, forming a training data set for defective solder ball detection; constructing an improved YOLO v8 network, and training through the training data set; testing the trained improved YOLO v8 network through a test data set to obtain a final improved YOLO v8 network; performing defective solder ball detection on the measured chip BGA image through the final improved YOLO v8 network to obtain a detection result, wherein the retrieval result includes a detection frame and a test index corresponding to the defective solder ball. The present invention improves the YOLO v8 network, adopts a one-to-many detection head to assist a one-to-one detection head in the head network; improves the calculation method of the network loss function; performs better in target detection tasks, and can more accurately detect and locate target objects.

Description

Technical Field

[0001] The present invention relates to the technical field of chip defect detection, and in particular to a defective solder ball detection method based on an improved YOLO v8 network. Background Art

[0002] As an important surface mount technology, Ball Grid Array (BGA) packaging has become the mainstream of microelectronics high-density packaging technology. BGA packaging has the advantages of small-pitch I / O pins, excellent electrothermal performance and lightweight. Among them, the preparation of high-precision, small-diameter solder balls is the key to BGA packaging technology. However, during the production and transportation process, the solder balls may be damaged or have other defects, which directly affects the reliability of electronic products. Therefore, it is very important to detect solder ball defects in BGA packaging.

[0003] Traditional BGA defect detection mainly relies on manual visual inspection or simple image processing algorithms, which have the problems of low detection efficiency and high false detection rate. In recent years, with the rapid development of machine vision technology, target detection algorithms based on deep learning have shown great potential in the field of industrial defect detection. Among them, the target detection model based on convolutional neural network (CNN) can automatically extract features from images, accurately locate and identify defective solder balls, greatly improving the accuracy, efficiency and automation level of BGA defect detection.

[0004] At present, common deep learning-based BGA defect detection methods are mainly divided into two categories: two-stage detection models and single-stage detection models. Two-stage models such as Faster R-CNN first generate candidate regions and then classify them. This type of method has higher detection accuracy, but the amount of calculation is larger. Single-stage models such as the YOLO series combine border positioning and target classification into a regression process, which has a faster detection speed, but the accuracy is slightly lower than the two-stage model.

[0005] In recent years, attention mechanisms have been increasingly developed and utilized in the field of deep learning, including selective attention mechanisms and self-attention mechanisms. Different from fully connected neural networks, convolutional neural networks, and recurrent neural networks, attention mechanisms can adaptively weight feature tensors propagated in the network according to task requirements, so that more important areas of the output receive more attention. The Transformer model, which has been widely promoted in recent years, is a deep learning network model based on the self-attention mechanism.

[0006] Although a variety of deep learning models have been used for BGA defect detection, due to the diversity and complexity of BGA defects, the existing models still need to be improved in terms of generalization ability and real-time performance. How to improve the existing target detection network model, enhance its detection robustness for complex BGA defects, and improve detection accuracy and speed is an important research topic in the current BGA defect detection field. Summary of the invention

[0007] Technical purpose: In view of the defects in the prior art, the present invention discloses a defective solder ball detection method based on an improved YOLO v8 network. By improving the YOLO v8 network, when constructing the backbone network, the c2f-BiFormer module is used to replace part of the c2f module, and the SPP-ELAN module is used to replace the fast spatial pyramid pooling SPPF module; a one-to-many detection head is used in the head network to assist the design of a one-to-one detection head; the calculation method of the network loss function is improved; the target detection task performs better and can detect and locate the target object more accurately.

[0008] Technical solution: In order to achieve the above technical objectives, the present invention adopts the following technical solution.

[0009] A defective solder ball detection method based on an improved YOLO v8 network, comprising:

[0010] Collect grayscale images of chip BGA packages and build a training data set for defective solder ball detection;

[0011] Build an improved YOLO v8 network and train it using the training dataset;

[0012] The trained improved YOLO v8 network is tested using the test data set to obtain the final improved YOLO v8 network;

[0013] The defective solder balls are detected on the measured chip BGA image through the final improved YOLO v8 network to obtain the detection results. The retrieval results include the detection frame and test indicators corresponding to the defective solder balls.

[0014] Beneficial effects:

[0015] (1) Compared with the conventional YOLO v8 network, the improved YOLO v8 network proposed in the present invention requires less computing resources in the test phase, which can improve computing efficiency; it performs better in target detection tasks and can detect and locate target objects more accurately;

[0016] (2) The present invention adopts the c2f-BiFormer module to replace part of the c2f module. The attention mechanism BRA module is set in the c2f-BiFormer module, so that it has stronger positioning ability when processing small targets or edge situations;

[0017] (3) The present invention adopts the SPP-ELAN module to replace the fast spatial pyramid pooling SPPF module, which not only reduces the number of network model parameters, but also improves the accuracy of target detection and achieves a balance between computational efficiency and detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of the method of the present invention;

[0019] Figure 2 It is a schematic diagram of the structure of the improved YOLO v8 network of the present invention;

[0020] Figure 3 for Figure 2 Schematic diagram of the C2f-BiFormer module structure;

[0021] Figure 4 for Figure 2 Schematic diagram of BiFormer module structure;

[0022] Figure 5 for Figure 2 Schematic diagram of the SPP-ELAN module structure;

[0023] Figure 6 This is a schematic diagram of the detection results of the conventional YOLO v8 network;

[0024] Figure 7 It is a schematic diagram of the detection results of the improved YOLO v8 network of the present invention;

[0025] Figure 8 It is a schematic diagram for comparing the detection results of the conventional YOLO v8 network and the detection results of the improved YOLO v8 network of the present invention with respect to the precision rate;

[0026] Fig. 9 It is a schematic diagram comparing the detection results of the conventional YOLO v8 network and the detection results of the improved YOLO v8 network of the present invention with respect to the recall rate;

[0027] Fig.10 It is a schematic diagram comparing the detection results of the conventional YOLO v8 network and the detection results of the improved YOLO v8 network of the present invention for the mAP value when the IoU threshold is set to 0.5;

[0028] Fig.11It is a schematic diagram comparing the detection results of the conventional YOLO v8 network and the detection results of the improved YOLO v8 network of the present invention for the mAP value when the IoU threshold is set to 0.5-0.95;

[0029] Fig.12 A schematic diagram showing a comparison of the detection results of a conventional YOLO v8 network for training bounding box regression loss and the detection results of an improved YOLO v8 network of the present invention;

[0030] Fig.13 A schematic diagram showing a comparison between the detection results of a conventional YOLO v8 network for verifying border regression loss and the detection results of an improved YOLO v8 network of the present invention;

[0031] Fig.14 It is a schematic diagram comparing the detection results of the conventional YOLO v8 network for training DFL and the detection results of the improved YOLO v8 network of the present invention;

[0032] Fig.15 A schematic diagram showing a comparison between the detection results of a conventional YOLO v8 network for verifying DFL and the detection results of an improved YOLO v8 network of the present invention;

[0033] Fig.16 A schematic diagram showing a comparison of the detection results of a conventional YOLO v8 network for training target classification loss and the detection results of an improved YOLO v8 network of the present invention;

[0034] Fig.17 It is a schematic diagram comparing the detection results of the conventional YOLO v8 network for verification target classification loss and the detection results of the improved YOLO v8 network of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] Example

[0037] As attached Figure 1 As shown, a defective solder ball detection method based on an improved YOLO v8 network of the present invention comprises the following steps:

[0038] Step A, collecting grayscale images of chip BGA packages to form a training data set for defective solder ball detection; in this embodiment, the size of all collected grayscale images of chip BGA packages is adjusted to 640×640; a labeling box is established on the grayscale image of the chip BGA package for various defective solder balls, and the labeling parameters include the center coordinates, width, height and defect type of the labeling box;

[0039] Step B, build an improved YOLO v8 network and train it with the training data set; input the training data set into the backbone network, extract basic features to obtain basic feature maps; send the basic feature maps into the neck network for feature fusion to obtain fused feature maps; input the fused feature maps into the detection head network, and use the one-to-one detection head and the one-to-many detection head to process them respectively to obtain their respective prediction box parameters; match the annotation box with the prediction box within its coverage, and select the prediction box with a high degree of matching for each annotation box as a positive sample. The one-to-one detection head retains one positive sample for each annotation box, and the one-to-many detection head retains multiple positive samples for each annotation box; calculate the loss function based on the prediction box and annotation box parameters, and back propagate to update the network model parameters; repeatedly train the network model until the network model parameters converge. The training process includes the following steps:

[0040] Step B1: Figure 2 -Attached Figure 5 As shown in the figure, an improved YOLO v8 network is constructed, wherein the improved YOLO v8 network includes a backbone network, a neck network and a detection head network. When constructing the backbone network, the c2f-BiFormer module is used to replace part of the c2f module, and the SPP-ELAN module is used to replace the fast spatial pyramid pooling SPPF module.

[0041] Specifically, as attached Figure 2 As shown:

[0042] The backbone network consists of ten components: five Conv modules, three c2f-BiFormer modules, one c2f module, and one SPP-ELAN module;

[0043] The output of the first Conv module is input to the second Conv module, the output of the second Conv module is input to the first c2f-BiFormer module, the output of the first c2f-BiFormer module is input to the third Conv module, the output of the third Conv module is input to the second c2f-BiFormer module, the output of the second c2f-BiFormer module is input to the fourth Conv module, the output of the fourth Conv module is input to the third c2f-BiFormer module, the output of the third c2f-BiFormer module is input to the fifth Conv module, the output of the fifth Conv module is input to the c2f module, and the output of the c2f module is finally input to the SPP-ELAN module;

[0044] Among them, as attached Figure 3 As shown in the figure, each c2f-BiFormer module contains two Conv submodules and one Sequential submodule. The input first passes through the first Conv submodule, and the output is used as the input of the Sequential submodule; the Sequential submodule processes the features through several BiFormer blocks, and its output is then passed to the second Conv submodule. Finally, the output of the second Conv submodule is used as the final output of the entire c2f-BiFormer module.

[0045] As attached Figure 4 As shown in the figure, the BiFormer module includes a deep convolution module, two layer normalization modules, a BRA module and a multi-layer perceptron module. The input first passes through a deep convolution module, and its output is added to the original input as the input of the first layer normalization module; the output of the first layer normalization module is superimposed with the output obtained by the BRA module and the input of the first layer normalization module, and finally the output of the second layer normalization module and the multi-layer perceptron module is used as the final output of the BiFormer module. The BiFormer module is part of the C2f-Biformer module. C2f is a fast implementation of CSP. BiFormer is equivalent to the bottleneck module in C2f. BRA is the most important component of the BiFormer module.

[0046] The working principle of the attention mechanism BRA module includes:

[0047] The input feature map of the BRA module is , dividing it into non-overlapping regions to achieve grid division. In some embodiments of the present invention, S The value is 8, so the size of each area is Each channel in each region is a two-dimensional matrix, which is connected row by row into a one-dimensional row vector. Transformed into .

[0048] The query Q, key K and value V are obtained through linear projection, which are expressed as follows:

[0049] ,

[0050] in, , respectively, the projection weight matrices of query, key, and value; .

[0051] In each channel of query Q and key K, we average each row separately to get the region-level query and key ,and ;right and Perform matrix multiplication to obtain the adjacency matrix for:

[0052] ,

[0053] in, . Adjacency Matrix Any element in It can be used to measure the degree of semantic association between the i-th region and the j-th region; Each line has Elements are used to represent the area corresponding to this row and all The semantic correlation of the regions.

[0054] Keep the indexes of the top k regions with the strongest correlation with each region, k is much smaller than , in this embodiment k Set the value to 7 to create a routing index matrix ,and , routing index matrix Any row of Save the adjacency matrix Middle Before the line The index of the element with the largest value, .

[0055] Q Regions, ,according to The index in K and V is the most relevant to it. The areas are concentrated to obtain and ,and .Will Combine into a three-dimensional tensor to get a dense key tensor and dense value tensors ,and .

[0056] The output of the attention mechanism BRA module is calculated as follows:

[0057] ,

[0058] in, .

[0059] The present invention adopts the c2f-BiFormer module to replace part of the c2f module. The attention mechanism BRA module is set in the c2f-BiFormer module, so that it has stronger positioning ability when processing small targets or edge situations, as follows:

[0060] 1. Improve the YOLO v8 network to focus on the most relevant parts of the image when processing images. Selective focus helps the model focus on key areas, thereby improving the accuracy of positioning and detection;

[0061] 2. Spatial attention helps improve the YOLO v8 network to prioritize certain more important spatial regions in the image;

[0062] 3 The attention mechanism dynamically adjusts the focus based on surrounding information, thereby enhancing the understanding of the context. Localization depends not only on isolated object features but also on its context.

[0063] 4. The attention mechanism can capture long-distance dependencies and global relationships in images.

[0064] As attached Figure 5As shown, the SPP-ELAN module includes two Conv submodules and three maximum pooling submodules; the input data first passes through the first Conv submodule to obtain the output cv1; cv1 is input to the first maximum pooling submodule to obtain the output mx1; mx1 is input to the second maximum pooling submodule to obtain the output mx2; mx2 is input to the third maximum pooling submodule to obtain the output mx3; cv1 and mx1, mx2, and mx3 are connected through the Concat module in the channel dimension to obtain cc1; cc1 is input to the second Conv submodule to obtain the output cv2; cv2 is used as the final output of the SPP-ELAN module; the present invention adopts the SPP-ELAN module to replace the fast spatial pyramid pooling SPPF module, which can reduce the number of network model parameters. The number of parameters of the SPPF module is 535562, and the number of parameters of the SPP-ELAN module is 71178. At the same time, the accuracy of target detection is improved, and the balance between computational efficiency and detection performance is achieved.

[0065] Step B2, divide the training data set into several batches, each batch includes several grayscale images of chip BGA packages, referred to as chip BGA images. In this embodiment, each batch includes 16 chip BGA images, the input tensor dimension is (16, 3, 640, 640), the number of channels is 3, and the number of BGA package defect types is 8, which is also the number of solder ball defect categories.

[0066] Step B3: For each batch of training data sets, input them into the backbone network in the improved YOLO v8 network for feature extraction to obtain basic feature maps of three scales of the batch; in this embodiment, by adjusting the grid division parameter value, three grid divisions of 20×20, 40×40, and 80×80 are realized corresponding to the chip BGA image, wherein 20×20, 40×40, and 80×80 are the feature map sizes after feature fusion, and basic feature maps of large, medium, and small scales are obtained, and the scales are (16, 256, 20, 20), (16, 128, 40, 40), and (16, 64, 80, 80), respectively; wherein, in the scale (x1, x2, x3, x3), x1 represents the number of data in the currently annotated training data set, x2 represents the depth of the feature map; x3 represents the grid division parameter.

[0067] Step B4: Input the basic feature map into the neck network in the improved YOLO v8 network to achieve feature fusion, and obtain fused feature maps of three scales, with scales of (16, 256, 20, 20), (16, 128, 40, 40), and (16, 64, 80, 80).

[0068] Step B5, inputting the fused feature map into the detection head network in the improved YOLO v8 network, performing border regression reasoning and target classification reasoning, and obtaining the border correction value and category probability value of the prediction box; comprising the following steps:

[0069] Step B51, construct a detection head network in the improved YOLO v8 network, the detection head network includes several groups of detection heads, the number of detection head groups is the same as the number of scales of the fused feature map, and each group of detection heads includes 1 one-to-one detection head and 1 one-to-many detection head; that is, in this embodiment, when constructing the detection head network, it is necessary to construct three groups of detection heads for large, medium and small scales, and each group of detection heads includes 1 one-to-one detection head and 1 one-to-many detection head. The fused feature maps of the three scales are respectively input into the corresponding three groups of detection heads in the detection head network to perform border regression inference and target classification inference; in the training stage, the one-to-one detection head and the one-to-many detection head are optimized synchronously. In the testing and actual use stages, one-to-many detection heads are not used, only one-to-one detection heads are used. The scales of the prediction results of the multi-scale detection heads are (16,72,20,20), (16,72,40,40), (16,72,80,80), respectively, where , is the number of species, which is 8 in the embodiment, is a hyperparameter for controlling the number of channels of the bounding box regression branch, which is 16 in the embodiment;

[0070] Step B52: For each element in the fused feature map, the detection head network outputs a prediction box border code and category probability value. Specifically, the multi-scale prediction results (16,72,20,20), (16,72,40,40), (16,72,80,80) are integrated into a tensor of scale (16,8400,72) through dimensional transformation, and then split into a category probability value of scale (16,8400,8) and a border code of scale (16,8400,64). .

[0071] Step B53: Decode the border code of each prediction box to obtain the border correction value and category probability value of the prediction box; specifically, transform the regression code scale to (16, 8400, 4, 16), perform softmax processing on its last dimension, and obtain the probability that the anchor point is n away from the four edges of the upper left, lower right and lower sides of the prediction box, where . The softmax processed tensor is combined with the vector Multiply, that is, weighted sum of the last dimension of the tensor to obtain the decoded border correction value, whose scale is (16,8400,4). This step is a common operation in the YOLO v8 network and will not be repeated here.

[0072] Step B6: For each batch of training data sets, the improved YOLO v8 network is trained and optimized according to the border correction value and category probability value of the prediction box, including the following steps:

[0073] Step B61, based on the frame correction value and category probability value of the prediction frame, and the annotation frame parameters, calculate the matching degree between the prediction frame and the annotation frame. The one-to-many detection head selects multiple prediction frames with the highest matching degree as positive samples for each annotation frame, while the one-to-one detection head selects only one prediction frame with the highest matching degree as a positive sample for each annotation frame. In some embodiments of the present invention, the one-to-many detection head selects 7 prediction frames with the highest matching degree as positive samples for each annotation frame, while the one-to-one detection head selects only one prediction frame with the highest matching degree as a positive sample for each annotation frame.

[0074] Label assignment is required during both training and testing. When matching the predicted box with the labeled box, both the one-to-one detection head and the one-to-many detection head use the following measurement formula to evaluate the matching degree between the predicted box and the labeled box:

[0075] ,

[0076] in, They are the prediction box and the annotation box respectively. The spatial prior indicating whether the anchor point is within the annotation box, is the classification score, is the intersection-over-union ratio between the predicted box and the labeled box, Hyperparameters for balancing target classification and regression positioning.

[0077] Step B62: Substitute the border correction value, category probability value and corresponding annotation box parameters of each positive sample prediction box into the loss function to obtain the loss corresponding to the prediction box and the annotation box. ; Superimpose the losses corresponding to all prediction boxes in the current batch of training data to obtain the loss value of the current batch of training data , based on the obtained loss value, back propagation updates the network model parameters.

[0078] For each set of predicted boxes and labeled boxes, the loss The function calculation formula includes:

[0079] ,

[0080] in, represents the bounding box regression loss, is the distribution focal loss (DFL, Distribution FocalLoss), represents the classification loss, is the gain coefficient corresponding to the above three losses;

[0081] Bounding Box Regression Loss , the calculation formula includes:

[0082] ,

[0083] Among them, r is the regularization factor; Used to measure the degree of center alignment between the prediction box and the annotation box; represents the intersection-over-union loss, is the intersection-over-union ratio between the predicted box and the labeled box, are two hyperparameters, and .

[0084] Distribution Focus Loss It is expressed as follows:

[0085] ,

[0086] in, Represent the distribution focal losses of the predicted box and the labeled box at the left boundary, right boundary, upper boundary and lower boundary respectively.

[0087] The distribution focal loss at the left boundary is calculated as follows:

[0088] ,

[0089] in, Indicates the distance between the left edge of the annotation box and the center of the annotation box. express The value rounded up, express The value to round down. Indicates that the distance between the left boundary of the prediction box and the center of the annotation box is The probability of Indicates that the distance between the left boundary of the prediction box and the center of the annotation box is probability.

[0090] The distributed focal loss calculation method at the right boundary, upper boundary and lower boundary is the same as that at the left boundary.

[0091] Classification Loss It is expressed as follows:

[0092] ,

[0093] in, The marked boxes are j The probability value of the category and the prediction box j Class probability values, ; N is the number of defective solder ball categories, The possible values ​​are .

[0094] The loss of the one-to-one detection head is , the loss of one-to-many detection head is , the overall loss corresponding to the current batch of training data is:

[0095] ,

[0096] in, and is a normalized weighting coefficient between 0 and 1, ,and , The number of positive samples selected by the one-to-many detection head for each labeled box. The loss of the one-to-one detection head and the loss of one-to-many detection heads The loss corresponding to each pair of prediction box and annotation box sum.

[0097] In order to improve the test speed of the network model while ensuring the defect detection performance, a one-to-many detection head assisted one-to-one detection head design is adopted in the head network, and the non-maximum suppression process is omitted in the testing phase.

[0098] Step B63: input all batches of training data into the network model one by one, and repeatedly train the network model until the network model parameters converge;

[0099] Step C: Test the trained improved YOLO v8 network through the test data set to obtain the final improved YOLO v8 network; collect the grayscale image of the chip BGA package to construct the test data set.

[0100] Input the chip BGA image test data set and perform network model test on the trained improved YOLO v8 network. At this time, only one-to-one detection head is used in the detection head network. According to the prediction box and annotation box parameters, the detection box corresponding to the defective solder ball and various test indicators are output.

[0101] The test indicators include accuracy , recall rate , F1 score , and its calculation formula is as follows:

[0102] ,

[0103] ,

[0104] ,

[0105] in, They are the number of correctly detected targets, the number of falsely detected targets, and the number of missed targets.

[0106] Step D: perform defective solder ball detection on the measured chip BGA image through the final improved YOLO v8 network to obtain the detection result, and the retrieval result includes the border correction value and category probability value of the prediction box.

[0107] This embodiment uses simulated and measured chip BGA images as data sets to perform performance tests on the improved YOLOv8 network proposed in the present invention. After expanding the data set using data enhancement methods such as cropping, rotation, brightness adjustment, contrast adjustment, and scale transformation, the training data set includes 3,800 images, the verification data set includes 950 images, and the test data set includes 200 images. The image size is adjusted to 640×640 pixels and includes three channels: red, green, and blue.

[0108] The defect label information includes defect category, normalized center coordinates of the border, normalized height and width of the border. Defect categories include common chip BGA defects such as bridge, burr, missing, residue, scratch, etc.

[0109] In this example, the initial learning rate is set to 0.00083, the weight decay rate is 0.0005, the inertia is 0.9, the training rounds are set to 50, the batch_size is set to 16, and the border regression loss gain coefficient is Set to 7.5, distributed focus loss gain factor Set to 1.5, classification loss gain coefficient The probability of dropout is set to 0.5, the probability of dropout is set to 0.001, the optimizer is AdamW, and the learning rate scheduling is the cosine annealing method.

[0110] The parameters of the improved YOLO v8 network model used in this embodiment are 2812296, which is about 10.92% less than that of the conventional YOLO v8 network model; the floating point operation amount is 7.9GFLOPS, which is about 11.23% less. The experimental results show that the precision of the improved YOLOv8 network is 1.87% higher than that of the conventional YOLO v8 network, the recall rate is 2.53%, the mAP is 4.42%, and the test speed is 29.21%. The above results show that the improved YOLO v8 network requires fewer computing resources, and has higher computing efficiency and detection accuracy.

[0111] Figure 6 and Figure 7The comparison of the results of the two network models in chip BGA image defect detection is given. Bar 6 is the detection result of the conventional YOLO v8 network, and bar 7 is the detection result of the improved YOLO v8 network. It can be seen that in the same measured image, the conventional YOLO v8 network has 2 missed detections and 1 false detection; while the improved YOLO v8 has no false detection or missed detection.

[0112] When training and testing the regular YOLO v8 network and the improved YOLO v8 network, the performance indicators are compared. Figures 8 to 11 As shown in the figure, the performance indicators of the improved YOLO v8 network are all improved compared with the conventional YOLO v8 network. The loss curve comparison is shown in Figure 12 to Fig.17 As shown in the figure, it can be seen that the improved YOLO v8 network converges faster than the conventional YOLO v8 network. After the loss function converges, the loss value of the improved YOLO v8 network is smaller.

[0113] The present invention also discloses an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program so that the device executes the aforementioned defective solder ball detection method based on an improved YOLO v8 network.

[0114] The present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is executed, the device executing the computer program implements the aforementioned defective solder ball detection method based on the improved YOLO v8 network.

[0115] The "first" and "second" in the names such as "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.

[0116] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, and the memory can be various types of memory, which can be a random access memory, a read-only memory, a flash memory, etc., such as a read-only memory (English: read-only memory, ROM) / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0117] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A defective solder ball detection method based on an improved YOLO v8 network, characterized in that: include: Collect grayscale images of chip BGA packages and build a training data set for defective solder ball detection; Build an improved YOLO v8 network and train it using the training dataset; The trained improved YOLO v8 network is tested using the test data set to obtain the final improved YOLO v8 network; The defective solder balls are detected on the measured chip BGA image through the final improved YOLO v8 network to obtain the detection result, which includes the detection frame and test index corresponding to the defective solder balls; Build and improve the YOLO v8 network and train it using the training dataset, including: Build and improve the YOLO v8 network; The training data set is divided into several batches, each batch includes several grayscale images of chip BGA packages; For each batch of training data sets, feature extraction is performed by adjusting the backbone network input into the improved YOLO v8 network in the grid division to obtain at least three scales of basic feature maps for the batch; Input the basic feature map into the neck network in the improved YOLO v8 network to achieve feature fusion, and obtain fused feature maps of at least three scales; The fused feature map is input into the detection head network in the improved YOLO v8 network to perform bounding box regression reasoning and target classification reasoning, and output the bounding box correction value and category probability value of the predicted box; For each batch of training data sets, the improved YOLO v8 network is trained and optimized according to the bounding box correction value and category probability value of the prediction box; Improved YOLO v8 network including backbone network, neck network and detection head network, introduced double-layer routing attention mechanism BRA module in backbone network, adopted c2f-BiFormer module to replace part of c2f module, set attention mechanism BRA module in c2f-BiFormer module, adopted SPP-ELAN module to replace fast spatial pyramid pooling SPPF module; In the detection head network, it is necessary to build three groups of detection heads for large, medium and small scales. Each group of detection heads includes one one-to-one detection head and one one-to-many detection head. Both the one-to-one detection head and the one-to-many detection head adopt a decoupled structure, that is, they are composed of a target classification branch and a bounding box regression branch. In the training phase, the one-to-one detection head and the one-to-many detection head are optimized simultaneously; in the testing phase, the one-to-many detection head is not used, only the one-to-one detection head is used; For each batch of training data sets, the improved YOLO v8 network is trained and optimized according to the bounding box correction value and category probability value of the prediction box, including: Based on the bounding box correction value and category probability value of the predicted box, as well as the annotation box parameters, the matching degree between the predicted box and the annotated box is calculated; the one-to-many detection head selects multiple predicted boxes with the highest matching degree as positive samples for each annotated box, while the one-to-one detection head only selects one predicted box with the highest matching degree as a positive sample for each annotated box; Substitute the bounding box correction value, category probability value and corresponding annotation box parameters of each positive sample prediction box into the loss function to obtain the loss corresponding to the pair of prediction boxes and annotation boxes; superimpose the losses corresponding to all prediction boxes in the current batch of training data to obtain the loss value of the current batch of training data; based on the obtained loss value, back-propagate and update the network model parameters; All batches of training data are input into the network model one by one, and the network model is trained repeatedly until the network model parameters converge.

2. A defective solder ball detection method based on an improved YOLO v8 network according to claim 1, characterized in that: The calculation formula for the matching degree between the prediction box and the annotation box include: ， in, They are the prediction box and the annotation box respectively. The spatial prior indicating whether the anchor point is within the annotation box, is the classification score, is the intersection-over-union ratio between the predicted box and the labeled box, Hyperparameters for balancing target classification and regression positioning.

3. A defective solder ball detection method based on an improved YOLO v8 network according to claim 1, characterized in that: For each set of predicted boxes and labeled boxes, the loss The function calculation formula includes: ， in, represents the bounding box regression loss, is the distribution focal loss, represents the classification loss, is the gain coefficient corresponding to the above three losses; Bounding Box Regression Loss , the calculation formula includes: ， Among them, r is the regularization factor; Used to measure the degree of center alignment between the prediction box and the annotation box; represents the intersection-over-combination loss, is the intersection-over-union ratio between the predicted box and the labeled box, are two hyperparameters, and .

4. A defective solder ball detection method based on an improved YOLO v8 network according to claim 1, characterized in that: The loss of the one-to-one detection head is , the loss of one-to-many detection head is , the overall loss corresponding to the current batch of training data is: ， in, and is a normalized weighting coefficient between 0 and 1, ,and , The number of positive samples selected by the one-to-many detection head for each labeled box.

Images