Mini LED die bonding dispensing positioning and bonding pad abnormity detection method

Abstract

The invention relates to the technical field of optical positioning and visual detection, and provides a Mini LED die bonding dispensing positioning and bonding pad anomaly detection method, which comprises the following steps of: acquiring a plurality of pieces of Mini LED image data with Mini LED positioning information labels and Mini LED anomaly information labels to form a data set; constructing a positioning and anomaly detection model; training a positioning and anomaly detection model by using the data set to obtain a trained positioning and anomaly detection model; acquiring image data of a to-be-detected Mini LED, and inputting the image data of the to-be-detected Mini LED into the trained positioning and anomaly detection model to obtain die bonding and glue dispensing positioning information of the to-be-detected Mini LED and anomaly information of a bonding pad where the to-be-detected Mini LED is located; the trained positioning and anomaly detection model can simultaneously realize accurate positioning of MiniLED die bond dispensing and accurate detection of pad anomaly.

Description

Technical Field

[0001] The present invention relates to the technical field of optical positioning and visual detection, and more specifically, to a MiniLED die-bonding glue dispensing positioning and pad abnormality detection method. Background Art

[0002] With the continuous advancement of display technology, MiniLED, as an emerging backlight technology, is widely used in high-end display devices with its advantages of high brightness, high contrast and wider color gamut. In the MiniLED packaging process, die bonding dispensing technology plays a vital role, and its accuracy and reliability directly determine the performance and stability of the final product.

[0003] The core of die bonding dispensing is to precisely fix the tiny LED chip on the substrate or pad. This process requires extremely high positioning accuracy. Any slight deviation will lead to poor electrical connection between the chip and the substrate, which will affect the luminous efficiency and life of the LED. In MiniLED technology, due to the extremely small chip size and limited pad area, a slight deviation may cause a short circuit, open circuit or chip damage. Therefore, accurate positioning can not only improve the yield of the die bonding process, but also ensure uniform contact between the pad and the chip, thereby improving the overall quality of the product.

[0004] To achieve this goal, modern die bonding dispensing equipment is usually combined with a high-precision visual positioning system, which captures the position data of the substrate and chip in real time, adjusts the movement path of the dispensing head, and ensures that the glue is evenly distributed and accurately positioned. At the same time, advanced deep learning algorithms are also used to optimize positioning and dispensing parameters to cope with changes and challenges that may arise during the production process. Therefore, the current die bonding dispensing positioning and pad detection of MiniLED are facing challenges. In actual production, due to inaccurate positioning or defects in the pad, the final die bonding yield is not high, resulting in increased production costs and reduced production efficiency, which affects the production and development of MiniLED.

[0005] In order to improve the accuracy and robustness of MiniLED die-bonding positioning and pad status detection, the present invention proposes a MiniLED die-bonding glue dispensing positioning and pad detection method. Summary of the invention

[0006] In order to overcome the defects of low precision in MiniLED die-bonding positioning and pad status detection in the above-mentioned prior art, the present invention provides a MiniLED die-bonding glue positioning and pad abnormality detection method with high positioning and detection precision.

[0007] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0008] The MiniLED is soldered on the pad by using a die-bonding glue dispensing technology, and the MiniLED die-bonding glue dispensing positioning and pad abnormality detection method comprises the following steps:

[0009] Collect several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set;

[0010] Construct a positioning and anomaly detection model for calculating the MiniLED die bonding glue positioning information and the anomaly information of the MiniLED pad;

[0011] Using the data set to train the positioning and anomaly detection model to obtain a trained positioning and anomaly detection model;

[0012] The image data of the MiniLED to be detected is obtained, and the image data of the MiniLED to be detected is input into the trained positioning and anomaly detection model to obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0013] The present invention also proposes a MiniLED die-bonding glue dispensing positioning and pad abnormality detection system for realizing the MiniLED die-bonding glue dispensing positioning and pad abnormality detection method, the system comprising:

[0014] A data acquisition module is used to collect a number of MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set;

[0015] A model building module, used to build a positioning and anomaly detection model for calculating the positioning information of the MiniLED die bonding glue dispensing and the anomaly information of the MiniLED pad;

[0016] A model training module, used to train the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model;

[0017] The positioning and detection module is used to obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0018] The present invention also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned MiniLED solid crystal dispensing positioning and pad abnormality detection method.

[0019] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0020] Acquire image data of MiniLED; train the positioning and anomaly detection model based on the image data of MiniLED, and acquire the die-bonding glue positioning information of the MiniLED to be detected and the anomaly information of the soldering pad where the MiniLED to be detected is located based on the trained positioning and anomaly detection model, wherein the trained positioning and anomaly detection model can simultaneously realize the precise positioning of the MiniLED die-bonding glue and the precise detection of soldering pad anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the process of MiniLED die bonding glue positioning and pad abnormality detection method proposed in Example 1;

[0022] Figure 2 It is the training evaluation parameter diagram proposed in Example 2;

[0023] Figure 3 This is a schematic diagram of the overall framework of the positioning and anomaly detection model proposed in Example 2;

[0024] Figure 4 This is a schematic diagram of the structure of the C2F layer proposed in Example 2;

[0025] Figure 5 This is a schematic diagram of the SPPF layer proposed in Example 2;

[0026] Figure 6 This is an overall framework diagram of a MiniLED die bonding glue positioning and pad anomaly detection system proposed in Example 3. DETAILED DESCRIPTION

[0027] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0028] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0029] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0031] Example 1

[0032] This embodiment proposes a MiniLED die bonding glue positioning and pad abnormality detection method. Figure 1Schematic diagram of the process of MiniLED die bonding glue positioning and pad abnormality detection method of this embodiment;

[0033] In this embodiment, a MiniLED die-bonding dispensing positioning and pad abnormality detection method is proposed. The MiniLED is soldered on the pad by die-bonding dispensing technology. The method includes the following steps:

[0034] S1: Collect several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set;

[0035] S2: constructing a positioning and anomaly detection model for calculating the positioning information of the MiniLED die bonding glue dispensing and the anomaly information of the MiniLED pad;

[0036] S3: Using the data set to train the positioning and anomaly detection model to obtain a trained positioning and anomaly detection model;

[0037] S4: Obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into a trained positioning and anomaly detection model, and obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0038] In the specific implementation process, image data of MiniLED is obtained; the positioning and abnormality detection model is trained based on the image data of MiniLED, and the die-bonding dispensing positioning information of the MiniLED to be detected and the abnormality information of the soldering pad where the MiniLED to be detected is located are obtained based on the trained positioning and abnormality detection model. The trained positioning and abnormality detection model can simultaneously realize the precise positioning of the MiniLED die-bonding dispensing and the precise detection of soldering pad abnormalities.

[0039] As an exemplary illustration, the data set includes one thousand seven hundred MiniLED image data with MiniLED positioning information labels and MiniLED abnormality information labels.

[0040] In an optional embodiment, the trained positioning and anomaly detection model includes a trained YOLOv8 model.

[0041] In an optional embodiment, before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, the structure of the trained positioning and anomaly detection model is optimized to obtain an optimized positioning and anomaly detection model;

[0042] The steps of optimizing the structure of the trained positioning and anomaly detection model include:

[0043] Replace the standard convolutions in the backbone network of the trained YOLOv8 model with depthwise separable convolutions.

[0044] In an optional embodiment, before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, the structure of the trained positioning and anomaly detection model is optimized twice to obtain an optimized positioning and anomaly detection model;

[0045] The steps of performing secondary optimization on the structure of the trained positioning and anomaly detection model include:

[0046] The standard convolution in the backbone network of the trained YOLOv8 model is replaced with depthwise separable convolution, and a convolutional block attention module is added after the cross-stage partial stacking module in the head of the trained YOLOv8 model.

[0047] In an optional embodiment, the optimized positioning and anomaly detection model includes a feature acquisition module, a feature fusion module and a positioning and detection module;

[0048] The feature acquisition module is used to extract features from MiniLED image data, the first layer of the feature acquisition module is a Conv layer, the second, fourth, sixth and eighth layers are DWConv layers, the third, fifth, seventh and ninth layers are C2F layers, and the tenth layer is an SPPF layer; the layers of the feature acquisition module are connected in sequence, and the fifth, seventh and tenth layers are connected to the feature fusion module, and the fifth, seventh and tenth layers output extracted features F1, F2 and F3 respectively;

[0049] The feature fusion module is used to fuse and enhance the extracted features output by the feature acquisition module into enhanced features, and the 1st and 5th layers of the feature fusion module are Upsample layers, the 2nd, 6th, 10th and 14th layers are Concat layers, the 3rd, 7th, 11th and 15th layers are C2F layers, the 4th, 8th, 12th and 16th layers are CBAM layers, and the 9th and 13th layers are Conv layers; the layers of the feature fusion module are connected in sequence, and the 4th layer is connected to the 10th layer, the 1st and 14th layers are connected to the 10th layer of the feature acquisition module, the 2nd layer is connected to the 7th layer of the feature acquisition module, and the 6th layer is connected to the 5th layer of the feature acquisition module; the 8th, 12th and 16th layers are connected to the positioning and detection module, and the 8th, 12th and 16th layers output enhanced features F7, F9 and F10, respectively. 11 ;

[0050] The positioning and detection module converts the enhanced features output by the feature fusion module into the positioning information of the MiniLED and the abnormal information of the pad where the MiniLED is located based on the decoupling head.

[0051] In an optional embodiment, the step of the feature fusion module fusing and enhancing the extracted features output by the feature acquisition module into enhanced features comprises:

[0052] The feature fusion module causes the extracted feature F3 to pass through the first and second layers and then be fused with the extracted feature F2 to obtain the enhanced feature F4. The enhanced feature F4 passes through the third and fourth layers and then be fused with the extracted feature F5 to obtain the enhanced feature F6. The enhanced feature F6 passes through the seventh and eighth layers and then be fused with the extracted feature F1 to obtain the enhanced feature F6. The enhanced feature F7 passes through the ninth and tenth layers and then be fused with the enhanced feature F5 to obtain the enhanced feature F8. The enhanced feature F8 passes through the eleventh and twelfth layers and then be fused with the enhanced feature F9. The enhanced feature F9 passes through the thirteenth and fourteenth layers and then be fused with the extracted feature F3 to obtain the enhanced feature F9. 10 , enhanced feature F 10 After the 15th and 16th layers, the enhanced feature F is obtained 11 .

[0053] As an exemplary illustration, the Conv represents convolution; the DWConv represents depthwise separable convolution; the C2F layer represents the C2F module in YOLOv8; and the SPPF layer represents the SPPF module in YOLOv8.

[0054] As an exemplary explanation, Upsample means upsampling; Concat stands for Concatenate, which means connection; CBAM stands for Convolutional Block Attention, which means convolutional attention mechanism.

[0055] In an optional embodiment, the MiniLED die bonding dispensing positioning information includes: the coordinates of the upper left corner of the MiniLED positioning frame, and the coordinates of the lower right corner of the MiniLED positioning frame;

[0056] The abnormal information of the pad where the MiniLED is located includes abnormal result information and normal result information;

[0057] In the model reasoning link, the positioning and anomaly detection model calculates the anomaly information of the pad where the MiniLED is located based on the die bonding and glue dispensing positioning information of the MiniLED, and the steps include:

[0058] Calculating the angle deviation and / or line width deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED;

[0059] If the angle deviation of the MiniLED is greater than the angle deviation threshold, or its line width deviation is greater than the line width deviation threshold, the pad where the MiniLED is located is judged to be abnormal and the abnormal result information is output; otherwise, the pad where the MiniLED is located is judged to be normal and the normal result information is output.

[0060] In an optional embodiment, the calculation expression for calculating the angle deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes:

[0061]

[0062] Wherein, θ represents the angle deviation of the MiniLED; x0 and y0 represent the upper left corner abscissa and ordinate of the positioning frame of the MiniLED, respectively; x1 and y1 represent the lower right corner abscissa and ordinate of the positioning frame of the MiniLED, respectively; θ0 represents the diagonal angle of the MiniLED;

[0063] The calculation expression for calculating the line width deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes:

[0064] d=|y0-y1|

[0065] Wherein, y0 represents the upper left corner ordinate of the MiniLED positioning frame, and y1 represents the lower right corner ordinate of the MiniLED positioning frame;

[0066] The abnormal result information includes an abnormal mark;

[0067] The normal result information includes the upper left corner coordinates, center coordinates and upper right corner coordinates of the positioning frame of the MiniLED, and the angle deviation of the MiniLED;

[0068] The calculation expression of the coordinates of the center point of the positioning frame of the MiniLED includes:

[0069] x2=x0+dx

[0070] y2=y0+dy

[0071]

[0072] In the formula, x2 and y2 represent the horizontal and vertical coordinates of the center point of the MiniLED positioning frame respectively; x0 and y0 represent the horizontal and vertical coordinates of the upper left corner of the MiniLED positioning frame respectively; x1 and y1 represent the horizontal and vertical coordinates of the lower right corner of the MiniLED positioning frame respectively.

[0073] As an exemplary description, several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels are collected under several backgrounds and several lighting conditions when forming a data set.

[0074] As an exemplary explanation, when collecting data sets, clear images of MiniLEDs at multiple angles and under different lighting conditions are collected to improve the labeling accuracy of the data sets; when labeling the labels in the data sets, a layered labeling strategy is adopted to first label the position of the MiniLED, and then label the pad status based on its position information to improve the accuracy of pad status detection.

[0075] Example 2

[0076] This embodiment is based on the MiniLED die bonding glue positioning and pad abnormality detection method proposed in Example 1, and proposes the following implementation examples:

[0077] S1: First, collect MiniLED image data, which covers a variety of backgrounds, lighting conditions, and pad states. The collected data should be as diverse as possible to ensure the generalization ability of the model.

[0078] S2: Label the collected data. The labeling process involves manually marking the specific location of the MiniLED and its pad in each image. The labeling should be as accurate as possible to ensure the high quality of the training data. The labeling results include the location information of the MiniLED and the label of the pad status. Generate a dataset containing the location information of the MiniLED and the label of the abnormal pad image set, and divide the labeled dataset into training set, validation set and test set in a ratio of 7:1.5:1.5. This dataset is used to train, validate and test the model to ensure that the model can accurately locate the MiniLED.

[0079] S3: Based on the dataset annotated in step S2, preliminary training is performed based on the YOLOv8 model. In this process, data augmentation techniques (such as random cropping, color transformation, etc.) are used to expand the scale and diversity of the dataset and reduce the risk of overfitting. During the training process, the optimizer Adam is used to continuously adjust the weights of the model to minimize the loss function. The hyperparameters during training (such as learning rate, batch size, number of iterations, etc.) are also set reasonably to ensure model convergence. At the same time, the performance of the model is evaluated using the validation set, and its adaptability to MiniLED positioning and pad status detection tasks is analyzed by calculating indicators such as the model's precision and recall on the validation set. The evaluation results of this stage provide a reference for the optimization of the subsequent model structure.

[0080] S4: Based on the results of the preliminary training and validation set evaluation of S3, analyze the performance of the model in terms of speed, accuracy, etc., and identify the parts that need to be improved. The standard convolution in the backbone network of the YOLOv8 model is replaced with a depthwise separable convolution, and the CBAM attention mechanism is introduced after the C2F module of the detection head of the model. The optimized model is the positioning and anomaly detection model proposed in Example 1; the structure includes a feature acquisition module, a feature fusion module, and a positioning and detection module. The feature acquisition module has a total of 10 layers, the first layer is a Conv layer, the second, fourth, sixth, and eighth layers are DWConv layers, the third, fifth, seventh, and ninth layers are C2F layers, and the tenth layer is an SPPF layer. Features are extracted from the fifth, seventh, and tenth layers as inputs to the feature fusion layer, and the outputs of the fifth, seventh, and tenth layers are extracted features F1, extracted features F2, and extracted features F3, respectively; The processing process of the feature fusion module is as follows: after upsampling, the extracted feature F3 is fused with the extracted feature F2 to obtain the enhanced feature F4; the enhanced feature F4 passes through the C2F1 layer and the CBAM attention mechanism to obtain the enhanced feature F5; the enhanced feature F5 is upsampled in turn and fused with the extracted feature F1 to obtain the enhanced feature F6; the enhanced feature F6 passes through the C2F2 layer and the CBAM attention mechanism to obtain the enhanced feature F7; the enhanced feature F7 passes through the convolution layer in turn and is fused with the enhanced feature F5 to obtain the enhanced feature F8; the enhanced feature F8 passes through the C2F3 layer and the CBAM attention mechanism to obtain the enhanced feature F9; the enhanced feature F9 passes through the convolution layer in turn and is fused with the extracted feature F3 to obtain the enhanced feature F 10 , enhanced feature F 10 After the C2F4 layer and the CBAM attention mechanism, the enhanced feature F is obtained 11 Finally, the enhanced features F7, F9, and F 11 As the output of the feature fusion layer; the detection layer includes three identical decoupling heads, which enhance the features F7, F9, and F 11 The data are input into the three decoupling heads respectively, and the model is retrained based on the optimized YOLOv8 model structure. Data enhancement technology is also used in the training process, and the number of training rounds is set to 200 rounds, the batch size is 32, and the initial learning rate is set to 1×10 -3 . The original training set and validation set are used for training and evaluation. After training, the validation set is used again to evaluate the optimized model and compare its performance with that of the initial model. By analyzing the comparison results, the effect of the model structure optimization is verified, including detection accuracy, speed, and adaptability to MiniLED positioning and pad status detection tasks.

[0081] S5: The trained and optimized YOLOv8 model is tested for inference. In the inference stage, the model is applied to the MiniLED image data of the test set to obtain the positioning frame of the pad. During the inference process, the model will generate the coordinates of the positioning frame based on the features in the input image. Each positioning frame consists of the coordinates of the upper left corner and the lower right corner, accurately marking the position of the MiniLED and its pad. These positioning frames will serve as the basis for subsequent analysis and calculation to help evaluate the actual application effect of the model.

[0082] S6: According to the positioning frame obtained in step S5, first extract the center position of each pad and its distance from the image boundary. By calculating the coordinates of the upper left corner and lower right corner of the positioning frame, the center position of the pad can be accurately determined, and its distance to the left and upper boundaries of the image can be measured. According to the threshold pre-set by the user, determine whether the pad is abnormal, and verify the positioning accuracy and pad status detection ability of the model in practical applications. By performing statistical analysis on the detection results of the inference test set, evaluate key indicators such as the accuracy of position detection, the accuracy of pad status, and the speed of model inference. Combined with these analysis results, the adaptability and stability of the model can be further verified, providing a reliable basis for application in actual production.

[0083] As an example, the data enhancement strategies introduced in the training process in step S3, such as random cropping, color transformation, CutMix, etc., can effectively improve the generalization ability of the model. By diversifying the training data, the model can better cope with various changes that may occur in the actual production environment, such as lighting conditions, background complexity, etc., thereby improving the accuracy of MiniLED positioning and pad status detection.

[0084] As an example, Figure 2 This is a training evaluation parameter diagram proposed in this embodiment; Figure 2 The evaluation results of the S3 step are shown.

[0085] As an exemplary illustration, the Conv layer and DWConv layer described in step S4 are both composed of a convolutional layer, a BN layer (BatchNorm), and a SiLU activation function. The convolutional layer is used to extract local features from the input feature map. The convolutional layer slides on the input through multiple filters (or convolution kernels) to generate multiple output feature maps, each of which corresponds to a different filter. After the convolution operation, the BN layer performs batch normalization to standardize each channel in the output feature map. This step helps to reduce the differences between features of different channels, stabilize the training process, and accelerate convergence. The SiLU activation function is also called the Swish activation function. This activation function enhances the expressive power of the model by introducing nonlinear characteristics and performing nonlinear transformations on the normalized feature maps. It retains the advantages of ReLU while avoiding the "dead zone" problem of ReLU activation.

[0086] As an example, Figure 3 This is a schematic diagram of the overall framework of the positioning and anomaly detection model proposed in this embodiment; Figure 3 The overall framework diagram of the positioning and anomaly detection model is shown.

[0087] As an example, Figure 4 This is a schematic diagram of the structure of the C2F layer proposed in this embodiment; Figure 4 The structure of the C2F layer is shown; as an example, Figure 4 As shown, the C2F module described in step S4 consists of four branches. The first branch consists of a CBS1 module (including convolutional layer, batch normalization layer and SiLU activation function) for extracting basic features; the second branch consists of a CBS1 module and a separation module, and the separation module is used to enhance the feature expression capability; the third branch consists of a CBS1 module, the same separation module (Split) and a bottleneck module (Bottleneck), and the bottleneck module further refines information by compressing and enhancing features; the fourth branch consists of a CBS1 module, the same separation module and two repeated bottleneck modules to further strengthen the features. The output feature maps of the four branches are finally fused (Concat) together and integrated through the CBS2 module to generate the final output feature map.

[0088] As an example, Figure 5 This is a schematic diagram of the SPPF module proposed in this embodiment, as shown in Figure 5 As shown in FIG. 1 , the SPPF module described in step S4 consists of the following parts: first, the input feature map passes through the CBS module to initially reduce the number of channels of the feature map and reduce the amount of calculation; then, the feature map is subjected to pooling operations of different scales through multiple maximum pooling layers (MaxPool), and these pooling layers use pooling kernels of different sizes (e.g., 5x5, 3x3, 2x2) to generate feature maps of multiple different scales. Subsequently, these pooled feature maps are spliced ​​with the initial input feature map in the channel dimension to fuse multi-scale information; finally, the spliced ​​feature map is processed by the CBS module to integrate information from different scales and generate the final output feature map.

[0089] As an exemplary illustration, the standard convolution in the YOLOv8 model backbone network described in step S4 is replaced with a depthwise separable convolution (DW convolution), which decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution, significantly reducing the amount of calculation and parameters, thereby improving the inference speed of the model. Due to the low computational complexity of DW convolution, memory consumption is also reduced accordingly. This is particularly important for deployment on memory-constrained embedded systems or edge devices, and can effectively reduce hardware costs. By replacing the convolution operation, the size and complexity of the model are effectively controlled, making it more lightweight, thereby improving the practical application flexibility of the model.

[0090] As an exemplary illustration, the CBAM attention mechanism is introduced after the C2F module in the detection head of the YOLOv8 model described in step S4. By weighting the channel and spatial features, the key feature areas are automatically focused on, which improves the model's ability to capture MiniLED features. This feature-focused capability enhances the model's detection accuracy in complex backgrounds and pad states. CBAM can effectively combine feature information of different scales and improve the model's ability to detect small targets. In practical applications, this multi-scale feature fusion can better cope with target deformation and occlusion, and improve the overall reliability of detection. By dynamically adjusting the weights of features, CBAM enhances the model's robustness to noise and background interference, and reduces the possibility of false detection and missed detection.

[0091] As an exemplary illustration, a convolutional block attention module (CBAM) is added after the cross-stage partial stacking module (C2F) in the model head to enhance the model's ability to extract and fuse multi-scale features, thereby improving the accuracy of positioning and pad status detection.

[0092] As an example, the S5 step uses the GPU to accelerate the optimized YOLOv8 model, which can greatly improve the model's reasoning speed and meet the needs of real-time detection on the production line. This acceleration not only ensures the continuity of the production line, but also reduces the risk of potential defects not being discovered in time due to delays.

[0093] As an exemplary description, the method of the present application collects image data containing MiniLEDs and annotates them to generate a set of image labels containing MiniLED position information and abnormal pads, and sets thresholds for allowable line width deviations and angle changes; based on the annotated data set, the optimized YOLOv8 model is trained to enable it to identify and locate MiniLEDs. After the model is trained, the image to be detected is input, and the positioning frame of the MiniLED can be output. The center position of the MiniLED and the distance to the upper and left boundaries of the positioning frame are calculated through the coordinate information of the positioning frame, and the pad is detected based on the threshold set by the user. The trained model is verified and optimized to ensure that its performance in detection accuracy and speed meets production requirements; based on the detection results, the center position of each MiniLED and the distance to the upper and left boundaries of the positioning frame are analyzed to ensure the accurate positioning of the MiniLED and the status of its pads.

[0094] As an exemplary illustration, the method of the present application collects image data containing MiniLEDs and forms a data set after annotation for training the YOLOv8 model. The data set contains the location information of MiniLEDs and the labels of the abnormal pad image set. After the model training is completed, it can detect images in real time and directly output the center position of the MiniLED and the distance information to both sides. The user sets the distance threshold to determine whether the pad is abnormal. If the pad is normal, the pad position, length, width and horizontal tilt angle in the image are returned. If the pad is abnormal, the abnormal mark is returned. In order to further optimize the performance of the model, the standard convolution layer in the backbone network is replaced by a depth-separable convolution to reduce the amount of calculation and increase the detection speed. At the same time, a convolution block attention module is added after the cross-stage partial stacking module of the model head, which enhances the model's ability to extract and fuse multi-scale features. The method of the present invention helps to improve the accuracy and robustness of MiniLED solid crystal positioning and pad status detection.

[0095] As an example, by introducing deep separable convolution and CBAM attention mechanism, the optimized YOLOv8 model can more accurately identify the center position and pad status of MiniLED when processing MiniLED detection tasks. Deep separable convolution effectively reduces the computational complexity, while CBAM enhances the model's focus on key features and significantly improves detection accuracy; the optimization of the model structure reduces the amount of calculation, allowing YOLOv8 to quickly perform inference with lower computing resources. The application of deep separable convolution enables the model to perform well in real-time detection tasks, especially for rapid response requirements on production lines; after the introduction of the CBAM attention mechanism, the model's robustness to complex backgrounds and noise is significantly improved. This means that in an actual production environment, even in the face of background changes and different lighting conditions, the model can still stably locate MiniLEDs and detect pad status.

[0096] Example 3

[0097] This embodiment proposes a MiniLED die-bonding glue dispensing positioning and pad anomaly detection system, which is used to implement the MiniLED die-bonding glue dispensing positioning and pad anomaly detection method proposed in Example 1.

[0098] Figure 6 This is the overall framework diagram of the MiniLED die bonding positioning and pad abnormality detection system of this embodiment.

[0099] The MiniLED die bonding dispensing positioning and pad abnormality detection system includes:

[0100] A data acquisition module is used to collect a number of MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set;

[0101] A model building module, used to build a positioning and anomaly detection model for calculating the positioning information of the MiniLED die bonding glue dispensing and the anomaly information of the MiniLED pad;

[0102] A model training module, used to train the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model;

[0103] The positioning and detection module is used to obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0104] This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the MiniLED die-bonding glue positioning and pad abnormality detection method described in Example 1.

[0105] It can be understood that the MiniLED die bonding positioning and pad anomaly detection system and computer equipment of this embodiment improve the method of Example 1. The optional items in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0106] The same or similar reference numerals correspond to the same or similar components;

[0107] The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0108] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A MiniLED die bonding glue positioning and pad abnormality detection method, characterized in that: The MiniLED is soldered on the pad by die bonding glue dispensing technology, and the method comprises the following steps: Collect several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set; Construct a positioning and anomaly detection model for calculating the MiniLED die bonding glue positioning information and the anomaly information of the MiniLED pad; Using the data set to train the positioning and anomaly detection model to obtain a trained positioning and anomaly detection model; The image data of the MiniLED to be detected is obtained, and the image data of the MiniLED to be detected is input into the trained positioning and anomaly detection model to obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

2. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 1, characterized in that: The trained positioning and anomaly detection model includes a trained YOLOv8 model.

3. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 2 is characterized in that: Before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, optimizing the structure of the trained positioning and anomaly detection model to obtain an optimized positioning and anomaly detection model; The steps of optimizing the structure of the trained positioning and anomaly detection model include: Replace the standard convolutions in the backbone network of the trained YOLOv8 model with depthwise separable convolutions.

4. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 2, characterized in that: Before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, performing secondary optimization on the structure of the trained positioning and anomaly detection model to obtain an optimized positioning and anomaly detection model; The steps of performing secondary optimization on the structure of the trained positioning and anomaly detection model include: The standard convolution in the backbone network of the trained YOLOv8 model is replaced with depthwise separable convolution, and a convolutional block attention module is added after the cross-stage partial stacking module in the head of the trained YOLOv8 model.

5. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 4, characterized in that: The optimized positioning and anomaly detection model includes a feature acquisition module, a feature fusion module and a positioning and detection module; The feature acquisition module is used to extract features from MiniLED image data, the first layer of the feature acquisition module is a Conv layer, the second, fourth, sixth and eighth layers are DWConv layers, the third, fifth, seventh and ninth layers are C2F layers, and the tenth layer is an SPPF layer; the layers of the feature acquisition module are connected in sequence, and the fifth, seventh and tenth layers are connected to the feature fusion module, and the fifth, seventh and tenth layers output extracted features F1, F2 and F3 respectively; The feature fusion module is used to fuse and enhance the extracted features output by the feature acquisition module into enhanced features, and the 1st and 5th layers of the feature fusion module are Upsample layers, the 2nd, 6th, 10th and 14th layers are Concat layers, the 3rd, 7th, 11th and 15th layers are C2F layers, the 4th, 8th, 12th and 16th layers are CBAM layers, and the 9th and 13th layers are Conv layers; the layers of the feature fusion module are connected in sequence, and the 4th layer is connected to the 10th layer, the 1st and 14th layers are connected to the 10th layer of the feature acquisition module, the 2nd layer is connected to the 7th layer of the feature acquisition module, and the 6th layer is connected to the 5th layer of the feature acquisition module; the 8th, 12th and 16th layers are connected to the positioning and detection module, and the 8th, 12th and 16th layers output enhanced features F7, F9 and F10, respectively. 11 ; The positioning and detection module converts the enhanced features output by the feature fusion module into the positioning information of the MiniLED and the abnormal information of the pad where the MiniLED is located based on the decoupling head.

6. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 5, characterized in that: The step of the feature fusion module fusing and enhancing the extracted features output by the feature acquisition module into enhanced features comprises: The feature fusion module causes the extracted feature F3 to pass through the first and second layers and then be fused with the extracted feature F2 to obtain the enhanced feature F4. The enhanced feature F4 passes through the third and fourth layers and then be fused with the extracted feature F5 to obtain the enhanced feature F6. The enhanced feature F6 passes through the seventh and eighth layers and then be fused with the extracted feature F1 to obtain the enhanced feature F6. The enhanced feature F7 passes through the ninth and tenth layers and then be fused with the enhanced feature F5 to obtain the enhanced feature F8. The enhanced feature F8 passes through the eleventh and twelfth layers and then be fused with the enhanced feature F9. The enhanced feature F9 passes through the thirteenth and fourteenth layers and then be fused with the extracted feature F3 to obtain the enhanced feature F9. 10 , enhanced feature F 10 After the 15th and 16th layers, the enhanced feature F is obtained 11 .

7. The MiniLED die bonding glue positioning and pad abnormality detection method according to any one of claims 1 to 6, characterized in that: The MiniLED die bonding glue positioning information includes: the coordinates of the upper left corner of the MiniLED positioning frame, and the coordinates of the lower right corner of the MiniLED positioning frame; The abnormal information of the pad where the MiniLED is located includes abnormal result information and normal result information; In the model reasoning link, the positioning and anomaly detection model calculates the anomaly information of the pad where the MiniLED is located based on the die bonding and glue dispensing positioning information of the MiniLED, and the steps include: Calculating the angle deviation and / or line width deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED; If the angle deviation of the MiniLED is greater than the angle deviation threshold, or its line width deviation is greater than the line width deviation threshold, the pad where the MiniLED is located is judged to be abnormal and the abnormal result information is output; otherwise, the pad where the MiniLED is located is judged to be normal and the normal result information is output.

8. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 7, characterized in that: The calculation expression for calculating the angle deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes: Wherein, θ represents the angle deviation of the MiniLED; x0 and y0 represent the upper left corner abscissa and ordinate of the positioning frame of the MiniLED, respectively; x1 and y1 represent the lower right corner abscissa and ordinate of the positioning frame of the MiniLED, respectively; θ0 represents the diagonal angle of the MiniLED; The calculation expression for calculating the line width deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes: d=|y0-y1| Wherein, y0 represents the upper left corner ordinate of the MiniLED positioning frame, and y1 represents the lower right corner ordinate of the MiniLED positioning frame; The abnormal result information includes an abnormal mark; The normal result information includes the upper left corner coordinates, center coordinates and upper right corner coordinates of the positioning frame of the MiniLED, and the angle deviation of the MiniLED; The calculation expression of the coordinates of the center point of the positioning frame of the MiniLED includes: x2=x0+dx y2=y0+dy In the formula, x2 and y2 represent the horizontal and vertical coordinates of the center point of the MiniLED positioning frame respectively; x0 and y0 represent the horizontal and vertical coordinates of the upper left corner of the MiniLED positioning frame respectively; x1 and y1 represent the horizontal and vertical coordinates of the lower right corner of the MiniLED positioning frame respectively.

9. A MiniLED die-bonding glue positioning and pad abnormality detection system, used to implement the MiniLED die-bonding glue positioning and pad abnormality detection method according to any one of claims 1 to 8, characterized in that: include: A data acquisition module is used to collect a number of MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set; A model building module, used to build a positioning and anomaly detection model for calculating the positioning information of the MiniLED die bonding glue dispensing and the anomaly information of the MiniLED pad; A model training module, used to train the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model; The positioning and detection module is used to obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

10. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that: When the computer-readable instructions are executed by the processor, the processor executes the steps of the MiniLED die-bonding glue positioning and pad abnormality detection method according to any one of claims 1 to 8.

Images