Mini LED die bonding dispensing positioning and pad anomaly detection method

Abstract

The present application relates to the technical field of optical positioning and visual detection, and proposes a MiniLED die bonding dispensing positioning and pad anomaly detection method, comprising the following steps: collecting a plurality of MiniLED image data with MiniLED positioning information labels and MiniLED anomaly information labels to form a data set; constructing a positioning and anomaly detection model; training the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model; obtaining image data of a MiniLED to be detected, inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model to obtain die bonding dispensing positioning information of the MiniLED to be detected and anomaly information of a pad where the MiniLED to be detected is located; the trained positioning and anomaly detection model can simultaneously realize accurate positioning of MiniLED die bonding dispensing and accurate detection of pad anomaly.

Description

Technical Field

[0001] The present invention relates to the technical field of optical positioning and visual inspection, and more specifically, to a method for MiniLED die bonding and glue dispensing positioning and solder pad anomaly detection. Background Art

[0002] With the continuous advancement of display technology, MiniLED, as an emerging backlight technology, is widely used in high-end display devices due to its advantages of high brightness, high contrast, and wider color gamut. In the MiniLED packaging process, die-bonding dispensing technology plays a crucial role, and its precision and reliability directly determine the performance and stability of the final product.

[0003] The core of die-bonding dispensing lies in precisely securing tiny LED chips to substrates or pads. This process requires extremely high positioning accuracy; any slight deviation can lead to poor electrical connection between the chip and substrate, affecting the LED's luminous efficiency and lifespan. In MiniLED technology, due to the extremely small chip size and limited pad area, even the slightest deviation can cause short circuits, open circuits, or even chip damage. Therefore, accurate positioning not only improves the yield of the die-bonding process but also ensures uniform contact between the pad and chip, thereby enhancing overall product quality.

[0004] To achieve this goal, modern die-bonding dispensing equipment typically incorporates high-precision visual positioning systems. By capturing real-time positional data of the substrate and chip, it adjusts the dispensing head's movement path to ensure even glue distribution and precise positioning. At the same time, advanced deep learning algorithms are also used to optimize positioning and dispensing parameters to address potential variations and challenges during the production process. Consequently, MiniLED die-bonding dispensing positioning and pad inspection currently face challenges. In actual production, inaccurate positioning or defective pads result in a low final die-bonding yield, increasing production costs and reducing efficiency, hindering the development of MiniLED production.

[0005] In order to improve the accuracy and robustness of MiniLED die-bonding positioning and pad status detection, the present invention proposes a MiniLED die-bonding dispensing positioning and pad detection method. Summary of the Invention

[0006] In order to overcome the defects of low accuracy in MiniLED die-bonding positioning and pad status detection in the above-mentioned prior art, the present invention provides a MiniLED die-bonding dispensing positioning and pad abnormality detection method with high positioning and detection accuracy.

[0007] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0008] The MiniLED is soldered on the pad using a die-bonding glue dispensing technology. The MiniLED die-bonding glue dispensing positioning and pad abnormality detection method includes the following steps:

[0009] Collect several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set;

[0010] Build a positioning and anomaly detection model to calculate the MiniLED die bonding and dispensing positioning information and anomaly information of the MiniLED pads.

[0011] Using the data set to train the positioning and anomaly detection model to obtain a trained positioning and anomaly detection model;

[0012] Obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0013] The present invention also proposes a MiniLED die-bonding glue positioning and pad anomaly detection system for implementing the above-mentioned MiniLED die-bonding glue positioning and pad anomaly detection method, the system comprising:

[0014] The data acquisition module is used to collect a number of MiniLED image data with MiniLED positioning information labels and MiniLED abnormality information labels to form a data set;

[0015] A model building module is used to build a positioning and anomaly detection model for calculating the MiniLED die bonding and dispensing positioning information and anomaly information of the MiniLED pads;

[0016] A model training module, configured to train the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model;

[0017] The positioning and detection module is used to obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0018] The present invention also proposes a computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned MiniLED die-bonding glue positioning and pad abnormality detection method.

[0019] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0020] Acquire image data of the MiniLED; train the positioning and anomaly detection model based on the image data of the MiniLED, and obtain the die-bonding dispensing positioning information of the MiniLED to be detected and the anomaly information of the soldering pad where the MiniLED to be detected is located based on the trained positioning and anomaly detection model. The trained positioning and anomaly detection model can simultaneously achieve precise positioning of the MiniLED die-bonding dispensing and precise detection of soldering pad anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the MiniLED die-bonding dispensing positioning and pad anomaly detection method proposed in Example 1;

[0022] Figure 2 This is the training evaluation parameter diagram proposed in Example 2;

[0023] Figure 3 This is a schematic diagram of the overall framework of the positioning and anomaly detection model proposed in Example 2;

[0024] Figure 4 This is a schematic diagram of the structure of the C2F layer proposed in Example 2;

[0025] Figure 5 This is a schematic diagram of the SPPF layer proposed in Example 2;

[0026] Figure 6 This is an overall framework diagram of a MiniLED die-bonding glue positioning and pad anomaly detection system proposed in Example 3. DETAILED DESCRIPTION

[0027] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present embodiment;

[0028] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0029] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0030] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0031] Example 1

[0032] This embodiment proposes a MiniLED die-bonding glue positioning and pad abnormality detection method. Figure 1Schematic diagram of the process of MiniLED die bonding dispensing positioning and pad abnormality detection method of this embodiment;

[0033] In this embodiment, a method for MiniLED die-bonding dispensing positioning and pad abnormality detection is proposed. The MiniLED is soldered to the pad using die-bonding dispensing technology. The method includes the following steps:

[0034] S1: Collect several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set;

[0035] S2: Constructing a positioning and anomaly detection model for calculating the MiniLED die bonding and dispensing positioning information and the anomaly information of the MiniLED pad;

[0036] S3: Using the data set to train the positioning and anomaly detection model to obtain a trained positioning and anomaly detection model;

[0037] S4: Obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue dispensing positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0038] During the specific implementation process, image data of MiniLED is obtained; the positioning and abnormality detection model is trained based on the image data of MiniLED, and the die-bonding dispensing positioning information of the MiniLED to be detected and the abnormality information of the soldering pad where the MiniLED to be detected is located are obtained based on the trained positioning and abnormality detection model. The trained positioning and abnormality detection model can simultaneously realize the precise positioning of MiniLED die-bonding dispensing and the precise detection of soldering pad abnormalities.

[0039] As an exemplary illustration, the data set includes 1,700 MiniLED image data with MiniLED positioning information labels and MiniLED abnormality information labels.

[0040] In an optional embodiment, the trained positioning and anomaly detection model includes a trained YOLOv8 model.

[0041] In an optional embodiment, before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, the structure of the trained positioning and anomaly detection model is optimized to obtain an optimized positioning and anomaly detection model;

[0042] The steps of optimizing the structure of the trained positioning and anomaly detection model include:

[0043] Replace the standard convolution in the backbone network of the trained YOLOv8 model with depthwise separable convolution.

[0044] In an optional embodiment, before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, the structure of the trained positioning and anomaly detection model is secondary optimized to obtain an optimized positioning and anomaly detection model;

[0045] The steps of performing secondary optimization on the structure of the trained positioning and anomaly detection model include:

[0046] The standard convolution in the backbone network of the trained YOLOv8 model is replaced with depthwise separable convolution, and a convolution block attention module is added after the cross-stage partial stacking module in the head of the trained YOLOv8 model.

[0047] In an optional embodiment, the optimized positioning and anomaly detection model includes a feature acquisition module, a feature fusion module and a positioning and detection module;

[0048] The feature acquisition module is used to extract features from MiniLED image data. The first layer of the feature acquisition module is a Conv layer, the second, fourth, sixth, and eighth layers are DWConv layers, the third, fifth, seventh, and ninth layers are C2F layers, and the tenth layer is an SPPF layer. The layers of the feature acquisition module are connected in sequence, and the fifth, seventh, and tenth layers are connected to the feature fusion module. The fifth, seventh, and tenth layers output extracted features F1, F2, and F3, respectively.

[0049] The feature fusion module is used to fuse and enhance the extracted features output by the feature acquisition module into enhanced features, and the 1st and 5th layers of the feature fusion module are Upsample layers, the 2nd, 6th, 10th and 14th layers are Concat layers, the 3rd, 7th, 11th and 15th layers are C2F layers, the 4th, 8th, 12th and 16th layers are CBAM layers, and the 9th and 13th layers are Conv layers; the layers of the feature fusion module are connected in sequence, and its 4th layer is connected to its 10th layer, its 1st and 14th layers are connected to the 10th layer of the feature acquisition module, its 2nd layer is connected to the 7th layer of the feature acquisition module, and its 6th layer is connected to the 5th layer of the feature acquisition module; its 8th, 12th and 16th layers are connected to the positioning and detection module, and the 8th, 12th and 16th layers output enhanced features F7, F9 and F respectively. 11 ;

[0050] The positioning and detection module converts the enhanced features output by the feature fusion module into positioning information of the MiniLED and abnormal information of the pad where the MiniLED is located based on the decoupling head.

[0051] In an optional embodiment, the step of the feature fusion module fusing and enhancing the extracted features output by the feature acquisition module into enhanced features includes:

[0052] The feature fusion module makes the extracted feature F3 pass through the 1st and 2nd layers, and then fuse it with the extracted feature F2 to obtain the enhanced feature F4. The enhanced feature F4 passes through the 3rd and 4th layers to obtain the enhanced feature F5. The enhanced feature F5 passes through the 5th and 6th layers in sequence, and then fuses it with the extracted feature F1 to obtain the enhanced feature F6. The enhanced feature F6 passes through the 7th and 8th layers in sequence, and then fuses it with the enhanced feature F7. The enhanced feature F7 passes through the 9th and 10th layers in sequence, and then fuses it with the enhanced feature F5 to obtain the enhanced feature F8. The enhanced feature F8 passes through the 11th and 12th layers in sequence, and then fuses it with the extracted feature F3 to obtain the enhanced feature F9. 10 , enhanced feature F 10 After the 15th and 16th layers, the enhanced feature F is obtained 11 .

[0053] As an exemplary illustration, the Conv represents convolution; the DWConv represents depthwise separable convolution; the C2F layer represents the C2F module in YOLOv8; and the SPPF layer represents the SPPF module in YOLOv8.

[0054] As an exemplary illustration, Upsample stands for upsampling; Concat stands for Concatenate, which means connection; CBAM stands for Convolutional Block Attention, which means convolutional attention mechanism.

[0055] In an optional embodiment, the MiniLED die-bonding dispensing positioning information includes: the coordinates of the upper left corner of the MiniLED positioning frame, and the coordinates of the lower right corner of the MiniLED positioning frame;

[0056] The abnormal information of the pad where the MiniLED is located includes abnormal result information and normal result information;

[0057] In the model inference phase, the positioning and anomaly detection model calculates anomaly information of the pad where the MiniLED is located based on the die bonding and glue dispensing positioning information of the MiniLED, and the steps include:

[0058] Calculating the angle deviation and / or line width deviation of the MiniLED using the die bonding and dispensing positioning information of the MiniLED;

[0059] If the angle deviation of the MiniLED is greater than the angle deviation threshold, or its line width deviation is greater than the line width deviation threshold, the pad where the MiniLED is located is judged to be abnormal and the abnormal result information is output; otherwise, the pad where the MiniLED is located is judged to be normal and the normal result information is output.

[0060] In an optional embodiment, a calculation expression for calculating the angular deviation of the MiniLED using the die-bonding dispensing positioning information of the MiniLED includes:

[0061]

[0062] Wherein, θ represents the angular deviation of the MiniLED; x0 and y0 represent the abscissa and ordinate of the upper left corner of the MiniLED positioning frame, respectively; x1 and y1 represent the abscissa and ordinate of the lower right corner of the MiniLED positioning frame, respectively; θ0 represents the diagonal angle of the MiniLED;

[0063] The calculation expression for calculating the row width deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes:

[0064] d=|y0-y1|

[0065] Where y0 represents the upper left corner ordinate of the MiniLED positioning frame, and y1 represents the lower right corner ordinate of the MiniLED positioning frame;

[0066] The abnormal result information includes an abnormal mark;

[0067] The normal result information includes the upper left corner coordinates, center coordinates and upper right corner coordinates of the positioning frame of the MiniLED, and the angular deviation of the MiniLED;

[0068] The calculation expression for the coordinates of the center point of the MiniLED positioning frame includes:

[0069] x2=x0+dx

[0070] y2=y0+dy

[0071]

[0072] Where x2 and y2 represent the horizontal and vertical coordinates of the center point of the MiniLED positioning frame, respectively; x0 and y0 represent the horizontal and vertical coordinates of the upper left corner of the MiniLED positioning frame, respectively; x1 and y1 represent the horizontal and vertical coordinates of the lower right corner of the MiniLED positioning frame, respectively.

[0073] As an exemplary illustration, several MiniLED image data with MiniLED positioning information labels and MiniLED abnormality information labels are collected. When forming a data set, the data are collected under several backgrounds and several lighting conditions.

[0074] As an example, when collecting data sets, clear images of MiniLEDs at multiple angles and under different lighting conditions are collected to improve the labeling accuracy of the data sets; when annotating the labels in the data sets, a layered labeling strategy is adopted, first annotating the position of the MiniLED, and then annotating the pad status based on its position information to improve the accuracy of pad status detection.

[0075] Example 2

[0076] This embodiment is based on the MiniLED die-bonding dispensing positioning and pad abnormality detection method proposed in Example 1, and proposes the following implementation examples:

[0077] S1: First, collect MiniLED image data. The images cover a variety of backgrounds, lighting conditions, and pad states. The collected data should be as diverse as possible to ensure the generalization ability of the model.

[0078] S2: Label the collected data. This process involves manually marking the specific locations of the MiniLEDs and their solder pads in each image. This labeling should be as precise as possible to ensure high training data quality. The results include labels for the MiniLED locations and the solder pad status. A dataset containing MiniLED location information and labels for abnormal solder pad images is generated. The labeled dataset is divided into training, validation, and test sets in a 7:1.5:1.5 ratio. This dataset is used to train, validate, and test the model to ensure it can accurately locate the MiniLEDs.

[0079] S3: Based on the dataset annotated in step S2, preliminary training is performed based on the YOLOv8 model. During this process, data augmentation techniques (such as random cropping, color transformation, etc.) are used to expand the scale and diversity of the dataset and reduce the risk of overfitting. During the training process, the optimizer Adam is used to continuously adjust the weights of the model to minimize the loss function. The hyperparameters during training (such as learning rate, batch size, number of iterations, etc.) are also set reasonably to ensure model convergence. At the same time, the performance of the model is evaluated using the validation set. By calculating indicators such as the model's precision and recall on the validation set, its adaptability to the MiniLED positioning and pad status detection tasks is analyzed. The evaluation results of this stage provide a reference for the subsequent optimization of the model structure.

[0080] S4: Based on the results of the preliminary training and validation set evaluation of S3, analyze the performance of the model in terms of speed, accuracy, etc., and identify the parts that need improvement. The standard convolution in the backbone network of the YOLOv8 model is replaced with a depthwise separable convolution, and the CBAM attention mechanism is introduced after the C2F module of the detection head of the model. The optimized model is the positioning and anomaly detection model proposed in Example 1; the structure includes a feature acquisition module, a feature fusion module, and a positioning and detection module. The feature acquisition module has a total of 10 layers, the first layer is a Conv layer, the second, fourth, sixth, and eighth layers are DWConv layers, the third, fifth, seventh, and ninth layers are C2F layers, and the tenth layer is an SPPF layer. Features are extracted from the fifth, seventh, and tenth layers as inputs to the feature fusion layer, and the outputs of the fifth, seventh, and tenth layers are extracted features F1, extracted features F2, and extracted features F3, respectively. The processing process of the feature fusion module is as follows: after upsampling, the extracted feature F3 is fused with the extracted feature F2 to obtain the enhanced feature F4; the enhanced feature F4 passes through the C2F1 layer and the CBAM attention mechanism to obtain the enhanced feature F5; the enhanced feature F5 is upsampled in turn and fused with the extracted feature F1 to obtain the enhanced feature F6; the enhanced feature F6 passes through the C2F2 layer and the CBAM attention mechanism to obtain the enhanced feature F7; the enhanced feature F7 passes through the convolution layer in turn and is fused with the enhanced feature F5 to obtain the enhanced feature F8; the enhanced feature F8 passes through the C2F3 layer and the CBAM attention mechanism to obtain the enhanced feature F9; the enhanced feature F9 passes through the convolution layer in turn and is fused with the extracted feature F3 to obtain the enhanced feature F 10 , enhanced feature F 10 After the C2F4 layer and the CBAM attention mechanism, the enhanced feature F is obtained 11 Finally, the enhanced features F7, F9, and F 11 As the output of the feature fusion layer; the detection layer includes three identical decoupling heads, which enhance the feature F7, the enhanced feature F9, and the enhanced feature F 11 The data is input into the three decoupling heads respectively and the model is retrained based on the optimized YOLOv8 model structure. The data enhancement technology is also used in the training process. The number of training rounds is set to 200, the batch size is 32, and the initial learning rate is set to 1×10 -3 The original training and validation sets were used for training and evaluation. After training, the optimized model was evaluated again using the validation set and compared with the performance of the initial model. By analyzing and comparing the results, the effectiveness of the optimized model structure was verified, including detection accuracy, speed, and adaptability to MiniLED positioning and pad status detection tasks.

[0081] S5: The trained and optimized YOLOv8 model is used for inference testing. In the inference stage, the model is applied to the MiniLED image data in the test set to obtain the positioning frame of the pads. During the inference process, the model will generate the coordinates of the positioning frame according to the features in the input image. Each positioning frame is composed of the coordinates of the top-left corner and the bottom-right corner, accurately indicating the position of the MiniLED and its pads. These positioning frames will serve as the basis for subsequent analysis and calculation, helping to evaluate the actual application effect of the model.

[0082] S6: According to the positioning frame obtained in step S5, first extract the center position of each pad and its distance from the image boundary. By calculating the coordinates of the top-left corner and the bottom-right corner of the positioning frame, the center position of the pad can be accurately determined, and the distance from the left and upper boundaries of the image can be measured. According to the threshold value set by the user in advance, determine whether the pad is abnormal, verify the positioning accuracy and pad state detection ability of the model in actual application. Through statistical analysis of the detection results of the inference test set, evaluate key indicators such as the accuracy of position detection, the accuracy of pad state, and the inference speed of the model. Combined with these analysis results, the adaptability and stability of the model can be further verified, providing reliable basis for the application in actual production.

[0083] As an example, the data augmentation strategies introduced in the training process in step S3, such as random cropping, color transformation, CutMix, etc., can effectively improve the generalization ability of the model. Through the diversification of training data, the model can better cope with various changes that may occur in the actual production environment, such as lighting conditions and background complexity, thereby improving the accuracy of MiniLED positioning and pad state detection.

[0084] As an example, Figure 2 The training evaluation parameter diagram proposed for this embodiment is shown in the following figure: Figure 2 The evaluation results of step S3 are shown in the following figure:

[0085] As an example, the Conv layer and DWConv layer described in step S4 are composed of a convolution layer, a BN layer (BatchNorm), and a SiLU activation function. The convolution layer is used to extract local features from the input feature map. The convolution layer slides over the input through multiple filters (or convolution kernels) to generate multiple output feature maps, each corresponding to a different filter. The BN layer is applied after the convolution operation to perform batch normalization, standardizing each channel in the output feature map. This step helps to reduce the differences between different channel features, stabilizes the training process, and speeds up convergence. The SiLU activation function, also known as the Swish activation function, introduces non-linear characteristics to the normalized feature map through non-linear transformation, thereby enhancing the expression ability of the model. It retains the advantages of ReLU while avoiding the "dead zone" problem of ReLU activation.

[0086] As an example, Figure 3 This is a schematic diagram of the overall framework of the positioning and anomaly detection model proposed in this embodiment; Figure 3 The overall framework of the positioning and anomaly detection model is shown.

[0087] As an example, Figure 4 This is a schematic diagram of the structure of the C2F layer proposed in this embodiment; Figure 4 The structure of the C2F layer is shown; as an example, Figure 4 As shown in Figure 4, the C2F module described in step S4 consists of four branches. The first branch consists of a CBS1 module (including a convolutional layer, a batch normalization layer, and a SiLU activation function) to extract basic features. The second branch consists of a CBS1 module and a separation module, which is used to enhance feature expression capabilities. The third branch consists of a CBS1 module, the same separation module (Split), and a bottleneck module (Bottleneck). The bottleneck module further refines information by compressing and enhancing features. The fourth branch consists of a CBS1 module, the same separation module, and two repeated bottleneck modules to further enhance features. The output feature maps of the four branches are finally fused (Concat) together and integrated through the CBS2 module to generate the final output feature map.

[0088] As an example, Figure 5 This is a schematic diagram of the SPPF module proposed in this embodiment, as shown in FIG. Figure 5 As shown in Figure 4, the SPPF module described in step S4 consists of the following parts: First, the input feature map passes through the CBS module to initially reduce the number of channels in the feature map and reduce the amount of computation; then, the feature map passes through multiple maximum pooling layers (MaxPool) to perform pooling operations of different scales. These pooling layers use pooling kernels of different sizes (for example, 5x5, 3x3, 2x2) to generate multiple feature maps of different scales. Subsequently, these pooled feature maps are spliced ​​with the initial input feature map in the channel dimension to fuse multi-scale information; finally, the spliced ​​feature map is processed by the CBS module to integrate information from different scales and generate the final output feature map.

[0089] As an example, step S4 replaces the standard convolution in the YOLOv8 model backbone network with depthwise separable convolution (DW convolution), decomposing the traditional convolution operation into two steps: depthwise convolution and pointwise convolution, significantly reducing the amount of computation and the number of parameters, thereby improving the model's inference speed. Due to the low computational complexity of DW convolution, memory consumption is also reduced accordingly. This is especially important for deployment on memory-constrained embedded systems or edge devices, and can effectively reduce hardware costs. By replacing the convolution operation, the size and complexity of the model are effectively controlled, making it more lightweight, thereby improving the model's practical application flexibility.

[0090] As an example, the CBAM attention mechanism is introduced after the C2F module in the detection head of the YOLOv8 model described in step S4. By weighting channel and spatial features, it automatically focuses on key feature areas, improving the model's ability to capture MiniLED features. This feature-focused capability enhances the model's detection accuracy in complex backgrounds and pad states. CBAM can effectively combine feature information at different scales, improving the model's ability to detect small targets. In practical applications, this multi-scale feature fusion can better cope with target deformation and occlusion, improving the overall detection reliability. By dynamically adjusting feature weights, CBAM enhances the model's robustness to noise and background interference, reducing the possibility of false detection and missed detection.

[0091] As an example, a convolutional block attention module (CBAM) is added after the cross-stage partial stacking module (C2F) in the model head to enhance the model's ability to extract and fuse multi-scale features, thereby improving the accuracy of positioning and pad status detection.

[0092] As an example, using GPU-accelerated optimized YOLOv8 models in step S5 significantly improves the model's inference speed, meeting the real-time inspection requirements of the production line. This acceleration not only ensures production line continuity but also reduces the risk of potential defects going undetected due to delays.

[0093] As an exemplary illustration, the method of the present application generates a MiniLED-containing image data set by collecting and labeling, and sets a threshold value allowing line width deviation and angle change; based on the labeled data set, the optimized YOLOv8 model is trained to enable it to recognize and locate MiniLEDs. After training, the model can output the positioning frame of MiniLEDs by inputting the image to be detected, and through the coordinate information of the positioning frame, the center position of the MiniLED, the distance to the upper boundary and the left boundary of the positioning frame are calculated, and according to the threshold value set by the user, the abnormality of the solder pad is detected; the trained model is verified and optimized to ensure that its performance in detection accuracy and speed meets the production requirements; based on the detection results, the center position of each MiniLED, the distance to the upper boundary and the left boundary of the positioning frame are analyzed to ensure accurate positioning of the MiniLED and its solder pad state.

[0094] As an exemplary illustration, the method of the present application generates a MiniLED-containing image data set by collecting and labeling, and sets a threshold value allowing line width deviation and angle change; based on the labeled data set, the optimized YOLOv8 model is trained to enable it to recognize and locate MiniLEDs. After training, the model can output the positioning frame of MiniLEDs by inputting the image to be detected, and through the coordinate information of the positioning frame, the center position of the MiniLED, the distance to the upper boundary and the left boundary of the positioning frame are calculated, and according to the threshold value set by the user, the abnormality of the solder pad is detected; the trained model is verified and optimized to ensure that its performance in detection accuracy and speed meets the production requirements; based on the detection results, the center position of each MiniLED, the distance to the upper boundary and the left boundary of the positioning frame are analyzed to ensure accurate positioning of the MiniLED and its solder pad state.

[0095] As an example, by introducing depthwise separable convolution and the CBAM attention mechanism, the optimized YOLOv8 model can more accurately identify the center position and pad status of MiniLEDs when processing MiniLED inspection tasks. Depthwise separable convolution effectively reduces computational complexity, while CBAM enhances the model's focus on key features, significantly improving detection accuracy. The optimization of the model structure reduces the amount of computation, enabling YOLOv8 to perform inference quickly with lower computing resources. The application of depthwise separable convolution enables the model to perform well in real-time inspection tasks, making it particularly suitable for the rapid response requirements of production lines. After the introduction of the CBAM attention mechanism, the model's robustness to complex backgrounds and noise is significantly improved. This means that in actual production environments, even in the face of background changes and different lighting conditions, the model can still stably locate MiniLEDs and detect pad status.

[0096] Example 3

[0097] This embodiment proposes a MiniLED die-bonding glue dispensing positioning and pad anomaly detection system, which is used to implement the MiniLED die-bonding glue dispensing positioning and pad anomaly detection method proposed in Example 1.

[0098] Figure 6 This is the overall framework diagram of the MiniLED die bonding, dispensing, positioning, and pad anomaly detection system of this embodiment.

[0099] The MiniLED die bonding, dispensing, positioning, and pad anomaly detection system includes:

[0100] The data acquisition module is used to collect a number of MiniLED image data with MiniLED positioning information labels and MiniLED abnormality information labels to form a data set;

[0101] A model building module is used to build a positioning and anomaly detection model for calculating the MiniLED die bonding and dispensing positioning information and anomaly information of the MiniLED pads;

[0102] A model training module, configured to train the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model;

[0103] The positioning and detection module is used to obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

[0104] This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the MiniLED die-bonding dispensing positioning and pad abnormality detection method described in Example 1.

[0105] It can be understood that the MiniLED die bonding, dispensing, positioning, and pad anomaly detection system and computer equipment of this embodiment improve the method of Example 1. The optional options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0106] The same or similar reference numerals correspond to the same or similar components;

[0107] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting the present embodiment.

[0108] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for MiniLED die bonding and positioning and pad abnormality detection, characterized in that: The MiniLED is soldered on the pad using a die-bonding dispensing technique, and the method includes the following steps: Collect several MiniLED image data with MiniLED positioning information labels and MiniLED abnormal information labels to form a data set; Build a positioning and anomaly detection model to calculate the MiniLED die bonding and dispensing positioning information and anomaly information of the MiniLED pads. Using the data set to train the positioning and anomaly detection model to obtain a trained positioning and anomaly detection model; Acquire image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into a trained positioning and anomaly detection model, and obtain die bonding and dispensing positioning information of the MiniLED to be detected and anomaly information of the pad where the MiniLED to be detected is located; The MiniLED die bonding and dispensing positioning information includes: the coordinates of the upper left corner of the MiniLED positioning frame, and the coordinates of the lower right corner of the MiniLED positioning frame; The abnormal information of the pad where the MiniLED is located includes abnormal result information and normal result information; In the model inference phase, the positioning and anomaly detection model calculates anomaly information of the pad where the MiniLED is located based on the die bonding and glue dispensing positioning information of the MiniLED, and the steps include: Calculating the angle deviation and / or line width deviation of the MiniLED using the die bonding and dispensing positioning information of the MiniLED; If the angle deviation of the MiniLED is greater than the angle deviation threshold, or its line width deviation is greater than the line width deviation threshold, the pad where the MiniLED is located is judged to be abnormal and the abnormal result information is output; otherwise, the pad where the MiniLED is located is judged to be normal and the normal result information is output.

2. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 1, characterized in that: The trained positioning and anomaly detection model includes a trained YOLOv8 model.

3. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 2, characterized in that: Before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, optimizing the structure of the trained positioning and anomaly detection model to obtain an optimized positioning and anomaly detection model; The steps of optimizing the structure of the trained positioning and anomaly detection model include: Replace the standard convolution in the backbone network of the trained YOLOv8 model with depthwise separable convolution.

4. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 2, characterized in that: Before inputting the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, performing secondary optimization on the structure of the trained positioning and anomaly detection model to obtain an optimized positioning and anomaly detection model; The steps of performing secondary optimization on the structure of the trained positioning and anomaly detection model include: The standard convolution in the backbone network of the trained YOLOv8 model is replaced with depthwise separable convolution, and a convolution block attention module is added after the cross-stage partial stacking module in the head of the trained YOLOv8 model.

5. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 4, characterized in that: The optimized positioning and anomaly detection model includes a feature acquisition module, a feature fusion module and a positioning and detection module; The feature acquisition module is used to extract features from MiniLED image data. The first layer of the feature acquisition module is the Conv layer, the second, fourth, sixth and eighth layers are DWConv layers, the third, fifth, seventh and ninth layers are C2F layers, and the tenth layer is the SPPF layer. The layers of the feature acquisition module are connected in sequence, and the fifth, seventh and tenth layers are connected to the feature fusion module. The fifth, seventh and tenth layers output the extracted features respectively. 、 and ; The feature fusion module is used to fuse and enhance the extracted features output by the feature acquisition module into enhanced features, and the 1st and 5th layers of the feature fusion module are Upsample layers, the 2nd, 6th, 10th and 14th layers are Concat layers, the 3rd, 7th, 11th and 15th layers are C2F layers, the 4th, 8th, 12th and 16th layers are CBAM layers, and the 9th and 13th layers are Conv layers; the layers of the feature fusion module are connected in sequence, and its 4th layer is connected to its 10th layer, its 1st and 14th layers are connected to the 10th layer of the feature acquisition module, its 2nd layer is connected to the 7th layer of the feature acquisition module, and its 6th layer is connected to the 5th layer of the feature acquisition module; its 8th, 12th and 16th layers are connected to the positioning and detection module, and the 8th, 12th and 16th layers output enhanced features respectively. 、 and ; The positioning and detection module converts the enhanced features output by the feature fusion module into positioning information of the MiniLED and abnormal information of the pad where the MiniLED is located based on the decoupling head.

6. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 5, characterized in that: The step of the feature fusion module fusing and enhancing the extracted features output by the feature acquisition module into enhanced features includes: The feature fusion module extracts features After the first and second layers, the feature extraction Fusion to obtain enhanced features , enhanced features Enhanced features are obtained after the 3rd and 4th layers , enhanced features After passing through the 5th and 6th layers, the feature extraction Fusion to obtain enhanced features , enhanced features Enhanced features are obtained after the 7th and 8th layers , enhanced features After the 9th and 10th layers, the enhanced features Fusion to obtain enhanced features , enhanced features Enhanced features are obtained after the 11th and 12th layers , enhanced features After passing through the 13th and 14th layers, the feature extraction Fusion to obtain enhanced features , enhanced features Enhanced features are obtained after the 15th and 16th layers .

7. The MiniLED die bonding glue positioning and pad abnormality detection method according to claim 1, characterized in that: The calculation expression for calculating the angular deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes: Where, represents the angular deviation of the MiniLED; and Respectively represent the horizontal and vertical coordinates of the upper left corner of the MiniLED positioning frame; and Respectively represent the horizontal and vertical coordinates of the lower right corner of the MiniLED positioning frame; Indicates the diagonal angle of MiniLED; The calculation expression for calculating the row width deviation of the MiniLED using the die bonding dispensing positioning information of the MiniLED includes: Where, Respectively represent the upper left corner vertical coordinate of the MiniLED positioning frame, Indicates the vertical coordinate of the lower right corner of the MiniLED positioning frame; The abnormal result information includes an abnormal mark; The normal result information includes the upper left corner coordinates, center coordinates and upper right corner coordinates of the positioning frame of the MiniLED, and the angular deviation of the MiniLED; The calculation expression for the coordinates of the center point of the MiniLED positioning frame includes: Where, and Respectively represent the horizontal and vertical coordinates of the center point of the MiniLED positioning frame; and Respectively represent the horizontal and vertical coordinates of the upper left corner of the MiniLED positioning frame; and They respectively represent the horizontal and vertical coordinates of the lower right corner of the MiniLED positioning frame.

8. A MiniLED die-bonding glue positioning and pad anomaly detection system, used to implement the MiniLED die-bonding glue positioning and pad anomaly detection method according to any one of claims 1 to 7, characterized in that: include: The data acquisition module is used to collect a number of MiniLED image data with MiniLED positioning information labels and MiniLED abnormality information labels to form a data set; A model building module is used to build a positioning and anomaly detection model for calculating the MiniLED die bonding and dispensing positioning information and anomaly information of the MiniLED pads; A model training module, configured to train the positioning and anomaly detection model using the data set to obtain a trained positioning and anomaly detection model; The positioning and detection module is used to obtain image data of the MiniLED to be detected, input the image data of the MiniLED to be detected into the trained positioning and anomaly detection model, and obtain the die bonding and glue positioning information of the MiniLED to be detected and the anomaly information of the pad where the MiniLED to be detected is located.

9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, wherein: When the computer-readable instructions are executed by the processor, the processor performs the steps of the MiniLED die-bonding dispensing positioning and pad abnormality detection method according to any one of claims 1 to 7.

Images