A method and system for detecting surface defects in Micro LEDs

Abstract

This application discloses a method and system for detecting surface defects in Micro LEDs, relating to the field of image detection, which can improve the accuracy, speed, and robustness of defect detection. The method includes: constructing a sample dataset containing sample image data, specifically images of Micro LEDs; constructing a model architecture for an LED-YOLO model, training the model architecture using the sample image data from the sample dataset to obtain a trained defect detection model, wherein the model architecture includes a feature extraction network, a feature fusion network, and a detection head, the feature extraction network including multiple LDFM modules and ECATt modules, and the detection head associated with a preset regression loss function DIoU_W; acquiring image data to be detected, inputting the image data to be detected into the defect detection model for defect detection, and obtaining the defect detection result corresponding to the image data to be detected.

Description

Technical Field

[0001] This application relates to the field of image detection, and in particular to a method and system for detecting surface defects in Micro LEDs. Background Technology

[0002] Micro LED (micro-light-emitting diode) is an advanced display technology that utilizes extremely small LEDs as display elements. Each pixel is composed of an individual micro-LED, capable of emitting light independently. With the rapid development of Micro LED technology, it has gradually become the mainstream of next-generation display technology due to its advantages such as high brightness, wide color gamut, and low power consumption. In actual production, Micro LEDs may suffer from various types of defects due to impurities in the semiconductor material, chip misalignment, or weak bonding between the electrodes and the Micro LED. These defects not only affect the display effect but also reduce product yield, thereby increasing production costs. However, the production process of Micro LEDs is complex, and due to their micrometer-scale size, defect detection is a major challenge in manufacturing. Therefore, how to effectively detect defects in production and improve production efficiency and quality has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, this application provides a method and system for detecting surface defects of Micro LEDs, the main purpose of which is to solve the problems of complex production processes of Micro LEDs and low efficiency and accuracy of defect detection due to their micron-sized dimensions.

[0004] According to a first aspect of this application, a method for detecting surface defects in Micro LEDs is provided, the method comprising:

[0005] Construct a sample dataset, which stores sample image data, specifically image data of micro light-emitting diodes (Micro LEDs).

[0006] The model architecture of the LED-YOLO model is constructed by training the model architecture with sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model. The model architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W.

[0007] Acquire the image data to be detected, input the image data to be detected into the defect detection model for defect detection, and obtain the defect detection result corresponding to the image data to be detected.

[0008] Optionally, the model architecture for constructing the LED-YOLO model is trained using the sample dataset to obtain a trained defect detection model, including:

[0009] The feature extraction network further includes multiple Conv modules, multiple C3k2 modules, and a Spatial Pyramid Pooling Feature Fusion (SPPF) module. The feature fusion network includes multiple Upsample modules, multiple Concat modules, multiple Conv modules, multiple C3k2 modules, and an LDFM module.

[0010] The model architecture parameters of the model architecture are initialized, and sample image data is extracted from the sample dataset and input into the LED-YOLO model;

[0011] The LED-YOLO model's feature extraction network is used to extract features from the sample image data, resulting in a first feature map, a second feature map, and a third feature map. The LED-YOLO model's feature fusion network is then used to fuse the first, second, and third feature maps, resulting in a low-level feature map, a mid-level feature map, and a deep feature map. The LED-YOLO model's detection head is then used to predict bounding boxes and their categories from the low-level, mid-level, and deep feature maps. The predicted bounding box categories are optimized using a classification loss function, and the predicted bounding boxes are optimized using a preset regression loss function DIoU_W.

[0012] Optionally, the step of using the feature extraction network of the LED-YOLO model to extract features from the sample image data to obtain a first feature map, a second feature map, and a third feature map includes:

[0013] The first Conv module of the feature extraction network is used to perform a convolution operation on the sample image data, the second Conv module is used to perform a convolution operation on the feature map output by the first Conv module, the first C3k2 module is used to extract features from the feature map output by the second Conv module, the third Conv module is used to perform a convolution operation on the feature map output by the first C3k2 module, and the second C3k2 module is used to extract features from the feature map output by the third Conv module to obtain the first feature map;

[0014] The first feature map is convolved using the fourth Conv module, and the second feature map is obtained by extracting features from the feature map output by the fourth Conv module using the first LDFM module.

[0015] The second feature map is obtained by performing a convolution operation on the fifth Conv module, extracting features from the feature map output by the fifth Conv module using the second LDFM module, extracting context information from the feature map output by the second LDFM module using the SPPF module, and enhancing features from the feature map output by the SPPF module using the ECATt module.

[0016] Optionally, the feature fusion network using the LED-YOLO model performs feature fusion on the first feature map, the second feature map, and the third feature map to obtain low-level feature maps, mid-level feature maps, and deep-level feature maps, including:

[0017] The first Upsample module of the feature fusion network upsamples the third feature map according to the nearest neighbor interpolation method to obtain a first feature map to be spliced. The first Concat module splices the second feature map and the first feature map to be spliced ​​along the channel dimension. The third C3k2 module performs feature fusion on the feature map output by the first Concat module to obtain a second feature map to be spliced.

[0018] The second Upsample module is used to upsample the feature map to be spliced ​​according to the nearest interpolation method to obtain the third feature map to be spliced. The second Concat module is used to splice the first feature map and the second feature map to be spliced ​​along the channel dimension. The fourth C3k2 module is used to perform feature fusion on the feature map output by the second Concat module to obtain the low-level feature map.

[0019] The sixth Conv module is used to perform convolution operation on the low-level feature map to obtain the fourth feature map to be concatenated. The third Concat module is used to concatenate the second feature map to be concatenated and the fourth feature map to be concatenated along the channel dimension. The fifth C3k2 module is used to perform feature fusion on the feature map output by the third Concat module to obtain the middle-level feature map.

[0020] The seventh Conv module is used to perform a convolution operation on the middle layer feature map to obtain the fifth feature map to be concatenated. The fourth Concat module is used to concatenate the third feature map and the fifth feature map to be concatenated along the channel dimension. The third LDFM module is used to perform feature fusion on the feature map output by the fourth Concat module to obtain the deep feature map.

[0021] Optionally, the preset regression loss function DIoU_W is defined as follows:

[0022]

[0023]

[0024] DIoU_W=R WIoU *DIoU

[0025] DIoU_W Loss =1-DIoU_W

[0026] in:

[0027]

[0028] IoU (Intersection over Union) is the crossover ratio between predicted bounding boxes and ground truth bounding boxes. A is the predicted bounding box, B is the ground truth bounding box, and b and b''''''''''''''''''''"""","' ... gt ρ represents the center point of the predicted box and the ground truth box, respectively; ρ represents the Euclidean distance between the two center points; c represents the diagonal length of the smallest rectangular region that can simultaneously contain both the predicted box and the ground truth box; β is defined as the outlier; δ and α are two hyperparameters. Indicates separating L from the computation graph IoU It does not participate in gradient calculation; For L IoU The mean.

[0029] Optionally, acquiring image data to be detected, inputting the image data to be detected into the defect detection model for defect detection, and obtaining the defect detection result corresponding to the image data to be detected includes: inputting the image data to be detected into the defect detection model, so that the defect detection model performs bounding box prediction and bounding box category prediction on the image data to be detected based on a feature extraction network, a feature fusion network, and a detection head, using the predicted bounding boxes and corresponding bounding box categories as the defect detection result, and sending the defect detection result to a designated terminal for visualization.

[0030] Optionally, constructing the sample dataset includes:

[0031] Based on the image acquisition system, sample image data of micro light-emitting diodes (Micro LEDs) are acquired. The image acquisition system includes a camera, a microscope lens, a light source, a host computer, and a motion control unit.

[0032] The sample image data is augmented using a preset data augmentation strategy to obtain the sample dataset. The data augmentation strategy includes, but is not limited to, horizontal flipping, vertical flipping, random brightness adjustment, random contrast adjustment, and random addition of Gaussian noise.

[0033] According to a second aspect of this application, a Micro LED surface defect detection system is provided, the system comprising:

[0034] The construction module is used to construct a sample dataset, which stores sample image data, specifically image data of micro light-emitting diodes (Micro LEDs).

[0035] The training module is used to construct the model architecture of the LED-YOLO model. The model architecture is trained using sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model. The model architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function (DIoU_W).

[0036] The detection module is used to acquire image data to be detected, input the image data to be detected into the defect detection model for defect detection, and obtain the defect detection result corresponding to the image data to be detected.

[0037] According to a third aspect of this application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0038] According to a fourth aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0039] Using the above technical solution, this application provides a method and system for detecting surface defects in Micro LEDs. First, a sample dataset is constructed, storing sample image data, which are image data of micro-light-emitting diodes (Micro LEDs). Further, an LED-YOLO model architecture is constructed, and the model architecture is trained using the sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model, and its architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W. Finally, the image data to be detected is acquired and input into the defect detection model for defect detection, yielding the corresponding defect detection result. This application's embodiment, by employing the LED-YOLO model and combining dynamic convolution, depthwise convolution, and channel mixing techniques in the LDFM module, significantly improves feature extraction capabilities under lightweight conditions. The LDFM module effectively captures complex textures and subtle features in images, ensuring high detection accuracy even with limited computational resources. The ECAtt module, by combining channel and spatial attention mechanisms, further enhances the model's ability to detect minute defects in Micro LED images, enabling precise localization and identification of potential defect areas. This combination of technologies not only improves the accuracy of Micro LED defect detection but also effectively reduces computational complexity and increases detection efficiency, thus providing a more efficient and accurate solution for product quality control.

[0040] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0042] Figure 1 This paper illustrates a schematic flowchart of a Micro LED surface defect detection method according to an embodiment of this application.

[0043] Figure 2This illustration shows a schematic diagram of the data augmentation results of a Micro LED surface defect detection method provided in an embodiment of this application;

[0044] Figure 3 A schematic flowchart of a Micro LED surface defect detection method provided in an embodiment of this application is shown;

[0045] Figure 4 This paper illustrates a Conv module architecture diagram of a Micro LED surface defect detection method provided in an embodiment of this application.

[0046] Figure 5 This paper shows a C3k2 module architecture diagram of a Micro LED surface defect detection method provided in an embodiment of this application;

[0047] Figure 6 This paper illustrates an LDFM module architecture diagram of a Micro LED surface defect detection method provided in an embodiment of this application.

[0048] Figure 7 This illustration shows a dynamic convolution diagram of a Micro LED surface defect detection method provided in an embodiment of this application;

[0049] Figure 8 This paper illustrates the SPPF module architecture diagram of a Micro LED surface defect detection method provided in an embodiment of this application.

[0050] Figure 9 This paper illustrates the ECAT module architecture diagram of a Micro LED surface defect detection method provided in an embodiment of this application.

[0051] Figure 10 This diagram shows the head section architecture of a Micro LED surface defect detection method provided in an embodiment of this application;

[0052] Figure 11 This diagram illustrates the detection results of a Micro LED surface defect detection method provided in an embodiment of this application.

[0053] Figure 12 A schematic diagram of the structure of a Micro LED surface defect detection system provided in an embodiment of this application is shown. Detailed Implementation

[0054] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0055] Micro LED (micro-light-emitting diode) is an advanced display technology that miniaturizes and matrixes LEDs to form a high-density array of tiny LEDs. Each pixel can be individually controlled and driven to emit light, thus achieving display functionality. The size of a single light-emitting unit in a Micro LED is generally less than 50 micrometers. It is currently widely used in various fields, such as screen displays in smartphones, televisions, watches, and other devices, as well as in automotive lighting, dashboards, and central control screens.

[0056] This application employs an improved YOLOv11s model, namely the LED-YOLO model, for Micro LED surface defect detection. Micro LED surface defect detection is a crucial step in ensuring the quality and performance of Micro LED products. Through surface defect detection, defective Micro LED chips can be identified and eliminated in a timely manner, reducing waste in subsequent manufacturing processes and improving the overall product yield.

[0057] This application provides a method for detecting surface defects in Micro LEDs, such as... Figure 1 As shown, the method includes:

[0058] S10. Construct a sample dataset, which stores sample image data, specifically image data of micro LEDs.

[0059] In this embodiment, a Micro LED defect acquisition system is first used to acquire image data of the micro-light-emitting diode (MicroLED), which is then used as sample image data. The Micro LED defect acquisition system consists of a camera, a microscope lens, a light source, a host computer, and a motion control unit. Further, a preset data augmentation strategy is used to augment the sample image data. This data augmentation strategy includes, but is not limited to, horizontal flipping, vertical flipping, random brightness adjustment, random contrast adjustment, and random addition of Gaussian noise. The augmented image is shown below. Figure 2 As shown, data augmentation of sample image data is performed to simulate various industrial disturbances present in actual production processes, such as dust particles, lighting changes, and vibrations. Finally, a sample dataset is constructed using the data-augmented sample image data.

[0060] S20. Construct the model architecture of the LED-YOLO model, and train the model architecture using sample image data from the sample dataset to obtain a trained defect detection model.

[0061] The defect detection model in this application is based on the LED-YOLO model, which is an improved YOLOv11s model. L, E, and D represent the "LDFM module," "ECAtt module," and "DIoU_W loss function," respectively. The model architecture of the LED-YOLO model is as follows: Figure 3 As shown, the network includes a feature extraction network (Backbone), a feature fusion network (Neck), and a detection head (Head). The feature extraction network comprises multiple Conv modules, multiple C3k2 modules, multiple Lightweight Dynamic Fusion Module (LDFM), a Spatial Pyramid Pooling Feature Fusion Module (SPPF), and an Enhanced Coordinated Attention Module (ECAtt). The feature fusion network comprises multiple Upsample modules, multiple Concat modules, multiple Conv modules, multiple C3k2 modules, and an LDFM module. The detection head is associated with a predefined regression loss function DIoU_W.

[0062] The architecture of the Conv module is as follows: Figure 4 As shown, it includes a two-dimensional convolutional layer Conv2d, a two-dimensional batch normalization layer BatchNorm2d, and a SiLU activation function. First, features are extracted through the convolutional layer, then the training process is stabilized through the batch normalization layer, and finally, nonlinearity is introduced through the SiLU activation function.

[0063] The architecture of the C3k2 module is as follows: Figure 5 As shown, when the c3k parameter is set to True, the C3k2 module includes a convolutional layer (Conv), a splitting layer (Split), n C3k modules, and a concatenation layer (Concat). When the c3k parameter is set to False, the C3k2 module includes a convolutional layer (Conv), a splitting layer (Split), n bottleneck layers (Bottleneck), and a concatenation layer (Concat). It should be noted that when the shortcut connection layer (Shortcut) is True, the bottleneck layer (Bottleneck) includes both convolutional and additive layers; when the shortcut connection layer (Shortcut) is False, the bottleneck layer (Bottleneck) only includes convolutional layers. The C3k2 module is designed to improve network performance while controlling computational costs. By using bottleneck layers and shortcut connections, the C3k2 module can reduce the number of parameters and computational cost while maintaining network depth. This modular design also makes the network architecture more flexible and can be adjusted according to different application scenarios.

[0064] The LDFM module is a component of a neural network architecture that preserves feature representation by combining dynamic convolution and depthwise convolution. Furthermore, channel blending operations are added to further enhance feature interaction. The specific structure depends on the context; the module architecture is as follows: Figure 6 As shown, when the c3k parameter is set to True, the LDFM module includes a convolutional layer (Conv), a splitting layer (Split), n LD-C3k modules, and a concatenation layer (Concat). When the c3k parameter is set to False, the LDFM module includes a convolutional layer (Conv), a splitting layer (Split), n bottleneck layers (LD-Bottleneck), and a concatenation layer (Concat). The bottleneck layers are used to reduce the dimensionality of the feature maps, thereby reducing computational cost. The LD-C3k module includes a convolutional layer (Conv), n bottleneck layers (LD-Bottleneck), and a concatenation layer (Concat). It should be noted that when the shortcut connection layer (Shortcut) is True, the bottleneck layer (LD-Bottleneck) includes a dynamic convolutional layer (DynamicConv), a depthwise separable convolutional layer (DWConv), and a concatenation layer (Concat). When the shortcut connection layer (Shortcut) is False, the bottleneck layer (LD-Bottleneck) includes a dynamic convolutional layer (DynamicConv) and a depthwise separable convolutional layer (DWConv). The dynamic convolutional layer extends traditional convolution operations, allowing the network to adaptively adjust its parameters based on the input. It enhances the model's feature representation capability by dynamically selecting and combining different convolutional kernels for each input sample; this selection is achieved through an attention mechanism. Given input features Input(x), it passes through an attention network containing pooling layers, flattening operations, fully connected layers, and activation functions. The weights of each convolutional kernel are dynamically calculated, and the output Output(y) is the weighted sum of all convolutional operations. Specifically... Figure 7As shown, the parameters of the convolutional kernels can be dynamically adjusted through an attention mechanism, enabling the network to adaptively adjust the convolutional operations based on different features of the input data. AvgPool is an average pooling layer used to reduce the dimensionality of the input data and extract important features. Flatten flattens the pooled data, converting it into a one-dimensional array for input into the linear layer. Linear is a linear layer used to learn the linear relationship between input features and convolutional kernel parameters. Sigmoid is an activation function used to convert the output of the linear layer into probability values ​​between 0 and 1, which will be used as weights for the convolutional kernels. Wi, W2, ..., Wk are dynamically generated convolutional kernel weights, each associated with a convolutional layer (Convi, Conv2, ..., Convk). Each layer (Convi, Conv2, ..., Convk) uses different weights (Wi, W2, ..., Wk) for convolutional operations. Concat concatenates the outputs of all dynamically generated convolutional layers to form a new feature map. Conv2d applies standard two-dimensional convolutional operations to the concatenated feature map to further extract features. Finally, the weighted sum of all convolutional operations is output through Output(y). Depthwise convolution, by performing convolution on each input channel independently, reduces computational complexity and the number of parameters, making the model more lightweight. The LDFM module uses depthwise convolution while discarding the original standard 3x3 convolutional layers.

[0065] The SPPF (Spatial Pyramid Pooling-Fast) module is a pooling structure for convolutional neural networks that allows the network to capture features at different scales, thereby improving robustness to changes in target size. Its structure is as follows: Figure 8 As shown, the network includes convolutional layers (Conv), multiple max-pooling layers (MaxPool2d), each using a different pooling window size or stride. It also includes a concatenation layer (Concat) to combine the outputs of the different max-pooling layers. This concatenation operation allows the network to capture features at different scales, thus forming multi-scale feature representations.

[0066] The Efficient Channel Attention (ECAtt) module is a mechanism for improving the performance of convolutional neural networks. It combines channel attention and spatial attention, and adds residual connections to avoid the vanishing gradient problem, thereby enhancing the model's feature representation capabilities. Its structure is as follows: Figure 9As shown, it includes a Conv2d convolutional layer, a Sigmoid activation function, an Add layer, a Multiplication layer, and a Concat concatenation layer. The ECAtt module is designed to improve model performance by introducing a channel attention mechanism without significantly increasing computational cost.

[0067] In this step, the model architecture parameters are first initialized, and sample image data is extracted from the sample dataset and input into the LED-YOLO model. Further, the feature extraction network of the LED-YOLO model extracts features from the sample image data to obtain a first feature map, a second feature map, and a third feature map. The feature fusion network of the LED-YOLO model then fuses these three feature maps to obtain low-level, mid-level, and deep feature maps. Finally, the detection head of the LED-YOLO model performs bounding box prediction and bounding box category prediction on the low-level, mid-level, and deep feature maps. A classification loss function is used to optimize the predicted bounding box category, and a preset regression loss function DIoU_W is used to further optimize the predicted bounding boxes. The improved LED-YOLO model architecture is shown in Table 1 below.

[0068] Table 1. Improved YOLOv11s Model Architecture

[0069]

[0070]

[0071] In this context, `Layer` represents the current layer's index, and `From` represents the source of the current layer's input, which can be the index of the previous layer (e.g., -1 represents the previous layer) or a specific layer index. `Module` represents the module or operation type used by the current layer. `Arguments` represents the parameters of the current layer, defining the specific configuration of the module. Specifically, `Conv`'s `Arguments` are the number of input channels, the number of output channels, the kernel size, and the stride. `C3k2`'s `Arguments` are the number of input channels, the number of output channels, the c3k parameter, and the scaling factor. `LDFM`'s `Arguments` are the number of input channels, the number of output channels, and whether to use depthwise convolution. `SPPF`'s `Arguments` are the number of input channels, the number of output channels, and the pooling kernel size. `ECAtt`'s `Arguments` are the number of input channels, the number of output channels, and whether to use residual connections. `Upsample`'s `Arguments` are the input, the upsampling ratio, and the interpolation method. `Concat`'s `Arguments` are the concatenation dimensions. `Detect:`'s `Arguments` are the number of classes and the anchor box size.

[0072] Specifically, such as Figure 3 As shown, a feature extraction network is used ( Figure 3 The first Conv module of the Backbone section (in the middle) Figure 3 The Conv module in layer 0 performs convolution operations on the sample image data, and the second Conv module is used. Figure 3 The first layer (Conv) performs a convolution operation on the feature map output by the first Conv module, using the first C3k2 module ( Figure 3 The second layer (C3k2) extracts features from the feature map output by the second Conv module, and the third Conv module ( Figure 3 The third layer (Conv) performs a convolution operation on the feature map output by the first C3k2 module, and then uses the second C3k2 module ( Figure 3 The C3k2 module of the fourth layer extracts features from the feature map output by the third Conv module to obtain the first feature map. Further, the fourth Conv module (…) is used… Figure 3 The fifth layer (Conv) performs a convolution operation on the first feature map and uses the first LDFM module ( Figure 3 The LDFM in layer 6 is used to extract features from the feature map output by the fourth Conv module to obtain the second feature map. Next, the fifth Conv module ( Figure 3 The 7th layer (Conv) performs a convolution operation on the second feature map, and the second LDFM module is used. Figure 3 The LDFM in the 8th layer extracts features from the feature map output by the fifth Conv module, the SPPF module extracts context information from the feature map output by the second LDFM module, and the ECATt module enhances the features from the feature map output by the SPPF module to obtain the third feature map.

[0073] Employing feature fusion network ( Figure 3 The first Upsample module in the middle Neck section ( Figure 3 The 11th layer of the upsample module upsamples the third feature map using the nearest neighbor interpolation method to obtain the first feature map to be concatenated, and then uses the first Concat module ( Figure 3 The 12th layer of the concat module stitches the second feature map and the first feature map to be stitched along the channel dimension, and the third C3k2 module is used. Figure 3 The C3k2 module (layer 13) performs feature fusion on the feature map output by the first Concat module to obtain the second feature map to be concatenated. Next, the second Upsample module (layer 13) is used... Figure 3The 14th layer of the upsample module upsamples the feature map to be concatenated using the nearest interpolation method to obtain the third feature map to be concatenated, and then uses the second Concat module ( Figure 3 The 15th layer of the concat module stitches the first feature map and the second feature map to be stitched along the channel dimension, and uses the fourth C3k2 module ( Figure 3 The C3k2 module (layer 16) is used to fuse features from the output of the second Concat module to obtain a low-level feature map. Further, the sixth Conv module (layer 16) is used... Figure 3 The 17th layer of the Conv module performs a convolution operation on the lower-level feature maps to obtain the fourth feature map to be concatenated, and then uses the third Concat module ( Figure 3 The 18th layer of the concat module stitches the second and fourth feature maps to be stitched along the channel dimension, and the fifth C3k2 module is used. Figure 3 The C3k2 module (layer 19) is used to fuse features from the feature map output by the third Concat module to obtain the mid-layer feature map. Finally, the seventh Conv module (layer 19) is used to fuse features from the C3k2 module (layer 19) to obtain the mid-layer feature map. Figure 3 The 20th layer (Conv) performs a convolution operation on the middle layer feature map to obtain the fifth feature map to be concatenated, and then uses the fourth Concat module ( Figure 3 The concat layer of layer 21 stitches the third feature map and the fifth feature map to be stitched along the channel dimension, and uses the third LDFM module ( Figure 3 The LDFM in the 22nd layer performs feature fusion on the feature map output by the fourth Concat module to obtain the deep feature map.

[0074] A detection head network (Head part) is used for prediction to generate prediction results. A loss function is calculated based on the prediction results and the actual results, and the model parameters are optimized based on the loss function. The structure of the detection head network is as follows: Figure 10As shown, the model includes two branches: one for predicting the bounding box location and the other for predicting the class. In actual operation, low-level, mid-level, and deep feature maps are fed into the branch for bounding box prediction and the branch for class prediction. After a series of convolutional operations, the bounding box parameters, such as center point coordinates, width, and height, are predicted through the bounding box prediction branch. The class prediction branch predicts the class probability for each bounding box. The bounding box prediction is then optimized by minimizing the difference between the predicted and ground truth boxes using the DIoU_W loss function, which combines the convergence speed of DIoU with the accuracy of WIoU. DIoU considers the distance between the center points of the predicted and ground truth boxes, while WIoUv3's dynamic non-monotonic focusing mechanism ensures the model always enjoys a dynamic optimal gradient during convergence. The accuracy of class prediction is further optimized using the CLSLoss loss function, which can be a cross-entropy loss function. The loss value calculated by the loss function is used in the backpropagation algorithm to update the model's weights using gradient descent. During training, the model's parameters are continuously optimized through multiple iterations to minimize the difference between the predicted and actual results. Hyperparameters, such as the learning rate and batch size, also need to be tuned to achieve better model performance. Finally, the model's performance is evaluated on a validation set, using metrics such as mean squared accuracy (mAP) to measure the predictive accuracy of the detection head.

[0075] It should be noted that the DIoU_W loss function is defined as shown in Equations 1 to 4 below:

[0076] Formula 1:

[0077] Formula 2:

[0078] Formula 3: DIoU_W = R WIoU *DIoU

[0079] Formula 4: DIoU_W Loss =1-DIoU_W

[0080] in,

[0081] IoU (Intersection over Union) is the crossover ratio between predicted bounding boxes and ground truth bounding boxes. A is the predicted bounding box, B is the ground truth bounding box, and b and b''''''''''''''''''''"""","' ... gt ρ represents the center point of the predicted box and the ground truth box, respectively; ρ represents the Euclidean distance between the two center points; c represents the diagonal length of the smallest rectangular region that can simultaneously contain both the predicted box and the ground truth box; β is defined as the outlier; δ and α are two hyperparameters. Indicates separating L from the computation graph IoUIt does not participate in gradient calculation; For L IoU The mean.

[0082] S30. Obtain the image data to be detected, input the image data to be detected into the defect detection model to perform defect detection, and obtain the defect detection result corresponding to the image data to be detected.

[0083] The LED-YOLO model constructed in this application is an improvement upon the original YOLOv11s model. Sample image data from the sample dataset is used to train the model architecture, resulting in a usable defect detection model. The defect detection model mainly consists of three parts: a feature extraction network (Backbone), a feature fusion network (Neck), and a detection head. The image data to be detected is input into the defect detection model, which then performs bounding box prediction and bounding box category prediction based on the feature extraction network, feature fusion network, and detection head. Finally, the predicted bounding boxes and their corresponding categories are used as the defect detection results, which are then sent to a designated terminal for visualization. In actual operation, the Backbone performs multi-scale feature extraction by downsampling the input image by 8x, 16x, and 32x. After feature extraction, an ECATt attention module is inserted to focus on the defect region, serving as the three-layer input to the Neck. The Neck improves the multi-scale feature capture capability through information fusion at different levels of the feature pyramid. LDFM modules are employed in the Backbone and Neck to improve the model's feature extraction capabilities. Finally, three feature maps of different sizes and channel numbers obtained from the Neck are sent to the Head for prediction computation. The prediction results using the improved YOLOv11s model, specifically the LED-YOLO model, are shown below. Figure 11 As shown.

[0084] The Micro LED surface defect detection method provided in this embodiment first constructs a sample dataset containing sample image data, specifically images of micro-light-emitting diodes (Micro LEDs). Further, an LED-YOLO model architecture is constructed, and the model is trained using the sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model, and its architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W. Finally, the image data to be detected is acquired and input into the defect detection model for defect detection, yielding the corresponding defect detection result. This embodiment, by employing the LED-YOLO model and combining dynamic convolution, depthwise convolution, and channel mixing techniques in the LDFM module, significantly improves feature extraction capabilities under lightweight conditions. The LDFM module effectively captures complex textures and subtle features in images, ensuring the model maintains high detection accuracy even with limited computational resources. The ECAtt module further enhances the model's ability to focus on minute defects in Micro LED images by combining channel and spatial attention mechanisms, enabling the model to accurately locate and identify potential defect areas. This combination of technologies not only improves the accuracy of Micro LED defect detection but also effectively reduces computational complexity and increases detection efficiency, thus providing a more efficient and accurate solution for product quality control.

[0085] Furthermore, as Figure 1 To specifically implement the method, this application provides a Micro LED surface defect detection system, such as... Figure 12 As shown, the system includes: a construction module 1201, a training module 1202, and a detection module 1203.

[0086] Construction module 1201 is used to construct a sample dataset, which stores sample image data, and the sample image data is image data of micro light-emitting diodes (Micro LEDs).

[0087] Training module 1202 is used to construct the model architecture of the LED-YOLO model. The model architecture is trained using sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model. The model architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W.

[0088] The detection module 1203 is used to acquire image data to be detected, input the image data to be detected into the defect detection model for defect detection, and obtain the defect detection result corresponding to the image data to be detected.

[0089] The system provided in this application first constructs a sample dataset containing sample image data, specifically images of micro LEDs. Further, it constructs an LED-YOLO model architecture, training the model using the sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model, and its architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W. Finally, it acquires the image data to be detected and inputs it into the defect detection model to obtain the corresponding defect detection result. This application significantly improves feature extraction capabilities under lightweight conditions by employing the LED-YOLO model and combining dynamic convolution, depthwise convolution, and channel mixing techniques in the LDFM module. The LDFM module effectively captures complex textures and subtle features in images, ensuring the model maintains high detection accuracy even with limited computational resources. The ECAtt module further enhances the model's ability to focus on minute defects in Micro LED images by combining channel and spatial attention mechanisms, enabling the model to accurately locate and identify potential defect areas. This combination of technologies not only improves the accuracy of Micro LED defect detection but also effectively reduces computational complexity and increases detection efficiency, thus providing a more efficient and accurate solution for product quality control.

[0090] It should be noted that other corresponding descriptions of the functional units involved in the Micro LED surface defect detection system provided in this application embodiment can be found in the following references. Figures 1 to 11 The corresponding description in [the document] will not be repeated here.

[0091] Based on the above, Figures 1 to 11 Accordingly, this embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the Micro LED surface defect detection method.

[0092] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), including several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0093] Based on the above, Figures 1 to 11 The method shown, and Figure 12 To achieve the above objectives, the illustrated embodiment of the Micro LED surface defect detection system further provides a device in an exemplary embodiment. This device includes a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device. The various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory to perform the Micro LED surface defect detection method described in the above embodiment.

[0094] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0095] Those skilled in the art will understand that the physical device structure of the Micro LED surface defect detection method provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0096] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs to be identified. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the technical solution of this application, a sample dataset is first constructed, which stores sample image data, specifically image data of micro light-emitting diodes (Micro LEDs). Further, the model architecture of the LED-YOLO model is constructed, and the model architecture is trained using the sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model, and its architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W. Finally, the image data to be detected is obtained and input into the defect detection model for defect detection, yielding the defect detection result corresponding to the image data to be detected. Compared with existing technologies, the embodiments of this application significantly improve feature extraction capabilities under lightweight conditions by employing the LED-YOLO model and combining dynamic convolution, depthwise convolution, and channel blending techniques in the LDFM module. The LDFM module can effectively capture complex textures and subtle features in images, ensuring that the model maintains high detection accuracy even with limited computing resources. The ECAtt module, by combining channel and spatial attention mechanisms, further enhances the model's ability to focus on minute defects in Micro LED images, enabling the model to accurately locate and identify potential defect areas. This combination of technologies not only improves the accuracy of Micro LED defect detection but also effectively reduces computational complexity and increases detection efficiency, thus providing a more efficient and accurate solution for product quality control.

[0098] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0099] Those skilled in the art will understand that the modules in the apparatus of the implementation scenario can be distributed within the apparatus of the implementation scenario as described, or they can be located in one or more apparatuses different from this implementation scenario, with corresponding changes. The modules of the above-described implementation scenario can be combined into one module, or they can be further divided into multiple sub-modules.

[0100] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenario.

[0101] The above disclosures are only a few specific implementation scenarios of this application. However, this application is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for detecting surface defects in Micro LEDs, characterized in that, include: Construct a sample dataset, which stores sample image data, specifically image data of micro light-emitting diodes (Micro LEDs). The model architecture of the LED-YOLO model is constructed by training the model architecture with sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model. The model architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function DIoU_W. Acquire the image data to be detected, input the image data to be detected into the defect detection model for defect detection, and obtain the defect detection result corresponding to the image data to be detected.

2. The method according to claim 1, characterized in that, The model architecture for constructing the LED-YOLO model is trained using the sample dataset to obtain a trained defect detection model, including: The feature extraction network further includes multiple Conv modules, multiple C3k2 modules, and a Spatial Pyramid Pooling Feature Fusion (SPPF) module. The feature fusion network includes multiple Upsample modules, multiple Concat modules, multiple Conv modules, multiple C3k2 modules, and an LDFM module. The model architecture parameters of the model architecture are initialized, and sample image data is extracted from the sample dataset and input into the LED-YOLO model; The LED-YOLO model's feature extraction network is used to extract features from the sample image data, resulting in a first feature map, a second feature map, and a third feature map. The LED-YOLO model's feature fusion network is then used to fuse the first, second, and third feature maps, resulting in a low-level feature map, a mid-level feature map, and a deep feature map. The LED-YOLO model's detection head is then used to predict bounding boxes and their categories from the low-level, mid-level, and deep feature maps. The predicted bounding box categories are optimized using a classification loss function, and the predicted bounding boxes are optimized using a preset regression loss function DIoU_W.

3. The method according to claim 2, characterized in that, The feature extraction network using the LED-YOLO model extracts features from the sample image data to obtain a first feature map, a second feature map, and a third feature map, including: The first Conv module of the feature extraction network is used to perform a convolution operation on the sample image data, the second Conv module is used to perform a convolution operation on the feature map output by the first Conv module, the first C3k2 module is used to extract features from the feature map output by the second Conv module, the third Conv module is used to perform a convolution operation on the feature map output by the first C3k2 module, and the second C3k2 module is used to extract features from the feature map output by the third Conv module to obtain the first feature map; The first feature map is convolved using the fourth Conv module, and the second feature map is obtained by extracting features from the feature map output by the fourth Conv module using the first LDFM module. The second feature map is obtained by performing a convolution operation on the fifth Conv module, extracting features from the feature map output by the fifth Conv module using the second LDFM module, extracting context information from the feature map output by the second LDFM module using the SPPF module, and enhancing features from the feature map output by the SPPF module using the ECATt module.

4. The method according to claim 3, characterized in that, The feature fusion network using the LED-YOLO model performs feature fusion on the first feature map, the second feature map, and the third feature map to obtain low-level feature maps, mid-level feature maps, and deep-level feature maps, including: The first Upsample module of the feature fusion network upsamples the third feature map according to the nearest neighbor interpolation method to obtain a first feature map to be spliced. The first Concat module splices the second feature map and the first feature map to be spliced ​​along the channel dimension. The third C3k2 module performs feature fusion on the feature map output by the first Concat module to obtain a second feature map to be spliced. The second Upsample module is used to upsample the feature map to be spliced ​​according to the nearest interpolation method to obtain the third feature map to be spliced. The second Concat module is used to splice the first feature map and the second feature map to be spliced ​​along the channel dimension. The fourth C3k2 module is used to perform feature fusion on the feature map output by the second Concat module to obtain the low-level feature map. The sixth Conv module is used to perform convolution operation on the low-level feature map to obtain the fourth feature map to be concatenated. The third Concat module is used to concatenate the second feature map to be concatenated and the fourth feature map to be concatenated along the channel dimension. The fifth C3k2 module is used to perform feature fusion on the feature map output by the third Concat module to obtain the middle-level feature map. The seventh Conv module is used to perform a convolution operation on the middle layer feature map to obtain the fifth feature map to be concatenated. The fourth Concat module is used to concatenate the third feature map and the fifth feature map to be concatenated along the channel dimension. The third LDFM module is used to perform feature fusion on the feature map output by the fourth Concat module to obtain the deep feature map.

5. The method according to claim 2, characterized in that, The preset regression loss function DIoU_W is defined as follows: DIoU_W=R WIoU *DIoU DIoU_W Loss =1-DIoU_W in: IoU (Intersection over Union) is the crossover ratio between predicted bounding boxes and ground truth bounding boxes. A is the predicted bounding box, B is the ground truth bounding box, and b and b''''''''''''''''''''"""","' ... gt ρ represents the center point of the predicted box and the ground truth box, respectively; ρ represents the Euclidean distance between the two center points; c represents the diagonal length of the smallest rectangular region that can simultaneously contain both the predicted box and the ground truth box; β is defined as the outlier; δ and α are two hyperparameters. Indicates separating L from the computation graph IoU It does not participate in gradient calculation; For L IoU The mean.

6. The method according to claim 1, characterized in that, The process involves acquiring image data to be detected, inputting the image data to be detected into the defect detection model for defect detection, and obtaining the defect detection result corresponding to the image data to be detected. This includes: inputting the image data to be detected into the defect detection model, so that the defect detection model performs bounding box prediction and bounding box category prediction on the image data to be detected based on a feature extraction network, a feature fusion network, and a detection head; using the predicted bounding boxes and corresponding bounding box categories as the defect detection result; and sending the defect detection result to a designated terminal for visualization.

7. The method according to claim 1, characterized in that, The construction of the sample dataset includes: Based on the image acquisition system, sample image data of micro light-emitting diodes (Micro LEDs) are acquired. The image acquisition system includes a camera, a microscope lens, a light source, a host computer, and a motion control unit. The sample image data is augmented using a preset data augmentation strategy to obtain the sample dataset. The data augmentation strategy includes, but is not limited to, horizontal flipping, vertical flipping, random brightness adjustment, random contrast adjustment, and random addition of Gaussian noise.

8. A Micro LED surface defect detection system, characterized in that, include: The construction module is used to construct a sample dataset, which stores sample image data, specifically image data of micro light-emitting diodes (Micro LEDs). The training module is used to construct the model architecture of the LED-YOLO model. The model architecture is trained using sample image data from the sample dataset to obtain a trained defect detection model. The LED-YOLO model is an improved YOLOv11s model. The model architecture includes a feature extraction network, a feature fusion network, and a detection head. The feature extraction network includes multiple lightweight dynamic fusion modules (LDFM) and an enhanced coordinated attention module (ECAtt). The detection head is associated with a preset regression loss function (DIoU_W). The detection module is used to acquire image data to be detected, input the image data to be detected into the defect detection model for defect detection, and obtain the defect detection result corresponding to the image data to be detected.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images