Mini LED lamp bead defect detection method and system

Abstract

The invention discloses a Mini LED lamp bead defect detection method and system, and relates to the technical field of machine vision, and the method comprises the steps: carrying out defect label labeling on Mini LED lamp bead sample image data; generating a defect detection data set based on the sample image data marked by the defect label, and preprocessing the defect detection data set; performing data enhancement processing on the preprocessed defect detection data set; training a YOLOv8 OBB model on the basis of the defect detection data set subjected to data enhancement processing; the trained YOLOv8 OBB model is exported as a preset format, and a defect detection model is obtained; inputting target image data of a to-be-detected Mini LED lamp bead into the defect detection model for defect detection to obtain a defect category and a rotating rectangular frame parameter; and performing defect post-processing based on the defect category and the rotating rectangular frame parameter. According to the method, the fusion effect of the multi-scale features is improved, and the defect detection precision can be improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method and system for detecting defects in MiniLED beads. Background Technology

[0002] Currently, the main methods for MiniLED defect detection include manual inspection and traditional computer vision inspection. Due to the extremely small size of MiniLEDs, manual inspection typically requires the use of a microscope to examine each chip on the module. This is not only inefficient but also prone to subjective bias, leading to inconsistent defect assessments from different individuals. Traditional computer vision inspection methods mostly employ template matching or fixed threshold methods. However, these methods require repeated parameter adjustments or the creation of new templates for different product models, making the entire defect detection process cumbersome and lacking in generalization. They are also susceptible to factors such as lighting, image noise, and sample differences, making it difficult to guarantee stability in industrial production environments.

[0003] In recent years, deep learning has made significant progress in the field of object detection. Among them, the YOLO series of algorithms, due to their fast detection speed and end-to-end training and inference capabilities, have been increasingly applied to industrial inspection tasks. However, existing detection methods based on horizontal bounding boxes of the YOLO model still have shortcomings in MiniLED defect detection. On the one hand, small or elongated defects are easily submerged during the model's downsampling process, leading to missed or false detections. On the other hand, defects in actual production are often distributed at arbitrary angles, making it difficult for horizontal bounding box detection methods to accurately fit the true shape of the defects, resulting in insufficient positioning accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and system for detecting defects in MiniLED beads, which can improve the model's ability to extract features of small and slender defects at different scales, improve the fusion effect of multi-scale features, and improve the accuracy of defect detection.

[0005] To address the aforementioned technical problems, this invention provides a method for detecting defects in MiniLED chips, the method comprising: Obtain sample image data of MiniLED beads, and label the sample image data with defect tags to obtain sample image data with defect tags; A defect detection dataset is generated based on the sample image data after defect labeling, and the defect detection dataset is preprocessed to obtain a preprocessed defect detection dataset. Data augmentation is performed on the preprocessed defect detection dataset to obtain the data augmented defect detection dataset. The YOLOv8 OBB model is trained on the defect detection dataset after data augmentation to obtain a trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. Export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; Acquire target image data of the MiniLED lamp bead to be detected, and input the target image data into the defect detection model to perform defect detection, and obtain the defect category and rotation rectangle parameters; Defect post-processing is performed based on the defect category and the rotated rectangle parameters.

[0006] Optionally, the step of labeling the sample image data with defect tags to obtain sample image data with defect tags includes: Identify the types of defects in the sample image data, and determine data annotation rules based on the types of defects; Using X-Anylabeling annotation software, defect labels are applied to the sample image data according to the data annotation rules to obtain defect-labeled sample image data.

[0007] Optionally, preprocessing the defect detection dataset to obtain a preprocessed defect detection dataset includes: Determine the image adjustment size, and perform image scaling on the defect detection dataset based on the image adjustment size to obtain the preprocessed defect detection dataset.

[0008] Optionally, performing data augmentation on the preprocessed defect detection dataset to obtain a data-augmented defect detection dataset includes: The preprocessed defect detection dataset is cropped and stitched together to obtain the stitched defect detection dataset. The image quality of the stitched defect detection dataset is adjusted to obtain the image quality adjusted defect detection dataset. The defect detection dataset after image quality adjustment is subjected to brightness adjustment and defect pixel value generation to obtain a data augmentation-processed defect detection dataset.

[0009] Optionally, the multi-scale feature enhancement module includes a 1*1 convolution branch, a 3*3 convolution branch, a dilated convolution branch, a lightweight pyramid pooling branch, and a coordinate attention module. The lightweight pyramid pooling branch includes a global average pooling layer and a 1*1 convolution layer.

[0010] Optionally, the expression for the rotated frame regression loss function is: , in, The loss function is the rotation box regression function. For geometric loss, These are the weighting coefficients for the geometric loss. For angle constraints, The weighting coefficients for the angle constraint term.

[0011] Optionally, the step of inputting the target image data into a defect detection model for defect detection to obtain the defect category and rotated bounding box parameters includes: Based on the multi-scale feature enhancement module in the defect detection model, several branch features of the target image data are extracted, and the several branch features are spliced ​​and dimensionality reduced to obtain a feature map; The feature map is subjected to global pooling in the height direction based on the coordinate attention module in the multi-scale feature enhancement module to obtain the first directional feature, and the feature map is subjected to global pooling in the width direction to obtain the second directional feature. A horizontal attention map is generated based on the first directional feature and the second directional feature, and a vertical attention map is generated based on the first directional feature and the second directional feature. Enhanced features are generated based on the horizontal and vertical attention maps; The gated fusion module in the defect detection model generates gated features using several branch features, and determines the target features based on the gated features and the enhanced features. The detection output head in the defect detection model uses the target features to output the defect category and the parameters of the rotated rectangle.

[0012] Optionally, the gated fusion module based on the defect detection model generates gated features using several branch features, including: Based on the gated fusion module, global average pooling, dimensionality reduction and activation processing are performed on several branch features to obtain the weight vector of each branch feature; The weight vector is normalized using the Softmax function to obtain adaptive weighting coefficients. Based on the adaptive weighting coefficients, several branch features are fused to obtain gated features.

[0013] Optionally, the post-processing of defects based on the defect category and the rotated rectangle parameters includes: Calculate the number of components, relative offset of pads, relative offset of LEDs, component rotation angle, and component offset based on the defect category and the parameters of the rotating rectangle. The number of components, relative offset of pads, relative offset of LEDs, component rotation angle, and component offset are compared with the corresponding preset thresholds to obtain the comparison results, and abnormal information is determined based on the comparison results.

[0014] In addition, the present invention also provides a MiniLED lamp bead defect detection system, the system comprising: Labeling module: used to acquire sample image data of MiniLED beads, and to label the sample image data with defect labels to obtain sample image data with defect labels; Preprocessing module: used to generate a defect detection dataset based on sample image data after defect labeling, and to preprocess the defect detection dataset to obtain a preprocessed defect detection dataset; Data augmentation module: Used to perform data augmentation on the preprocessed defect detection dataset to obtain the data-augmented defect detection dataset; Model training module: used to train the YOLOv8 OBB model based on the defect detection dataset after data augmentation, to obtain the trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature augmentation module, a gating fusion module, and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. Model export module: Used to export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; Defect detection module: used to acquire target image data of the MiniLED lamp bead to be detected, and input the target image data into the defect detection model to perform defect detection, and obtain the defect category and rotation rectangle parameters; Defect post-processing module: used to perform defect post-processing based on the defect category and the rotation rectangle parameters.

[0015] In this embodiment of the invention, defect labels are annotated on sample image data of MiniLED beads. A defect detection dataset is generated based on the defect-labeled sample image data. The defect detection dataset is preprocessed and then augmented to provide more reliable data support for model training. The YOLOv8 OBB model is trained on the augmented defect detection dataset. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model is a rotated bounding box regression loss function. The multi-scale feature enhancement module improves the model's ability to extract features of small and elongated defects at different scales, and improves the fusion effect of multi-scale features. The rotated bounding box regression loss function improves the stability and accuracy of the model in rotation angle prediction. The trained YOLOv8 OBB model is exported to a preset format for lightweight inference. Target image data is input into the defect detection model for defect detection to obtain defect categories and rotated bounding box parameters. Post-processing based on the defect categories and rotated bounding box parameters yields more comprehensive defect anomaly information. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the MiniLED lamp bead defect detection method in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for detecting defects in MiniLED beads according to another embodiment of the present invention. Figure 3 This is a schematic diagram of the structural composition of the MiniLED lamp bead defect detection system in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the effect of image defect labeling in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the effect of defect post-processing in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating the MiniLED chip defect detection method according to an embodiment of the present invention. The method includes: S11: Obtain sample image data of MiniLED beads, and label the sample image data with defect tags to obtain sample image data after defect tagging; In the specific implementation of this invention, sample image data of miniature light-emitting diode (MiniLED) beads are acquired, the types of defects in the sample image data are identified, and data labeling rules are determined based on the types of defects. The sample image data is labeled with defect labels using X-Anylabeling software based on the data labeling rules, which can ensure the learning effect of the model.

[0020] S12: Generate a defect detection dataset based on the sample image data after defect labeling, and preprocess the defect detection dataset to obtain a preprocessed defect detection dataset; In the specific implementation of this invention, a defect detection dataset is generated based on the sample image data after defect labeling, the image adjustment size is determined, and the defect detection dataset is scaled based on the image adjustment size to obtain a preprocessed defect detection dataset, ensuring that the image conforms to the input size of the model.

[0021] S13: Perform data augmentation on the preprocessed defect detection dataset to obtain the data augmented defect detection dataset; In the specific implementation of this invention, the preprocessed defect detection dataset is cropped and stitched to obtain a stitched defect detection dataset; the stitched defect detection dataset is then subjected to image quality adjustment to obtain an image quality adjusted defect detection dataset; the image quality adjusted defect detection dataset is then subjected to brightness adjustment and defect pixel value generation to obtain a data augmentation defect detection dataset, which can enhance the robustness of the model under different defect shapes and orientations.

[0022] S14: The YOLOv8 OBB model is trained based on the defect detection dataset after data augmentation to obtain a trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. In the specific implementation of this invention, the YOLOv8 Oriented Bounding Box (OBB) model is trained based on the defect detection dataset after data augmentation. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. Through the multi-scale feature fusion enhancement module, the feature extraction capability of small defects and slender defects at different scales can be improved. The loss function of the YOLOv8 OBB model adopts the rotation box regression loss function. In view of the problem of insufficient fitting accuracy of existing rotating targets, the rotation box regression loss function can improve the stability and accuracy of the model in rotation angle prediction.

[0023] S15: Export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; In the specific implementation of this invention, the trained YOLOv8 OBB model is exported to a preset format, which is either the Open Neural Network Exchange (ONNX) format or the TensorRT format, to obtain a defect detection model, realize lightweight inference, and ensure that real-time and efficient defect detection and localization tasks are completed on the industrial production line.

[0024] S16: Acquire the target image data of the MiniLED lamp bead to be detected, and input the target image data into the defect detection model to perform defect detection, and obtain the defect category and rotation rectangle parameters; In the specific implementation of this invention, target image data of the MiniLED bead to be detected is acquired. Based on the multi-scale feature enhancement module in the defect detection model, several branch features of the target image data are extracted, and these branch features are then stitched together and dimensionality reduced to obtain a feature map. Based on the coordinate attention module in the multi-scale feature enhancement module, the feature map undergoes global pooling in the height direction to obtain a first directional feature, and global pooling in the width direction to obtain a second directional feature. A horizontal attention map is generated based on the first and second directional features, and a vertical attention map is generated based on the first and second directional features. Enhanced features are generated based on the horizontal and vertical attention maps. Based on the gating fusion module in the defect detection model, gated features are generated using the several branch features, and target features are determined based on the gated features and enhanced features. Based on the detection output head in the defect detection model, the defect category and rotation rectangle parameters are output using the target features, significantly improving the detection and localization capabilities for small targets, slender targets, and rotating defect targets.

[0025] S17: Perform post-processing of defects based on the defect category and the parameters of the rotated rectangle.

[0026] In the specific implementation of this invention, the number of components, relative offset of pads, relative offset of LEDs, component rotation angle, and component offset are calculated based on the defect category and the parameters of the rotating rectangle. The number of components, relative offset of pads, relative offset of LEDs, component rotation angle, and component offset are compared with the corresponding preset thresholds to obtain the comparison results. Based on the comparison results, abnormal information is determined, which can obtain more comprehensive defect abnormal information.

[0027] In this embodiment of the invention, defect labels are annotated on sample image data of MiniLED beads. A defect detection dataset is generated based on the defect-labeled sample image data. The defect detection dataset is preprocessed and then augmented to provide more reliable data support for model training. The YOLOv8 OBB model is trained on the augmented defect detection dataset. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model is a rotated bounding box regression loss function. The multi-scale feature enhancement module improves the model's ability to extract features of small and elongated defects at different scales, and improves the fusion effect of multi-scale features. The rotated bounding box regression loss function improves the stability and accuracy of the model in rotation angle prediction. The trained YOLOv8 OBB model is exported to a preset format for lightweight inference. Target image data is input into the defect detection model for defect detection to obtain defect categories and rotated bounding box parameters. Post-processing based on the defect categories and rotated bounding box parameters yields more comprehensive defect anomaly information.

[0028] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for detecting defects in MiniLED beads according to another embodiment of the present invention, the method comprising: S201: Obtain sample image data of MiniLED beads, and label the sample image data with defect tags to obtain sample image data after defect tagging; In a specific implementation of the present invention, the step of labeling the sample image data with defect labels to obtain sample image data with defect labels includes: identifying the types of defects in the sample image data and determining data labeling rules based on the types of defects; using X-Anylabeling labeling software to label the sample image data with defect labels using the data labeling rules to obtain sample image data with defect labels.

[0029] Specifically, sample image data of MiniLED beads is obtained. The sample image data includes different image data collected under different lighting conditions. The types of defects in the sample image data are identified. The types of defects in the images can be identified by manual inspection, and data labeling rules are determined based on the types of defects.

[0030] Based on the X-Anylabeling annotation software, defect labels are applied to the sample image data using the aforementioned data annotation rules, resulting in defect-labeled sample image data. X-AnyLabeling is an open-source, industrial-grade image annotation tool that supports tasks such as object detection, image segmentation, and text recognition. It can integrate multiple deep learning models to achieve automatic annotation and is suitable for data preparation in computer vision projects. The defect labeling effect of the images is as follows: Figure 4 As shown, rotating rectangles can be used to label LEDs, pads, and defects on an image, and the output labels are the detected object category and the coordinates of the four corner points of the rotating rectangle.

[0031] S202: Generate a defect detection dataset based on the sample image data after defect labeling, and preprocess the defect detection dataset to obtain a preprocessed defect detection dataset; In a specific implementation of the present invention, the step of preprocessing the defect detection dataset to obtain a preprocessed defect detection dataset includes: determining the image adjustment size, and performing image scaling processing on the defect detection dataset based on the image adjustment size to obtain a preprocessed defect detection dataset.

[0032] Specifically, a defect detection dataset is generated based on the sample image data after defect labeling. The image adjustment size is determined, and the defect detection dataset is scaled based on the image adjustment size, that is, the images in the defect detection dataset are scaled to the image adjustment size, such as adjusting the image size to 256*256, to obtain the preprocessed defect detection dataset.

[0033] S203: Perform data augmentation on the preprocessed defect detection dataset to obtain the data augmented defect detection dataset; In the specific implementation of this invention, the step of performing data augmentation processing on the preprocessed defect detection dataset to obtain a data-augmented defect detection dataset includes: performing image cropping and stitching processing on the preprocessed defect detection dataset to obtain a stitched defect detection dataset; performing image quality adjustment processing on the stitched defect detection dataset to obtain an image quality adjusted defect detection dataset; and performing brightness adjustment processing and defect pixel value generation processing on the image quality adjusted defect detection dataset to obtain a data-augmented defect detection dataset.

[0034] Specifically, the preprocessed defect detection dataset undergoes image cropping and stitching to obtain a stitched defect detection dataset. Image cropping is used to extract target regions, and stitching is used to combine multiple images. The stitched defect detection dataset then undergoes image quality adjustment to obtain a quality-adjusted defect detection dataset. Image quality adjustment aims to improve image clarity, enhance details, and optimize colors.

[0035] The defect detection dataset, after image quality adjustment, undergoes brightness adjustment and defect pixel value generation to obtain a data-augmented defect detection dataset. Brightness adjustment aims to enhance image contrast and detail, improving visual quality; this can be achieved using color space adjustment algorithms. Defect pixel value generation involves extracting defect pixel values ​​from other labels and overlaying them onto normal images. This data augmentation enhances the model's robustness under different defect shapes and orientations.

[0036] S204: The YOLOv8 OBB model is trained based on the defect detection dataset after data augmentation to obtain a trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gating fusion module, and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. In the specific implementation of this invention, the multi-scale feature enhancement module includes a 1*1 convolution branch, a 3*3 convolution branch, a dilated convolution branch, a lightweight pyramid pooling branch, and a coordinate attention module. The lightweight pyramid pooling branch includes a global average pooling layer and a 1*1 convolution layer.

[0037] The expression for the rotated frame regression loss function is: , in, The loss function is the rotation box regression function. For geometric loss, These are the weighting coefficients for the geometric loss. For angle constraints, The weighting coefficients for the angle constraint term.

[0038] Specifically, the YOLOv8 OBB model is trained on a defect detection dataset after data augmentation. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The multi-scale feature enhancement module includes 1×1 convolutional branches, 3×3 convolutional branches, dilated convolutional branches, lightweight pyramid pooling branches, and a coordinate attention module. The lightweight pyramid pooling branch includes a global average pooling layer and a 1×1 convolutional layer. The 1×1 convolutional branch preserves local details, the 3×3 convolutional branch extracts standard receptive field features, and the dilated convolutional branch expands the receptive field to accommodate elongated defects. The lightweight pyramid pooling branch, composed of global average pooling and 1×1 convolutions, supplements semi-global and global contextual information. The coordinate attention module enhances the directional sensitivity and spatial localization capability of features. The gated fusion module is used for feature fusion. The detection output head is used to handle small target defects, thereby improving the recall rate and detection accuracy for extremely small targets.

[0039] To address the insufficient fitting accuracy of existing rotating target models, the rotation loss function is improved. First, the rotation intersection-union ratio (ROI) is calculated based on the overlapping region of the rotated rectangle. A geometric constraint term is introduced, jointly measuring the center distance between the predicted and ground truth boxes and their enclosing regions to obtain the geometric loss. Then, the rotation angle is numerically normalized to a fixed interval, and an angle constraint term is constructed using periodic angle differences, such as 1-cos(Δθ) or a smoothed L1 function to penalize angle deviations. The final expression for the rotated box regression loss function is: , in, The loss function is the rotation box regression function. For geometric loss, These are the weighting coefficients for the geometric loss. For angle constraints, These are the weighting coefficients for the angle constraint term. Combining geometric constraints with explicit angle constraints can significantly improve the stability and fitting accuracy of rotation angle prediction while ensuring the accuracy of the overlapping region.

[0040] S205: Export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; In the specific implementation of this invention, a fixed-size input tensor (e.g., 1×3×256×256) is constructed, and the trained YOLOv8 OBB model performs one forward inference to establish a static computation graph structure. Then, the export interface is called to convert the computation graph structure into a network model in ONNX format, thus obtaining the defect detection model. The ONNX file contains the network topology, convolution parameters, operator information, etc., exported in ONNX format for accelerating inference.

[0041] S206: Obtain the target image data of the MiniLED lamp bead to be detected, extract several branch features of the target image data based on the multi-scale feature enhancement module in the defect detection model, and perform splicing and dimensionality reduction processing on the several branch features to obtain a feature map; In the specific implementation of this invention, target image data of the MiniLED lamp bead to be detected is acquired. Based on the multi-scale feature enhancement module in the defect detection model, several branch features of the target image data are extracted. The branch features include features extracted by 1×1 convolution branch, 3×3 convolution branch, dilated convolution branch and lightweight pyramid pooling branch. The several branch features are then spliced ​​and dimensionality reduced. Each branch feature is spliced ​​in the channel dimension and reduced to the number of input channels by 1×1 convolution to obtain a feature map.

[0042] S207: Based on the coordinate attention module in the multi-scale feature enhancement module, the feature map is subjected to global pooling in the height direction to obtain a first directional feature, and the feature map is subjected to global pooling in the width direction to obtain a second directional feature. In the specific implementation of this invention, the feature map is subjected to global pooling in the height direction based on the coordinate attention module in the multi-scale feature enhancement module, that is, global pooling is performed along the height direction of the feature map to retain the spatial position information in the vertical direction and obtain the first directional feature. The feature map is also subjected to global pooling in the width direction, that is, global pooling is performed along the width direction of the feature map to retain the spatial position information in the horizontal direction and obtain the second directional feature.

[0043] S208: Generate a horizontal attention map based on the first directional feature and the second directional feature, and generate a vertical attention map based on the first directional feature and the second directional feature; generate an enhancement feature based on the horizontal attention map and the vertical attention map. In the specific implementation of this invention, a horizontal attention map is generated based on the first and second directional features, and a vertical attention map is generated based on the first and second directional features. The first and second directional features are concatenated in the spatial dimension, and channel compression and nonlinear activation are performed through 1×1 convolution. The output is then re-divided into two directional branches and restored to the original number of channels through independent 1×1 convolutions, resulting in the horizontal and vertical attention maps. Enhanced features are generated based on the horizontal and vertical attention maps, and the directional attention maps are applied element-wise to the features to obtain the enhanced features. The coordinate attention module can effectively enhance the network's directional sensitivity to slender targets and rotating defects, significantly improve feature representation ability and defect localization accuracy, while maintaining low computational cost and good deployment performance.

[0044] S209: The gated fusion module in the defect detection model generates gated features using several branch features, and determines the target features based on the gated features and enhanced features. The detection output head in the defect detection model outputs the defect category and the rotation rectangle parameters using the target features. In the specific implementation of this invention, the gated fusion module based on the defect detection model generates gated features using several branch features, including: performing global average pooling, dimensionality reduction, and activation processing on several branch features based on the gated fusion module to obtain the weight vector of each branch feature; normalizing the weight vector based on the Softmax function to obtain adaptive weighting coefficients; and fusing several branch features based on the adaptive weighting coefficients to obtain gated features.

[0045] Specifically, based on the gated fusion module, global average pooling, dimensionality reduction, and activation are performed on several branch features. First, the branch features are aggregated and global average pooled. After dimensionality reduction and activation, they are expanded into weight vectors corresponding to the number of branches, thus obtaining the weight vectors of each branch feature. The weight vectors are normalized using the Softmax function to obtain adaptive weighting coefficients. Based on these adaptive weighting coefficients, the several branch features are fused, that is, the branch features and the adaptive weighting coefficients are weighted and fused to obtain the gated features.

[0046] The target features are determined based on the gated and enhanced features. The gated and enhanced features are added together and then further refined using a 3×3 convolution. The final output, the target feature, is formed by adding the residual connections to the input features. The detection output head of the defect detection model uses these target features to output the defect category and the parameters of the rotated bounding box. The defect detection model can simultaneously achieve multi-scale feature extraction, orientation sensitivity enhancement, and adaptive branch selection, thereby significantly improving the model's ability to detect and locate small, slender, and rotated defective targets.

[0047] S210: Perform post-processing of defects based on the defect category and the parameters of the rotated rectangle.

[0048] In a specific implementation of the present invention, the post-processing of defects based on the defect category and the rotating rectangle parameters includes: calculating the number of components, the relative offset of the pads, the relative offset of the LEDs, the component rotation angle, and the component offset based on the defect category and the rotating rectangle parameters; comparing the number of components, the relative offset of the pads, the relative offset of the LEDs, the component rotation angle, and the component offset with the corresponding preset thresholds to obtain the comparison results, and determining the abnormal information based on the comparison results.

[0049] Specifically, based on the defect category and the parameters of the rotating rectangle, the number of components, the relative offset of the pads, the relative offset of the LEDs, the component rotation angle, and the component offset are calculated. These parameters are then compared with corresponding preset thresholds to obtain the comparison results. Based on these results, anomaly information is determined. The post-processing effect of the defect is shown in the image below. Figure 5 As shown, for example, if the relative offset of the pad is greater than the first preset threshold or the relative offset of the LED is greater than the second preset threshold, it indicates that there is an abnormal offset of the pad or LED. If the number of components is less than the third preset threshold, it indicates that there is a missing component. If the rotation angle of the component is greater than the fourth preset threshold, it indicates that there is a rotation abnormality. If the component offset is greater than the fifth preset threshold, it indicates that there is a component offset abnormality. The obtained abnormal results are summarized to obtain abnormal information.

[0050] In this embodiment of the invention, defect labels are annotated on sample image data of MiniLED beads. A defect detection dataset is generated based on the defect-labeled sample image data. The defect detection dataset is preprocessed and then augmented to provide more reliable data support for model training. The YOLOv8 OBB model is trained on the augmented defect detection dataset. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model is a rotated bounding box regression loss function. The multi-scale feature enhancement module improves the model's ability to extract features of small and elongated defects at different scales, and improves the fusion effect of multi-scale features. The rotated bounding box regression loss function improves the stability and accuracy of the model in rotation angle prediction. The trained YOLOv8 OBB model is exported to a preset format for lightweight inference. Target image data is input into the defect detection model for defect detection to obtain defect categories and rotated bounding box parameters. Post-processing based on the defect categories and rotated bounding box parameters yields more comprehensive defect anomaly information.

[0051] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of the MiniLED lamp bead defect detection system according to an embodiment of the present invention. The system includes: Labeling module 31: Used to acquire sample image data of MiniLED beads and to label the sample image data with defect labels to obtain sample image data after defect labeling; Preprocessing module 32: used to generate a defect detection dataset based on the sample image data after defect labeling, and to preprocess the defect detection dataset to obtain a preprocessed defect detection dataset; Data augmentation module 33: used to perform data augmentation on the preprocessed defect detection dataset to obtain the data-augmented defect detection dataset; Model training module 34: used to train the YOLOv8 OBB model based on the defect detection dataset after data augmentation, and obtain the trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature augmentation module, a gating fusion module and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. Model export module 35: Used to export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; Defect detection module 36: used to acquire target image data of the MiniLED lamp bead to be detected, and input the target image data into the defect detection model to perform defect detection, and obtain the defect category and rotation rectangle parameters; Defect post-processing module 37: used to perform defect post-processing based on the defect category and the rotation rectangle parameters.

[0052] In the specific implementation of this invention, the specific implementation methods of the system items can be referred to the implementation methods of the above-mentioned method items, and will not be repeated here.

[0053] In this embodiment of the invention, defect labels are annotated on sample image data of MiniLED beads. A defect detection dataset is generated based on the defect-labeled sample image data. The defect detection dataset is preprocessed and then augmented to provide more reliable data support for model training. The YOLOv8 OBB model is trained on the augmented defect detection dataset. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model is a rotated bounding box regression loss function. The multi-scale feature enhancement module improves the model's ability to extract features of small and elongated defects at different scales, and improves the fusion effect of multi-scale features. The rotated bounding box regression loss function improves the stability and accuracy of the model in rotation angle prediction. The trained YOLOv8 OBB model is exported to a preset format for lightweight inference. Target image data is input into the defect detection model for defect detection to obtain defect categories and rotated bounding box parameters. Post-processing based on the defect categories and rotated bounding box parameters yields more comprehensive defect anomaly information.

[0054] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0055] Furthermore, the above provides a detailed description of the MiniLED lamp bead defect detection method and system provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting defects in MiniLED chips, characterized in that, The method includes: Obtain sample image data of MiniLED beads, and label the sample image data with defect tags to obtain sample image data with defect tags; A defect detection dataset is generated based on the sample image data after defect labeling, and the defect detection dataset is preprocessed to obtain a preprocessed defect detection dataset. Data augmentation is performed on the preprocessed defect detection dataset to obtain the data augmented defect detection dataset. The YOLOv8 OBB model is trained on the defect detection dataset after data augmentation to obtain a trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature enhancement module, a gated fusion module, and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. Export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; Acquire target image data of the MiniLED lamp bead to be detected, and input the target image data into the defect detection model to perform defect detection, and obtain the defect category and rotation rectangle parameters; Defect post-processing is performed based on the defect category and the rotated rectangle parameters.

2. The method for detecting defects in MiniLED chips according to claim 1, characterized in that, The step of labeling the sample image data with defect tags to obtain sample image data with defect tags includes: Identify the types of defects in the sample image data, and determine data annotation rules based on the types of defects; Using X-Anylabeling annotation software, defect labels are applied to the sample image data according to the data annotation rules to obtain defect-labeled sample image data.

3. The method for detecting defects in MiniLED chips according to claim 1, characterized in that, The step of preprocessing the defect detection dataset to obtain a preprocessed defect detection dataset includes: Determine the image adjustment size, and perform image scaling on the defect detection dataset based on the image adjustment size to obtain the preprocessed defect detection dataset.

4. The method for detecting defects in MiniLED beads according to claim 1, characterized in that, The step of performing data augmentation on the preprocessed defect detection dataset to obtain a data-augmented defect detection dataset includes: The preprocessed defect detection dataset is cropped and stitched together to obtain the stitched defect detection dataset. The image quality of the stitched defect detection dataset is adjusted to obtain the image quality adjusted defect detection dataset. The defect detection dataset after image quality adjustment is subjected to brightness adjustment and defect pixel value generation to obtain a data augmentation-processed defect detection dataset.

5. The method for detecting defects in MiniLED chips according to claim 1, characterized in that, The multi-scale feature enhancement module includes a 1*1 convolution branch, a 3*3 convolution branch, a dilated convolution branch, a lightweight pyramid pooling branch, and a coordinate attention module. The lightweight pyramid pooling branch includes a global average pooling layer and a 1*1 convolution layer.

6. The method for detecting defects in MiniLED chips according to claim 1, characterized in that, The expression for the rotated frame regression loss function is: ， in, The loss function is the rotation box regression function. For geometric loss, These are the weighting coefficients for the geometric loss. For angle constraints, The weighting coefficients for the angle constraint term.

7. The method for detecting defects in MiniLED beads according to claim 1, characterized in that, The step of inputting the target image data into a defect detection model for defect detection to obtain defect categories and rotated bounding box parameters includes: Based on the multi-scale feature enhancement module in the defect detection model, several branch features of the target image data are extracted, and the several branch features are spliced ​​and dimensionality reduced to obtain a feature map; The feature map is subjected to global pooling in the height direction based on the coordinate attention module in the multi-scale feature enhancement module to obtain the first directional feature, and the feature map is subjected to global pooling in the width direction to obtain the second directional feature. A horizontal attention map is generated based on the first directional feature and the second directional feature, and a vertical attention map is generated based on the first directional feature and the second directional feature. Enhanced features are generated based on the horizontal and vertical attention maps; The gated fusion module in the defect detection model generates gated features using several branch features, and determines the target features based on the gated features and the enhanced features. The detection output head in the defect detection model uses the target features to output the defect category and the parameters of the rotated rectangle.

8. The method for detecting defects in MiniLED beads according to claim 7, characterized in that, The gated fusion module based on the defect detection model generates gated features using several branch features, including: Based on the gated fusion module, global average pooling, dimensionality reduction and activation processing are performed on several branch features to obtain the weight vector of each branch feature; The weight vector is normalized using the Softmax function to obtain adaptive weighting coefficients. Based on the adaptive weighting coefficients, several branch features are fused to obtain gated features.

9. The method for detecting defects in MiniLED beads according to claim 1, characterized in that, The post-processing of defects based on the defect category and the rotated rectangle parameters includes: Calculate the number of components, relative offset of pads, relative offset of LEDs, component rotation angle, and component offset based on the defect category and the parameters of the rotating rectangle. The number of components, relative offset of pads, relative offset of LEDs, component rotation angle, and component offset are compared with the corresponding preset thresholds to obtain the comparison results, and abnormal information is determined based on the comparison results.

10. A MiniLED lamp bead defect detection system, characterized in that, The system includes: Labeling module: used to acquire sample image data of MiniLED beads, and to label the sample image data with defect labels to obtain sample image data with defect labels; Preprocessing module: used to generate a defect detection dataset based on sample image data after defect labeling, and to preprocess the defect detection dataset to obtain a preprocessed defect detection dataset; Data augmentation module: Used to perform data augmentation on the preprocessed defect detection dataset to obtain the data-augmented defect detection dataset; Model training module: used to train the YOLOv8 OBB model based on the defect detection dataset after data augmentation, to obtain the trained YOLOv8 OBB model. The YOLOv8 OBB model includes a multi-scale feature augmentation module, a gating fusion module, and a detection output head. The loss function of the YOLOv8 OBB model adopts the rotated box regression loss function. Model export module: Used to export the trained YOLOv8 OBB model to a preset format to obtain a defect detection model; Defect detection module: used to acquire target image data of the MiniLED lamp bead to be detected, and input the target image data into the defect detection model to perform defect detection, and obtain the defect category and rotation rectangle parameters; Defect post-processing module: used to perform defect post-processing based on the defect category and the rotation rectangle parameters.

Images