Mini-led repair method and device based on automatic optical detection

Abstract

The present application relates to the technical field of Mini-LED chip detection, and a Mini-LED repair method and device based on automatic optical detection, comprising: performing Gaussian filtering on an optical chip image to obtain a filtered chip image, training a neural network based on an original chip image set to obtain a defect area detection model, detecting the filtered chip image according to the defect area detection model to obtain a defect image area group, if the defect image area group is not an empty set, recording the product to be repaired as a repairable product, performing local contrast enhancement on the filtered chip image to obtain a target chip image, performing single chip recognition on the target chip image to obtain a single chip area set, identifying a chip area group to be repaired according to the single chip area set, and generating a repair opinion report based on the chip area group to be repaired. The present application can improve the accuracy of Mini-LED product defect detection and reduce the labor consumption of Mini-LED products.

Description

A Mini-LED repair method and apparatus based on automated optical inspection Technical Field

[0001] This invention relates to the field of Mini-LED chip inspection technology, and in particular to a Mini-LED rework method and apparatus based on automatic optical inspection. Background Technology

[0002] With the rapid popularization of Mini-LED in the high-end display field, thousands of micro-level chips of 50-200 micrometers are often integrated on a single panel. The breakage of any chip, electrode oxidation, or phosphor shedding will lead to pixel-level dead pixels, directly reducing the image quality and reliability of the end product. Therefore, how to efficiently and accurately complete the Mini-LED rework has become a key link in determining product yield and manufacturing cost.

[0003] Traditional methods involve manual inspection or a combination of simple optical inspection and manual rework. For example, defects are identified by human visual judgment and experience, and then simple optical equipment is used to assist in the inspection. The rework is then carried out manually. However, this method is highly dependent on human subjective judgment in defect identification, which consumes a lot of manpower and reduces rework efficiency. Summary of the Invention

[0004] This invention provides a method and apparatus for Mini-LED rework based on automatic optical inspection. Its main purpose is to improve the accuracy of defect detection in Mini-LED products and reduce the manpower consumption of Mini-LED products.

[0005] To achieve the above objectives, the present invention provides a Mini-LED rework method based on automated optical inspection, comprising:

[0006] Receive optical inspection instructions, determine the set of products to be repaired based on the optical inspection instructions, wherein the set of products to be repaired includes multiple products to be repaired, and the products to be repaired are Mini-LED products, and the products to be repaired include multiple Mini-LED chips;

[0007] Products to be repaired are extracted sequentially from the set of products to be repaired and placed in a preset background stage to obtain the product to be inspected. The background stage includes a stage, a high-definition camera and a light source.

[0008] The product to be tested is photographed using a high-definition camera and a light source to obtain an image of the optical chip. The optical chip image is then subjected to Gaussian filtering to obtain a filtered chip image.

[0009] A set of raw chip images is acquired, and a pre-acquired neural network is trained based on the raw chip image set to obtain a defect region detection model. The raw chip image set includes multiple raw chip images, and the raw chip images are defective chip images or normal chip images.

[0010] The defect region detection model is used to detect the filter chip image to obtain a defect image region group, which may include multiple defect image regions or be an empty set.

[0011] If the defective image region group is not an empty set, the product to be repaired is recorded as a repairable product, and the local contrast enhancement is performed on the filter chip image to obtain the target chip image.

[0012] Individual chip identification is performed on the target chip image to obtain a single chip region set. Based on the single chip region set, a group of chip regions to be repaired is identified in the defective image region group. A repair opinion report of the reworkable product is generated based on the group of chip regions to be repaired.

[0013] The return feedback reports were compiled, resulting in multiple return feedback reports. Based on these multiple return feedback reports, the Mini-LED return was completed using automated optical inspection.

[0014] Optionally, the step of training the pre-acquired neural network based on the original chip image set to obtain the defect region detection model includes:

[0015] The original chip images are extracted sequentially from the original chip image set, and the image labels of the original chip images are obtained. The image labels include: normal labels or defect labels.

[0016] The original chip image is divided according to the preset region specifications to obtain multiple original image regions;

[0017] The original image regions are extracted sequentially from multiple original image regions, and the original image regions are standardized to obtain the target image region.

[0018] Feature extraction is performed on the target image region to obtain the target region feature vector;

[0019] The feature vector of the target region is labeled based on the image label, and the labeled region feature vector is obtained;

[0020] The feature vectors of the labeled regions corresponding to each original image region in multiple original image regions are summarized to obtain a set of labeled region feature vectors. The feature vector sets of the labeled regions corresponding to each original chip image in the original chip image set are then merged to obtain a set of labeled region feature vectors.

[0021] A loss function is set, and the neural network is trained using the loss function and the feature vector set of the labeled region to obtain a defect region detection model.

[0022] Optionally, setting the loss function includes:

[0023] The training dataset of the neural network during the training process is identified, wherein the training dataset includes multiple training data, and each training data includes: defect probability value and true label value;

[0024] Construct a distribution overlap loss term based on the training dataset, where the distribution overlap loss term is expressed as:

[0025]

[0026] in, This represents the loss term due to overlapping distributions. This indicates the number of training data points in the training dataset. Indicates the first in the training dataset The true label value of each training data point. Indicates the first in the training dataset Defect probability values ​​for each training data point This indicates the preset minimum value;

[0027] The weighted summation of the distribution overlap loss term and the preset classification loss term yields the loss function, where the classification loss term is the binary cross-entropy loss.

[0028] Optionally, the step of detecting the filter chip image according to the defect region detection model to obtain a group of defect image regions includes:

[0029] The filter chip image is divided into regions according to the region specifications to obtain multiple divided image regions;

[0030] Extract segmented image regions sequentially from multiple segmented image regions, and standardize the segmented image regions to obtain standard image regions;

[0031] Feature extraction is performed on the standard image region to obtain the feature vector of the segmented region;

[0032] The feature vector of the segmented region is input into the defect region detection model to obtain the region defect probability value;

[0033] If the probability value of a region defect is greater than the preset standard probability value, then the divided image region is recorded as a defective image region.

[0034] The defective image regions are summarized to obtain a defective image region group.

[0035] Optionally, the step of extracting features from the standard image region to obtain the region feature vector includes:

[0036] Identify color feature groups in a standard image region;

[0037] Generate a gray-level co-occurrence matrix for a standard image region, and calculate a texture feature set for the standard image region based on the gray-level co-occurrence matrix. The texture feature set includes: the contrast of the gray-level co-occurrence matrix, the entropy of the gray-level co-occurrence matrix, and the second angular moment of the gray-level co-occurrence matrix.

[0038] Morphological detection is performed on a standard image region to obtain a morphological feature set, which includes: edge density, number of connected components, and average area of ​​connected components.

[0039] The color feature group, texture feature group, and morphology feature group are merged to obtain the region segmentation feature group, and the region segmentation feature vector is constructed based on the region segmentation feature group.

[0040] Optionally, the step of performing local contrast enhancement on the filtered chip image to obtain the target chip image includes:

[0041] Set a low-frequency traversal template and use it to traverse the image of the filter chip to obtain multiple traversal regions. Each traversal region includes multiple region pixels, and each region pixel corresponds to a region grayscale value.

[0042] Extract the traversed regions sequentially from multiple traversed regions, and calculate the average gray value of the pixels in multiple regions within the traversed regions;

[0043] If the average gray value is not greater than the preset standard gray value, the traversed area is recorded as a low-frequency area.

[0044] By summarizing the low-frequency regions, multiple low-frequency regions are obtained;

[0045] If the average gray value is greater than the standard gray value, the traversed area is recorded as a high-frequency area, and the high-frequency area is enhanced by comparison to obtain the enhanced area.

[0046] By summing up the low-frequency region and the enhancement region respectively, multiple low-frequency regions and multiple enhancement regions are obtained;

[0047] The target chip image is obtained by combining multiple low-frequency regions and multiple enhancement regions.

[0048] Optionally, the enhancement of the high-frequency region to obtain the enhanced region includes:

[0049] The gray standard deviation of the high-frequency region is calculated based on the average gray value. The high-frequency region includes multiple high-frequency pixels, and each high-frequency pixel corresponds to a high-frequency gray value.

[0050] Set the background grayscale value of the image of the filter chip;

[0051] High-frequency pixels are extracted sequentially from multiple high-frequency pixels. Contrast gain is applied to these high-frequency pixels based on a preset gain formula, grayscale standard deviation, and background grayscale value to obtain enhanced pixels. The gain formula is expressed as:

[0052]

[0053] in, This indicates the enhancement of the grayscale value of the pixel. Indicates the background grayscale value. This represents the preset adjustment constant. Indicates the standard deviation of gray levels. This represents the high-frequency grayscale value corresponding to a high-frequency pixel.

[0054] The enhanced pixels corresponding to each high-frequency pixel in the plurality of high-frequency pixels are summarized to obtain an enhanced pixel set. The high-frequency region is updated based on the enhanced pixel set to obtain the enhanced region.

[0055] Optionally, the step of performing single-chip recognition on the target chip image to obtain a single-chip region set includes:

[0056] Contour extraction is performed on the target chip image to obtain a set of connected components, which includes multiple connected components. The contour extraction method is to use the Canny operator.

[0057] Identify the total chip area in the target chip image and set the noise area based on the total chip area;

[0058] The area of ​​each connected component in the connected component set is identified to obtain the connected component area set. The connected component area set is then filtered based on the total chip area and the noise area to obtain the effective connected component area set. The effective connected component area set includes multiple effective connected component areas, and the effective connected component area is greater than the noise area and less than the total chip area.

[0059] In the set of connected components, identify the set of valid connected components that corresponds to the set of valid connected component areas;

[0060] Extract the effective connected components sequentially from the set of effective connected components, and identify the center of each effective connected component, where the center of each connected component is the geometric center of the effective connected component.

[0061] Identify the image center in the target chip image and calculate the geometric distance between the image center and the center of the connected region, where the image center is the geometric center of the target chip image;

[0062] By summing the geometric distances, a geometric distance set is obtained;

[0063] Determine the minimum geometric distance in the geometric distance set, identify the center of the target connected region corresponding to the minimum geometric distance, and use the center of the target connected region as the target seed point;

[0064] A preset region growing algorithm is executed based on the target seed point to obtain a single-chip region set.

[0065] Optionally, the step of identifying the chip region group to be repaired in the defect image region group based on the single chip region set includes:

[0066] Perform the following operation on each defect image region in the defect image region group:

[0067] In a single chip region set, a defect chip region group is determined within the defect image region, wherein the defect chip region group includes multiple defect chip regions, and the defect chip regions are located within the defect image region.

[0068] Defective chip regions are extracted sequentially from the defective chip region group;

[0069] Based on a preset binarization method, identify the light-emitting region and electrode region in the defective chip area;

[0070] Obtain an electrode region template, and use the electrode region template to compare the electrode region to obtain the electrode template matching degree;

[0071] If the electrode template matching degree is not greater than the preset standard matching degree, the defective chip area is recorded as the chip area to be repaired.

[0072] If the electrode template matching degree is greater than the standard matching degree, then the elongated connected region of the luminescent area is identified to obtain the connected region of the luminescent area and determine the length of the connected region of the luminescent area.

[0073] If the length of the connected region is greater than the preset standard scratch length, the defective chip region is recorded as the chip region to be repaired.

[0074] The areas of chips to be repaired are summarized to obtain a group of chip areas to be repaired.

[0075] To achieve the above objectives, the present invention also provides a Mini-LED rework device based on automatic optical inspection, comprising:

[0076] The detection instruction receiving module is used to receive optical detection instructions, determine the set of products to be reworked based on the optical detection instructions, wherein the set of products to be reworked includes multiple products to be reworked, and the products to be reworked are Mini-LED products, and each product to be reworked includes multiple Mini-LED chips. The products to be reworked are extracted sequentially from the set of products to be reworked, and the products to be reworked are placed in a preset background stage to obtain the product to be inspected. The background stage includes: a stage, a high-definition camera and a light source.

[0077] The chip image capturing module is used to capture the product to be inspected using a high-definition camera and a light source to obtain an optical chip image. The optical chip image is then subjected to Gaussian filtering to obtain a filtered chip image. The original chip image set is obtained, and a pre-acquired neural network is trained based on the original chip image set to obtain a defect area detection model. The original chip image set includes multiple original chip images, and the original chip images are either defective chip images or normal chip images.

[0078] The filter image enhancement module is used to detect the filter chip image according to the defect region detection model to obtain a defect image region group. The defect image region group includes multiple defect image regions or the defect image region group is an empty set. If the defect image region group is not an empty set, the product to be repaired is recorded as a repairable product. The filter chip image is then locally contrast enhanced to obtain the target chip image.

[0079] The rework report generation module is used to perform single-chip identification on the target chip image to obtain a single-chip region set, identify the chip region group to be repaired in the defect image region group based on the single-chip region set, generate a rework opinion report for the reworkable product based on the chip region group to be repaired, and summarize the rework opinion reports to obtain multiple rework opinion reports.

[0080] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0081] Memory, storing at least one instruction; and

[0082] The processor executes the instructions stored in the memory to implement the Mini-LED rework method based on automatic optical inspection described above.

[0083] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the Mini-LED rework method based on automatic optical inspection described above. To address the problems described in the background art, this invention first acquires an original chip image set and trains a neural network based on this set to obtain a defect region detection model. This step, by constructing an original chip image set containing both defective and normal chip images and employing a neural network training strategy incorporating a distributed overlap loss term, solves the common problem of positive and negative sample imbalance in Mini-LED defect detection. This allows the defect region detection model to maintain high sensitivity even for a minority of samples (such as rare defects like fractures and electrode oxidation), significantly reducing the false negative rate. Next, the defect region detection model is used to detect filtered chip images, resulting in a defect image region group. The trained defect region detection model is then used to perform region-based probability evaluation on the filtered chip images. This step, through dynamic comparison of region defect probability values ​​with standard probability values, achieves pixel-level precise localization of defect regions, avoiding false positives or false negatives caused by uneven illumination in traditional threshold segmentation methods, thereby improving defect detection accuracy. Furthermore, after confirming that the defect image region group is not empty, local contrast enhancement technology is used to adaptively adjust the gain of high-frequency regions, significantly amplifying the defect regions. The grayscale difference from the normal area solves the problem of difficult identification of reflective or low-contrast defects on the surface of Mini-LED chips, providing a high-contrast image basis for subsequent chip-level defect localization. Then, individual chip identification is performed on the target chip image to obtain a single-chip region set. Based on the single-chip region set, the chip region group to be repaired is identified in the defect image region group. Based on the chip region group to be repaired, a repair opinion report of the reworkable product is generated. This step uses a single-chip identification method that combines Canny operator contour extraction and region growing algorithm to achieve accurate segmentation of individual chip regions in densely arranged Mini-LED arrays. Combined with the dual verification mechanism of electrode template matching and light-emitting area connectivity analysis, different types of defects such as electrode oxidation and chip scratches are effectively distinguished, avoiding misjudgment caused by chip adhesion or pseudo-connectivity in traditional methods. Finally, through multiple repair opinion reports, a defect distribution map covering the entire Mini-LED batch is constructed, enabling operators to directly locate the defect location of specific chips without full-area manual re-inspection, thereby greatly reducing manpower consumption and improving the efficiency of Mini-LED repair. Therefore, this invention can improve the accuracy of defect detection in Mini-LED products and reduce the manpower required for Mini-LED product manufacturing. Attached Figure Description

[0084] Figure 1 is a flowchart illustrating a Mini-LED repair method based on automatic optical inspection according to an embodiment of the present invention.

[0085] Figure 2 is a functional block diagram of a Mini-LED repair device based on automatic optical inspection provided in an embodiment of the present invention;

[0086] Figure 3 is a schematic diagram of the structure of an electronic device that implements the Mini-LED repair method based on automatic optical inspection according to an embodiment of the present invention.

[0087] Explanation of reference numerals in the attached figures:

[0088] 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0089] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0090] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0091] This application provides a Mini-LED rework method based on automated optical inspection. The executing entity of the Mini-LED rework method based on automated optical inspection includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the Mini-LED rework method based on automated optical inspection can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0092] Referring to Figure 1, a flowchart illustrating a Mini-LED rework method based on automated optical inspection according to an embodiment of the present invention is shown. In this embodiment, the Mini-LED rework method based on automated optical inspection includes:

[0093] S1. Receive optical inspection instructions, determine the set of products to be repaired based on the optical inspection instructions, wherein the set of products to be repaired includes multiple products to be repaired, and the products to be repaired are Mini-LED products, and the products to be repaired include multiple Mini-LED chips.

[0094] It is clear that the optical inspection command refers to a human-initiated command to inspect a specific Mini-LED product. The set of products to be repaired refers to a collection of multiple products to be repaired, and the products to be repaired refer to the specific Mini-LED product indicated by the optical inspection command. Such products to be repaired include, for example, Mini-LED TV panels, Mini-LED automotive displays, etc. These products to be repaired are composed of multiple Mini-LED chips, and Mini-LED chips refer to LED light-emitting units with a size of 50-200 micrometers.

[0095] S2. Extract the products to be repaired sequentially from the set of products to be repaired, and place the products to be repaired in a preset background stage to obtain the products to be inspected. The background stage includes: a stage, a high-definition camera and a light source.

[0096] Understandably, the background stage refers to the equipment used to photograph the product to be repaired. The background stage includes a stage, a high-definition camera, and a light source. The stage is a carrier for holding the product to be repaired, the high-definition camera is an optical sensor for capturing high-resolution images, such as a 50-megapixel industrial CCD camera, and the light source is an optical component that provides uniform illumination, such as a ring-shaped LED white light source. The product to be inspected refers to the product to be repaired placed on the background stage.

[0097] S3. Use a high-definition camera and light source to photograph the product to be tested to obtain an image of the optical chip. Perform Gaussian filtering on the optical chip image to obtain a filtered chip image.

[0098] It is clear that the optical chip image refers to the frontal image of the product to be tested obtained after being photographed, and this optical chip image includes all the Mini-LED chips in the product to be tested. The filtered chip image refers to the image after the optical chip image has been Gaussian filtered.

[0099] For example, the detailed steps for taking pictures of the product to be tested using a high-definition camera and light source are as follows: an operator, Xiao Zhang, fixes the product to be tested in the center of the stage, then Xiao Zhang adjusts the light source to a 45° incident angle (as specified by the operating standard) to ensure uniform illumination, sets the camera parameters: sets the focal length and exposure time, triggers the camera to take pictures and saves the image.

[0100] S4. Obtain the original chip image set and train the pre-acquired neural network based on the original chip image set to obtain the defect region detection model. The original chip image set includes multiple original chip images, and the original chip images are defective chip images or normal chip images.

[0101] It should be explained that the original chip image set refers to a collection of multiple original chip images. The original chip images refer to images of products with the same model number as the product to be repaired, which are obtained in advance by human intervention. The original chip images include defective chip images and normal chip images. The defective chip images refer to images of products with defects (such as chip breakage, electrode oxidation, phosphor shedding, etc.) that need to be repaired, while the normal chip images refer to images of products that do not need to be repaired. The original chip images in the above-mentioned original chip image set are all from repaired products and normally produced products from previous periods.

[0102] Furthermore, the neural network can be selected as either a Convolutional Neural Network (CNN) or a VisionTransformer (ViT). The defect region detection model refers to a model capable of determining whether a defect exists in a product based on an image of the product to be repaired. The input to this defect region detection model is a vector, and the output is a probability value (corresponding to subsequent region defect probability values). When this probability value is greater than a preset standard probability value (corresponding to subsequent standard probability values), it indicates that the product corresponding to the input image has a defect and needs to be repaired.

[0103] In detail, the step of training the pre-acquired neural network based on the original chip image set to obtain the defect region detection model includes:

[0104] The original chip images are extracted sequentially from the original chip image set, and the image labels of the original chip images are obtained. The image labels include: normal labels or defect labels.

[0105] The original chip image is divided according to the preset region specifications to obtain multiple original image regions;

[0106] The original image regions are extracted sequentially from multiple original image regions, and the original image regions are standardized to obtain the target image region.

[0107] Feature extraction is performed on the target image region to obtain the target region feature vector;

[0108] The feature vector of the target region is labeled based on the image label, and the labeled region feature vector is obtained;

[0109] The feature vectors of the labeled regions corresponding to each original image region in multiple original image regions are summarized to obtain a set of labeled region feature vectors. The feature vector sets of the labeled regions corresponding to each original chip image in the original chip image set are then merged to obtain a set of labeled region feature vectors.

[0110] A loss function is set, and the neural network is trained using the loss function and the feature vector set of the labeled region to obtain a defect region detection model.

[0111] It should be explained that the region specification refers to a manually set rectangular size, which is used to divide the original chip image into multiple rectangular regions. The size of these rectangular regions is the region specification. The division refers to dividing the original chip image into multiple rectangular regions with the specified region specification, and all rectangular regions do not overlap. The multiple original image regions refer to multiple regions in the original chip image after division, wherein the size of the original image regions is the aforementioned region specification. The target image region refers to the original image region after standardization, wherein standardization refers to normalizing the pixel values ​​(e.g., using z-score standardization to scale the pixel values ​​to the [0,1] range). The purpose of this standardization is to eliminate the interference of illumination differences on feature extraction.

[0112] It is clear that the target region feature vector refers to a vector that quantifies the features of the target image region. The steps for obtaining this target region feature vector are the same as those for obtaining the subsequent region segmentation feature vector, and the method for obtaining the region segmentation feature vector will be described in detail later. The image label refers to the label of the product status (normal or defective) corresponding to the original chip image, where the normal label index value is 0 and the defective label index value is 1. Both the normal and defective labels are set manually. The labeled region feature vector refers to the labeled target region feature vector, where labeling refers to associating the target region feature vector with the corresponding image label. The labeling step is to add a label field (i.e., a value of 0 or 1) to each target region feature vector. Merging the labeled region feature vector group corresponding to each original chip image in the original chip image set means: putting the labeled region feature vectors in all the labeled region feature vector groups into the same set, which is the labeled region feature vector set. The loss function refers to the function used to measure the prediction error of the model. The above process of training the neural network is supervised training, and the training method is existing technology, which will not be described in detail here.

[0113] Specifically, the setting of the loss function includes:

[0114] The training dataset of the neural network during the training process is identified, wherein the training dataset includes multiple training data, and each training data includes: defect probability value and true label value;

[0115] Construct a distribution overlap loss term based on the training dataset, where the distribution overlap loss term is expressed as:

[0116]

[0117] in, This represents the loss term due to overlapping distributions. This indicates the number of training data points in the training dataset. Indicates the first in the training dataset The true label value of each training data point. Indicates the first in the training dataset Defect probability values ​​for each training data point This indicates the preset minimum value;

[0118] The weighted summation of the distribution overlap loss term and the preset classification loss term yields the loss function, where the classification loss term is the binary cross-entropy loss.

[0119] It is clear that the training dataset refers to a collection of multiple training data, which includes: defect probability values ​​and true label values. The defect probability value refers to the probability value output by the neural network after a certain training iteration during the training process. The true label value and the defect probability value come from the same iteration process, and the true label value refers to the value represented by the image label in the feature vector of the labeled region during the iteration process. The distribution overlap loss term refers to the loss term used to constrain the consistency between the predicted probability distribution and the true distribution. The reason for introducing this distribution overlap loss term is to alleviate the problem of a large difference between the number of defective chip images and the number of normal chip images (i.e., imbalance of positive and negative samples). The higher the distribution overlap loss term, the higher the overlap between the predicted distribution and the true distribution, that is, the smaller the loss.

[0120] Furthermore, the minimum value refers to a manually set constant, the purpose of which is to prevent the denominator in the formula from being 0. The classification loss term refers to the binary cross-entropy loss; through the combined effect of this classification loss term and the distribution overlap loss term, the distribution alignment and classification accuracy of the model can be optimized simultaneously. The weighted summation is expressed as: ,in, Represents the loss function. and These represent the manually set overlap loss weight and the manually set classification loss weight, respectively. The term represents the classification loss term. The aforementioned overlap loss weight and classification loss weight refer to the weight coefficients of the distribution loss and classification loss, respectively. Optionally, the overlap loss weight and classification loss weight are set to 0.6 and 0.4, respectively.

[0121] S5. Detect the filter chip image according to the defect region detection model to obtain a defect image region group, wherein the defect image region group includes multiple defect image regions or the defect image region group is an empty set.

[0122] It is clear that the defect image region group refers to the combination of regions in the filter chip image where defects occur. Since there may be no defects in the filter chip image, there are no defect image regions in the filter chip image, that is, the defect image region group is an empty set.

[0123] Specifically, the step of detecting the filter chip image based on the defect region detection model to obtain a group of defect image regions includes:

[0124] The filter chip image is divided into regions according to the region specifications to obtain multiple divided image regions;

[0125] Extract segmented image regions sequentially from multiple segmented image regions, and standardize the segmented image regions to obtain standard image regions;

[0126] Feature extraction is performed on the standard image region to obtain the feature vector of the segmented region;

[0127] The feature vector of the segmented region is input into the defect region detection model to obtain the region defect probability value;

[0128] If the probability value of a region defect is greater than the preset standard probability value, then the divided image region is recorded as a defective image region.

[0129] The defective image regions are summarized to obtain a defective image region group.

[0130] It is clear that the image region segmentation refers to a portion of the filtered chip image after region segmentation. The steps for this region segmentation are the same as those for segmenting the original chip image, and will not be repeated here. The standard image region refers to the segmented image region after standardization, and this standardization is the same as the standardization steps for the original image region. The segmented region feature vector refers to the vector that quantifies the features of the standard image region. The region defect probability value refers to the output value of the defect region detection model; the higher the region defect probability value, the higher the probability of a defect occurring in the standard image region. The standard probability value refers to a manually set probability constant.

[0131] In detail, the step of extracting features from the standard image region to obtain the region feature vector includes:

[0132] Identify color feature groups in a standard image region;

[0133] Generate a gray-level co-occurrence matrix for a standard image region, and calculate a texture feature set for the standard image region based on the gray-level co-occurrence matrix. The texture feature set includes: the contrast of the gray-level co-occurrence matrix, the entropy of the gray-level co-occurrence matrix, and the second angular moment of the gray-level co-occurrence matrix.

[0134] Morphological detection is performed on a standard image region to obtain a morphological feature set, which includes: edge density, number of connected components, and average area of ​​connected components.

[0135] The color feature group, texture feature group, and morphology feature group are merged to obtain the region segmentation feature group, and the region segmentation feature vector is constructed based on the region segmentation feature group.

[0136] It should be explained that the color feature set refers to a combination of parameters characterizing the statistical properties of image color, including: the mean of RGB three channels, the variance of HSV saturation, etc. The gray-level co-occurrence matrix refers to a probability matrix describing the spatial relationships of texture. The contrast, entropy, and second moment of the gray-level co-occurrence matrix in this texture feature set can all be calculated using existing formulas. The morphological feature set refers to a combination of features describing the geometric structure of a standard image region. Here, edge density refers to the number of edge pixels in the standard image region divided by the total number of pixels in the standard image region; the number of connected components refers to the total number of connected components obtained after binarizing the standard image region; and the average area of ​​connected components refers to the average area of ​​all connected components in the standard image region after binarization.

[0137] For example, if the region feature group is (A1, A2, and A3), then the corresponding region feature vector is: .

[0138] S6. If the defective image region group is not an empty set, the product to be repaired is recorded as a repairable product, and the local contrast enhancement is performed on the filter chip image to obtain the target chip image.

[0139] It is clear that if the defect image area group is not an empty set, it indicates that a defect has occurred in the product to be repaired, meaning that the product needs to be repaired. To ensure the accuracy of subsequent repairs, after confirming that the product to be repaired is a repairable product, it is also necessary to determine the specific chips in the repairable product that need to be repaired.

[0140] Furthermore, the purpose of the local contrast enhancement is to improve the contrast between the defective area and the background. The target chip image refers to the filtered chip image after local contrast enhancement.

[0141] Specifically, the step of performing local contrast enhancement on the filtered chip image to obtain the target chip image includes:

[0142] Set a low-frequency traversal template and use it to traverse the image of the filter chip to obtain multiple traversal regions. Each traversal region includes multiple region pixels, and each region pixel corresponds to a region grayscale value.

[0143] Extract the traversed regions sequentially from multiple traversed regions, and calculate the average gray value of the pixels in multiple regions within the traversed regions;

[0144] If the average gray value is not greater than the preset standard gray value, the traversed area is recorded as a low-frequency area.

[0145] By summarizing the low-frequency regions, multiple low-frequency regions are obtained;

[0146] If the average gray value is greater than the standard gray value, the traversed area is recorded as a high-frequency area, and the high-frequency area is enhanced by comparison to obtain the enhanced area.

[0147] By summing up the low-frequency region and the enhancement region respectively, multiple low-frequency regions and multiple enhancement regions are obtained;

[0148] The target chip image is obtained by combining multiple low-frequency regions and multiple enhancement regions.

[0149] It should be explained that the low-frequency traversal template refers to a manually set sliding window (e.g., 15×15 pixels), the size of which is manually set. The multiple traversal regions refer to the multiple regions traversed by the low-frequency traversal template during the traversal of the filter chip image. The region pixels refer to the pixels within the traversal region, and the region grayscale value refers to the grayscale value of the region pixels. The average grayscale value refers to the average of the grayscale values ​​of multiple region pixels within the traversal region. The standard grayscale value refers to a pre-defined grayscale value constant. If the average grayscale value is not greater than the standard grayscale value, it indicates insufficient illumination in the region, and the traversal region is recorded as a low-frequency region. If the average grayscale value is greater than the standard grayscale value, it indicates a risk of overexposure in the region, and the traversal region is recorded as a high-frequency region. Since high-frequency regions easily mask subtle defects, contrast enhancement is required for these high-frequency regions. The enhanced region refers to the high-frequency region after contrast enhancement. Combining multiple low-frequency regions and multiple enhanced regions means recombining the processed regions according to their original coordinates.

[0150] Specifically, the enhancement of the high-frequency region to obtain the enhanced region includes:

[0151] The gray standard deviation of the high-frequency region is calculated based on the average gray value. The high-frequency region includes multiple high-frequency pixels, and each high-frequency pixel corresponds to a high-frequency gray value.

[0152] Set the background grayscale value of the image of the filter chip;

[0153] High-frequency pixels are extracted sequentially from multiple high-frequency pixels. Contrast gain is applied to these high-frequency pixels based on a preset gain formula, grayscale standard deviation, and background grayscale value to obtain enhanced pixels. The gain formula is expressed as:

[0154]

[0155] in, This indicates the enhancement of the grayscale value of the pixel. Indicates the background grayscale value. This represents the preset adjustment constant. Indicates the standard deviation of gray levels. This represents the high-frequency grayscale value corresponding to a high-frequency pixel.

[0156] The enhanced pixels corresponding to each high-frequency pixel in the plurality of high-frequency pixels are summarized to obtain an enhanced pixel set. The high-frequency region is updated based on the enhanced pixel set to obtain the enhanced region.

[0157] It is clear that the grayscale standard deviation refers to the standard deviation of multiple high-frequency grayscale values ​​corresponding to multiple high-frequency pixels in the high-frequency region. The background grayscale value refers to the grayscale value of the background portion in the high-frequency region, which is set manually after detecting the background portion, where the background portion refers to the black encapsulation area between the Mini-LED chips. The gain formula refers to the formula for calculating the grayscale value of the enhanced pixel, where the enhanced pixel refers to the high-frequency pixel after contrast gain. The adjustment constant refers to a dimensionless constant set manually, which controls the enhancement amplitude; optionally, the adjustment constant is set to 1.8.

[0158] Furthermore, updating the high-frequency region based on the enhanced pixel set means replacing multiple high-frequency pixels in the high-frequency region with the enhanced pixel set, and the high-frequency region after replacement is the enhanced region.

[0159] S7. Perform single-chip identification on the target chip image to obtain a single-chip region set. Based on the single-chip region set, identify the chip region group to be repaired in the defect image region group. Generate a repair opinion report for the reworkable product based on the chip region group to be repaired.

[0160] Understandably, the single-chip region set refers to a collection of multiple single-chip regions, where a single-chip region refers to a region containing only one Mini-LED chip. The chip region group to be repaired includes multiple chip regions to be repaired, and the chip region to be repaired refers to a single-chip region containing a Mini-LED chip with defects (such as large-area scratches). The repair feedback report refers to a report that comprehensively describes the repairable product, which includes the product number of the repairable product and the location of each chip region to be repaired in the chip region group corresponding to the repairable product.

[0161] In detail, the step of performing single-chip identification on the target chip image to obtain a single-chip region set includes:

[0162] Contour extraction is performed on the target chip image to obtain a set of connected components, which includes multiple connected components. The contour extraction method is to use the Canny operator.

[0163] Identify the total chip area in the target chip image and set the noise area based on the total chip area;

[0164] The area of ​​each connected component in the connected component set is identified to obtain the connected component area set. The connected component area set is then filtered based on the total chip area and the noise area to obtain the effective connected component area set. The effective connected component area set includes multiple effective connected component areas, and the effective connected component area is greater than the noise area and less than the total chip area.

[0165] In the set of connected components, identify the set of valid connected components that corresponds to the set of valid connected component areas;

[0166] Extract the effective connected components sequentially from the set of effective connected components, and identify the center of each effective connected component, where the center of each connected component is the geometric center of the effective connected component.

[0167] Identify the image center in the target chip image and calculate the geometric distance between the image center and the center of the connected region, where the image center is the geometric center of the target chip image;

[0168] By summing the geometric distances, a geometric distance set is obtained;

[0169] Determine the minimum geometric distance in the geometric distance set, identify the center of the target connected region corresponding to the minimum geometric distance, and use the center of the target connected region as the target seed point;

[0170] A preset region growing algorithm is executed based on the target seed point to obtain a single-chip region set.

[0171] It is clear that the total chip area refers to the total area of ​​the target chip image. The connected component area refers to the area of ​​a connected component. The noise area refers to the area of ​​a connected component in the target chip image caused by minor noise, and this noise area is calculated as: total chip area × 1%. The effective connected component area set refers to the set of connected component areas after filtering. The purpose of the above step of filtering the connected component area set based on the total chip area to obtain the effective connected component area set is to remove pseudo-connected components with areas that are too large (pallet edge) or too small (noise spots), and only retain the real chip area. The effective connected component set refers to the set of multiple connected components corresponding to each effective connected component area in the effective connected component area set. The geometric distance refers to the Euclidean geometric distance between the image center and the center of the connected component. The minimum geometric distance refers to the geometric distance with the smallest value in the geometric distance set. The target connected component center refers to the center of the connected component corresponding to the minimum geometric distance. The target seed point refers to the center point of the connected component that is closest (in terms of distance) to the center of the target chip image. The reason for determining the minimum geometric distance and the center of the target connected region, and using the center of the target connected region as the target seed point, is that the central region is least affected by optical distortion. This method ensures that region growth starts from the most complete chip. The above region growth algorithm is existing technology and will not be described in detail here.

[0172] Specifically, the step of identifying the chip region group to be repaired in the defect image region group based on the single chip region set includes:

[0173] Perform the following operation on each defect image region in the defect image region group:

[0174] In a single chip region set, a defect chip region group is determined within the defect image region, wherein the defect chip region group includes multiple defect chip regions, and the defect chip regions are located within the defect image region.

[0175] Defective chip regions are extracted sequentially from the defective chip region group;

[0176] Based on a preset binarization method, identify the light-emitting region and electrode region in the defective chip area;

[0177] Obtain an electrode region template, and use the electrode region template to compare the electrode region to obtain the electrode template matching degree;

[0178] If the electrode template matching degree is not greater than the preset standard matching degree, the defective chip area is recorded as the chip area to be repaired.

[0179] If the electrode template matching degree is greater than the standard matching degree, then the elongated connected region of the luminescent area is identified to obtain the connected region of the luminescent area and determine the length of the connected region of the luminescent area.

[0180] If the length of the connected region is greater than the preset standard scratch length, the defective chip region is recorded as the chip region to be repaired.

[0181] The areas of chips to be repaired are summarized to obtain a group of chip areas to be repaired.

[0182] It is clear that the defective chip region refers to the region of a single Mini-LED chip contained within the defective image region. The binarization method refers to: firstly, statistically analyzing the grayscale histogram of the defective chip region, and then binarizing the defective chip region based on the grayscale histogram to obtain the light-emitting region and the electrode region. The aforementioned grayscale histogram statistics and image binarization are existing technologies and will not be elaborated upon here. The light-emitting region refers to the light-emitting die portion of the Mini-LED chip contained within the defective chip region, and the electrode region refers to the metal structure of the soldered gold wires or pads. The electrode region template refers to the binary image of a standard defect-free electrode. This electrode region template is obtained by collecting 100 sets of normal electrode images and averaging them. The electrode template matching degree refers to a numerical value that quantifies the similarity between the electrode region template and the electrode region. The higher the electrode template matching degree, the higher the similarity between the electrode region template and the electrode region. The electrode template matching degree is calculated by calculating the Structural Similarity Index (SSIM) between the electrode region template and the electrode region. The standard matching degree refers to a matching degree constant set by humans. When the matching degree of the electrode template is not greater than the standard matching degree, it means that the electrode area is significantly different from the electrode area template, that is, the electrode area has a defect.

[0183] Furthermore, the connected region of the light-emitting area refers to a rectangular connected region within the light-emitting area. This connected region is identified using the Hough line detection method. The length of the connected region refers to the length of the longer side of the connected region. The standard scratch length is a manually set length. If the connected region length is greater than the standard scratch length, it indicates that a scratch has occurred in the light-emitting area of ​​the defective chip, requiring rework.

[0184] S8. Summarize the return feedback reports to obtain multiple return feedback reports, and complete the Mini-LED return based on automatic optical inspection based on multiple return feedback reports.

[0185] It should be explained that in the multiple return feedback reports, each return feedback report corresponds to a reworkable product, and each return feedback report indicates the location of the Mini-LED chip in the reworkable product that may be defective, which can greatly reduce the need for manual inspection of all chips in the reworkable product.

[0186] To address the problems described in the background art, this invention first acquires an original chip image set and trains a neural network based on this set to obtain a defect region detection model. This step, by constructing an original chip image set containing both defective and normal chip images and employing a neural network training strategy incorporating a distributed overlap loss term, solves the common problem of positive and negative sample imbalance in Mini-LED defect detection. This allows the defect region detection model to maintain high sensitivity even for a minority of samples (such as rare defects like fractures and electrode oxidation), significantly reducing the false negative rate. Next, the defect region detection model is used to detect filtered chip images, resulting in a defect image region group. The trained defect region detection model is then used to perform region-based probability evaluation on the filtered chip images. This step, through dynamic comparison of region defect probability values ​​with standard probability values, achieves pixel-level precise localization of defect regions, avoiding false positives or false negatives caused by uneven illumination in traditional threshold segmentation methods, thereby improving defect detection accuracy. Furthermore, after confirming that the defect image region group is not empty, local contrast enhancement technology is used to adaptively adjust the gain of high-frequency regions, significantly amplifying the defect regions. The grayscale difference from the normal area solves the problem of difficult identification of reflective or low-contrast defects on the surface of Mini-LED chips, providing a high-contrast image basis for subsequent chip-level defect localization. Then, individual chip identification is performed on the target chip image to obtain a single-chip region set. Based on the single-chip region set, the chip region group to be repaired is identified in the defect image region group. Based on the chip region group to be repaired, a repair opinion report of the reworkable product is generated. This step uses a single-chip identification method that combines Canny operator contour extraction and region growing algorithm to achieve accurate segmentation of individual chip regions in densely arranged Mini-LED arrays. Combined with the dual verification mechanism of electrode template matching and light-emitting area connectivity analysis, different types of defects such as electrode oxidation and chip scratches are effectively distinguished, avoiding misjudgment caused by chip adhesion or pseudo-connectivity in traditional methods. Finally, through multiple repair opinion reports, a defect distribution map covering the entire Mini-LED batch is constructed, enabling operators to directly locate the defect location of specific chips without full-area manual re-inspection, thereby greatly reducing manpower consumption and improving the efficiency of Mini-LED repair. Therefore, this invention can improve the accuracy of defect detection in Mini-LED products and reduce the manpower required for Mini-LED product manufacturing.

[0187] Figure 2 shows a functional block diagram of a Mini-LED rework device based on automatic optical inspection provided in an embodiment of the present invention.

[0188] The Mini-LED rework device 100 based on automatic optical inspection described in this invention can be installed in electronic devices. Depending on the functions implemented, the Mini-LED rework device 100 based on automatic optical inspection may include a detection command receiving module 101, a chip image capturing module 102, a filtered image enhancement module 103, and a rework report generation module 104. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0189] The detection instruction receiving module 101 is used to receive optical detection instructions, determine the set of products to be repaired based on the optical detection instructions, wherein the set of products to be repaired includes multiple products to be repaired, and the products to be repaired are Mini-LED products, and the products to be repaired include multiple Mini-LED chips. The products to be repaired are extracted sequentially from the set of products to be repaired, and the products to be repaired are placed in a preset background stage to obtain the products to be inspected. The background stage includes: a stage, a high-definition camera and a light source.

[0190] The chip image capturing module 102 is used to capture the product to be inspected using a high-definition camera and a light source to obtain an optical chip image, perform Gaussian filtering on the optical chip image to obtain a filtered chip image, acquire an original chip image set, and train a pre-acquired neural network based on the original chip image set to obtain a defect area detection model. The original chip image set includes multiple original chip images, and the original chip images are defective chip images or normal chip images.

[0191] The filtered image enhancement module 103 is used to detect the filtered chip image according to the defect region detection model to obtain a defect image region group. The defect image region group includes multiple defect image regions or the defect image region group is an empty set. If the defect image region group is not an empty set, the product to be repaired is recorded as a repairable product. The local contrast enhancement of the filtered chip image is performed to obtain the target chip image.

[0192] The repair report generation module 104 is used to perform single chip recognition on the target chip image to obtain a single chip region set, identify the chip region group to be repaired in the defect image region group based on the single chip region set, generate a repair opinion report for the repairable product based on the chip region group to be repaired, and summarize the repair opinion reports to obtain multiple repair opinion reports.

[0193] In detail, each module in the Mini-LED rework device 100 based on automatic optical detection described in this embodiment of the invention uses the same technical means as the Mini-LED rework method based on automatic optical detection described in FIG1 above, and can produce the same technical effect, which will not be repeated here.

[0194] Figure 3 shows a schematic diagram of an electronic device that implements a Mini-LED repair method based on automatic optical inspection, according to an embodiment of the present invention.

[0195] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a Mini-LED repair method program based on automatic optical inspection.

[0196] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a Mini-LED repair method program based on automatic optical inspection, but also to temporarily store data that has been output or will be output.

[0197] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a Mini-LED rework method program based on automatic optical inspection) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0198] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0199] Figure 3 only shows an electronic device with components. Those skilled in the art will understand that the structure shown in Figure 3 does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0200] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0201] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0202] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0203] The Mini-LED repair method program based on automatic optical inspection stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0204] Receive optical inspection instructions, determine the set of products to be repaired based on the optical inspection instructions, wherein the set of products to be repaired includes multiple products to be repaired, and the products to be repaired are Mini-LED products, and the products to be repaired include multiple Mini-LED chips;

[0205] Products to be repaired are extracted sequentially from the set of products to be repaired and placed in a preset background stage to obtain the product to be inspected. The background stage includes a stage, a high-definition camera and a light source.

[0206] The product to be tested is photographed using a high-definition camera and a light source to obtain an image of the optical chip. The optical chip image is then subjected to Gaussian filtering to obtain a filtered chip image.

[0207] A set of raw chip images is acquired, and a pre-acquired neural network is trained based on the raw chip image set to obtain a defect region detection model. The raw chip image set includes multiple raw chip images, and the raw chip images are defective chip images or normal chip images.

[0208] The defect region detection model is used to detect the filter chip image to obtain a defect image region group, which may include multiple defect image regions or be an empty set.

[0209] If the defective image region group is not an empty set, the product to be repaired is recorded as a repairable product, and the local contrast enhancement is performed on the filter chip image to obtain the target chip image.

[0210] Individual chip identification is performed on the target chip image to obtain a single chip region set. Based on the single chip region set, a group of chip regions to be repaired is identified in the defective image region group. A repair opinion report of the reworkable product is generated based on the group of chip regions to be repaired.

[0211] The return feedback reports were compiled, resulting in multiple return feedback reports. Based on these multiple return feedback reports, the Mini-LED return was completed using automated optical inspection.

[0212] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiments of Figures 1 to 3, which will not be repeated here.

[0213] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0214] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0215] Receive optical inspection instructions, determine the set of products to be repaired based on the optical inspection instructions, wherein the set of products to be repaired includes multiple products to be repaired, and the products to be repaired are Mini-LED products, and the products to be repaired include multiple Mini-LED chips;

[0216] Products to be repaired are extracted sequentially from the set of products to be repaired and placed in a preset background stage to obtain the product to be inspected. The background stage includes a stage, a high-definition camera and a light source.

[0217] The product to be tested is photographed using a high-definition camera and a light source to obtain an image of the optical chip. The optical chip image is then subjected to Gaussian filtering to obtain a filtered chip image.

[0218] A set of raw chip images is acquired, and a pre-acquired neural network is trained based on the raw chip image set to obtain a defect region detection model. The raw chip image set includes multiple raw chip images, and the raw chip images are defective chip images or normal chip images.

[0219] The defect region detection model is used to detect the filter chip image to obtain a defect image region group, which may include multiple defect image regions or be an empty set.

[0220] If the defective image region group is not an empty set, the product to be repaired is recorded as a repairable product, and the local contrast enhancement is performed on the filter chip image to obtain the target chip image.

[0221] Individual chip identification is performed on the target chip image to obtain a single chip region set. Based on the single chip region set, a group of chip regions to be repaired is identified in the defective image region group. A repair opinion report of the reworkable product is generated based on the group of chip regions to be repaired.

[0222] The return feedback reports were compiled, resulting in multiple return feedback reports. Based on these multiple return feedback reports, the Mini-LED return was completed using automated optical inspection.

[0223] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0224] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0225] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0226] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention and not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A Mini-LED rework method based on automated optical inspection, characterized in that, The method includes: receiving an optical inspection command; determining a set of products to be repaired based on the optical inspection command, wherein the set of products to be repaired includes multiple products to be repaired, and the products to be repaired are Mini-LED products, and each product to be repaired includes multiple Mini-LED chips; sequentially extracting the products to be repaired from the set of products to be repaired; placing the products to be repaired in a preset background stage to obtain the product to be inspected, wherein the background stage includes: a stage, a high-definition camera, and a light source; using the high-definition camera and the light source to photograph the product to be inspected to obtain an optical chip image; and performing Gaussian filtering on the optical chip image to obtain a filtered chip. The process involves obtaining a set of raw chip images and training a pre-acquired neural network based on these images to obtain a defect region detection model. The raw chip image set includes multiple raw chip images, which can be either defective or normal chip images. The loss function during training is set as follows: The training dataset for the neural network is identified, comprising multiple training data points, each including a defect probability value and a true label value. A distribution overlap loss term is constructed based on the training dataset, expressed as: in, This represents the loss term due to overlapping distributions. This indicates the number of training data points in the training dataset. Indicates the first in the training dataset The true label value of each training data point. Indicates the first in the training dataset The defect probability value of each training data point. The preset minimum value is represented by the distribution overlap loss term, which is a loss term used to constrain the consistency between the predicted probability distribution and the true distribution. The distribution overlap loss term and the preset classification loss term are weighted and summed to obtain the loss function, where the classification loss term is the binary cross-entropy loss. The filtered chip image is detected according to the defect region detection model to obtain a defect image region group, where the defect image region group includes multiple defect image regions or is an empty set. If the defect image region group is not an empty set, the product to be repaired is recorded as a repairable product, and local contrast enhancement is performed on the filtered chip image to obtain the target chip image. A low-frequency traversal template is set, and the filtered chip image is traversed using the low-frequency traversal template to obtain multiple traversal regions, where each traversal region includes multiple region pixels, and each region pixel corresponds to a region grayscale value. Traversal regions are extracted sequentially from the multiple traversal regions, and the traversal region is calculated. The average grayscale value of pixels in multiple regions is calculated. If the average grayscale value is not greater than a preset standard grayscale value, the traversed region is recorded as a low-frequency region. The low-frequency regions are summarized to obtain multiple low-frequency regions. If the average grayscale value is greater than the standard grayscale value, the traversed region is recorded as a high-frequency region, and the high-frequency region is enhanced to obtain an enhanced region. The low-frequency regions and enhanced regions are summarized to obtain multiple low-frequency regions and multiple enhanced regions. The multiple low-frequency regions and multiple enhanced regions are combined to obtain a target chip image. Single chip recognition is performed on the target chip image to obtain a single chip region set. Based on the single chip region set, a chip region group to be repaired is identified in the defect image region group. A repair opinion report for the reworkable product is generated based on the chip region group to be repaired. The repair opinion reports are summarized to obtain multiple repair opinion reports. The Mini-LED repair based on automatic optical inspection is completed based on the multiple repair opinion reports.

2. The Mini-LED rework method based on automated optical inspection as described in claim 1, characterized in that, The method of training a pre-acquired neural network based on a set of original chip images to obtain a defect region detection model includes: sequentially extracting original chip images from the original chip image set and obtaining image labels for the original chip images, wherein the image labels include: normal labels or defect labels; dividing the original chip images according to a preset region specification to obtain multiple original image regions; sequentially extracting original image regions from the multiple original image regions and standardizing the original image regions to obtain target image regions; extracting features from the target image regions to obtain target region feature vectors; labeling the target region feature vectors based on the image labels to obtain labeled region feature vectors; summarizing the labeled region feature vectors corresponding to each original image region in the multiple original image regions to obtain a labeled region feature vector group; merging the labeled region feature vector groups corresponding to each original chip image in the original chip image set to obtain a labeled region feature vector set; setting a loss function and training the neural network using the loss function and the labeled region feature vector set to obtain a defect region detection model.

3. The Mini-LED rework method based on automated optical inspection as described in claim 2, characterized in that, The step of detecting the filter chip image using the defect region detection model to obtain a defect image region group includes: dividing the filter chip image into regions according to the region specifications to obtain multiple divided image regions; sequentially extracting the divided image regions from the multiple divided image regions, standardizing the divided image regions to obtain standard image regions; extracting features from the standard image regions to obtain divided region feature vectors; inputting the divided region feature vectors into the defect region detection model to obtain region defect probability values; if the region defect probability value is greater than a preset standard probability value, then the divided image region is recorded as a defect image region; and summing the defect image regions to obtain a defect image region group.

4. The Mini-LED rework method based on automatic optical inspection as described in claim 3, characterized in that, The step of extracting features from a standard image region to obtain a region-dividing feature vector includes: identifying color feature groups in the standard image region; generating a gray-level co-occurrence matrix (GLCM) of the standard image region; calculating a texture feature group of the standard image region based on the GLCM, wherein the texture feature group includes: the contrast of the GLCM, the entropy of the GLCM, and the second angular moment of the GLCM; performing morphological detection on the standard image region to obtain a morphological feature group, wherein the morphological feature group includes: edge density, the number of connected components, and the average area of ​​connected components; merging the color feature group, texture feature group, and morphological feature group to obtain a region-dividing feature group, and constructing a region-dividing feature vector based on the region-dividing feature group.

5. The Mini-LED rework method based on automatic optical inspection as described in claim 4, characterized in that, The enhancement of the high-frequency region to obtain the enhanced region includes: calculating the grayscale standard deviation of the high-frequency region based on the average grayscale value, wherein the high-frequency region includes multiple high-frequency pixels, and each high-frequency pixel corresponds to a high-frequency grayscale value; setting the background grayscale value of the filter chip image; sequentially extracting high-frequency pixels from the multiple high-frequency pixels, and performing contrast gain on the high-frequency pixels according to a preset gain formula, grayscale standard deviation, and background grayscale value to obtain the enhanced pixels, wherein the gain formula is expressed as: in, This indicates the enhancement of the grayscale value of the pixel. Indicates the background grayscale value. This represents the preset adjustment constant. Indicates the standard deviation of gray levels. The high-frequency grayscale value corresponding to the high-frequency pixel is represented; the enhanced pixel value corresponding to each high-frequency pixel in the multiple high-frequency pixels is summarized to obtain the enhanced pixel set; the high-frequency region is updated based on the enhanced pixel set to obtain the enhanced region.

6. The Mini-LED rework method based on automated optical inspection as described in claim 5, characterized in that, The step of identifying a single chip in the target chip image to obtain a single-chip region set includes: extracting contours from the target chip image to obtain a connected component set, wherein the connected component set includes multiple connected components, and the contour extraction is performed using the Canny operator; identifying the total chip area in the target chip image, and setting a noise area based on the total chip area; confirming the connected component area of ​​each connected component in the connected component set to obtain a connected component area set; filtering the connected component area set based on the total chip area and the noise area to obtain a valid connected component area set, wherein the valid connected component area set includes multiple valid connected component areas, and the valid connected component area is greater than the noise area and less than the total chip area; The set of connected components is identified as the set of valid connected components corresponding to the set of valid connected component areas. Valid connected components are extracted sequentially from the set of valid connected components, and the centers of these components are identified, where each center is the geometric center of the valid connected component. The image center in the target chip image is identified, and the geometric distance between the image center and the center of the connected component is calculated, where the image center is the geometric center of the target chip image. These geometric distances are then summarized to obtain a set of geometric distances. The minimum geometric distance in the set of geometric distances is determined, and the target connected component center corresponding to the minimum geometric distance is identified. This target connected component center is then used as the target seed point. A preset region growing algorithm is executed based on the target seed point to obtain a single-chip region set.

7. The Mini-LED rework method based on automated optical inspection as described in claim 6, characterized in that, The step of identifying the chip region group to be repaired in the defective image region group based on the single chip region set includes: performing the following operations on each defective image region in the defective image region group: determining the defective chip region group contained in the defective image region within the single chip region set, wherein the defective chip region group includes multiple defective chip regions, and the defective chip regions are within the defective image region; sequentially extracting the defective chip regions in the defective chip region group; identifying the light-emitting region and electrode region in the defective chip region according to a preset binarization method; obtaining an electrode region template, comparing the electrode region with the electrode region template to obtain the electrode template matching degree; if the electrode template matching degree is not greater than a preset standard matching degree, then the defective chip region is recorded as the chip region to be repaired; if the electrode template matching degree is greater than the standard matching degree, then performing long strip connected component identification on the light-emitting region to obtain the light-emitting region connected component, and determining the connected component length of the light-emitting region connected component; if the connected component length is greater than a preset standard scratch length, then the defective chip region is recorded as the chip region to be repaired; summarizing the chip regions to be repaired to obtain the chip region group to be repaired.

8. An apparatus for using the Mini-LED rework method based on automated optical inspection as described in any one of claims 1 to 7, characterized in that, The device includes: a detection command receiving module, used to receive optical detection commands, determine a set of products to be reworked based on the optical detection commands, wherein the set of products to be reworked includes multiple products to be reworked, and the products to be reworked are Mini-LED products, and each product to be reworked includes multiple Mini-LED chips; sequentially extracting products to be reworked from the set of products to be reworked, placing the products to be reworked in a preset background stage to obtain products to be inspected, wherein the background stage includes: a stage, a high-definition camera, and a light source; and a chip image capturing module, used to capture images of the products to be inspected using the high-definition camera and the light source to obtain optical chip images, performing Gaussian filtering on the optical chip images to obtain filtered chip images, acquiring an original chip image set, and training a pre-acquired neural network based on the original chip image set to obtain a defect region detection model, wherein the original... The chip image set includes multiple original chip images, which are either defective or normal chip images. The filtered image enhancement module detects the filtered chip images using a defect region detection model to obtain a defect image region group. This defect image region group may include multiple defect image regions or be an empty set. If the defect image region group is not empty, the product to be repaired is recorded as a repairable product. Local contrast enhancement is performed on the filtered chip images to obtain the target chip image. The repair report generation module identifies individual chips in the target chip image to obtain a single-chip region set. Based on the single-chip region set, it identifies the chip region group to be repaired within the defect image region group. Based on the chip region group to be repaired, it generates a repair opinion report for the repairable product. The repair opinion reports are then summarized to obtain multiple repair opinion reports.

Images