A deep learning-based LED chip defect detection method and system

Abstract

The application discloses a kind of LED chip defect detection method and system based on deep learning, belong to artificial intelligence detection technical field. Including: using specific wavelength coaxial light illumination and telecentric lens acquisition LED chip image to be measured;The image to be measured is input into the generator module trained by confrontation, and good product image is reconstructed, and the generator encoding path includes substrate texture suppression operator to eliminate periodic texture interference;By the pixel-by-pixel comparison of the image to be measured and the reconstructed good product image, a structure residual image is generated, an adaptive segmentation threshold is calculated based on the local sub-block mean and standard deviation, and the abnormal area is extracted by binary segmentation, and the threshold multiple coefficient is dynamically corrected according to the batch yield;In continuous detection, the average reconstruction loss of continuous chips is calculated in real time and subjected to time-frequency domain analysis, and the light source attenuation or lens defocusing early warning signal is identified.The application has high robustness, low false alarm rate and real-time self-diagnosis capability.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence detection technology, specifically relating to a method and system for detecting defects in LED chips based on deep learning. Background Technology

[0002] As a core component of semiconductor lighting, the surface quality of LED chips directly affects the luminous efficacy and lifespan of end products. In mass production, automated optical inspection based on machine vision has gradually replaced manual visual inspection to identify defects such as chipping, scratches, and electrode contamination on the chip surface. In recent years, deep learning technology has been introduced into this field, automatically extracting image features by constructing convolutional neural networks, which has improved inspection efficiency and consistency to some extent.

[0003] However, existing supervised learning-based defect detection methods have inherent limitations. First, these methods rely on a large number of labeled defect samples for model training. However, in LED chip manufacturing, defect types are diverse and their forms are random, and good products make up the vast majority, making defect sample collection and labeling extremely difficult and model training costly. Second, supervised models can only identify defect types that have appeared in the training set. They often cannot effectively detect unseen defects caused by fluctuations in the production process or new anomalies, resulting in insufficient generalization ability.

[0004] Furthermore, LED chip surfaces commonly exhibit periodic micro-textures, such as patterned sapphire substrates, which form regular background patterns in images. Traditional detection methods, when processing such textures, easily misjudge normal texture fluctuations as defects, resulting in numerous false alarms. Simultaneously, factors such as changes in lighting on actual production lines, slight lens defocusing, or light source attenuation further interfere with detection stability. Existing methods lack adaptive adjustment mechanisms and online diagnostic capabilities for the system's own state, making it difficult to maintain high-precision detection over long periods.

[0005] Therefore, there is an urgent need for an LED chip defect detection solution that can break free from dependence on defect samples, effectively suppress substrate texture interference, and possess environmental adaptability and self-diagnosis capabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for detecting defects in LED chips based on deep learning, which can effectively solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep learning-based method for detecting defects in LED chips includes the following specific steps: A coaxial illumination device with a wavelength range of 450 to 470 nanometers blue light or 520 to 535 nanometers green light, and a telecentric lens with a magnification of 1 to 5 times are used to acquire the image of the LED chip under test. The image to be tested is input into a trained generator module to generate a reconstructed good image. The generator module adopts a symmetrical encoding and decoding path architecture. The encoding path consists of multiple convolutional units with a stride greater than 1 stacked together without pooling layers, and includes a substrate texture suppression operator based on an attention mechanism to suppress periodic texture interference from the patterned sapphire substrate during compression into latent space vectors. The decoding path consists of symmetrical transposed convolutional units, and there are skip connections between corresponding layers of the encoding and decoding paths. During the training phase, the generator module is adversarially trained with a fully convolutional Markov discriminator that outputs a local receptive field probability matrix. A structural residual map is generated by comparing the image to be tested with the reconstructed good product image pixel by pixel. The local adaptive segmentation threshold is calculated based on the statistical mean and standard deviation of each local sub-block in the image, and the structural residual map is binarized to extract abnormal regions. The multiplier k used in the calculation of the local adaptive segmentation threshold is dynamically corrected according to the deviation between the current production batch yield and the preset target yield. During continuous detection, the average reconstruction loss of a preset number of consecutive chips is calculated in real time, and the reconstruction loss signal is analyzed in the time and frequency domains. When the frequency domain distribution characteristics match the first offset mode, a light source attenuation warning signal is generated, and when they match the second offset mode, a lens defocus warning signal is generated.

[0008] Furthermore, the substrate texture suppression operator is specifically implemented as follows: at the intermediate level of the encoding path, the channel descriptor of the feature map is obtained through a global average pooling branch, and after being processed by a gating mechanism, an attention weight vector is generated. The attention weight vector is then weighted with the original feature map in the channel dimension to enhance or suppress the feature response of a specific channel, thereby achieving background suppression of the periodic texture of the patterned sapphire substrate.

[0009] Furthermore, the encoding path of the generator module consists of 5 layers of convolutional units stacked sequentially, with the stride of each convolutional unit set to 2, so as to halve the spatial resolution of the feature map after each convolution operation. The number of convolutional kernels in each layer of the encoding path increases progressively, with configurations of 64, 128, 256, 512, and 1024 kernels respectively. The substrate texture suppression operator is set after the convolutional units with 256 or 512 kernels in the encoding path.

[0010] Furthermore, the decoding path consists of 5 layers of transposed convolutional units, with the convolution stride of each transposed convolutional unit set to 2, so that the spatial resolution of the feature map is restored to the original size of the image to be tested layer by layer. After the transpose convolution operation of the i-th layer is completed, the feature map output of the (6-i)-th layer in the encoding path, which has been processed by the substrate texture suppression operator, is concatenated and spliced ​​with the feature map of the current decoding layer in the channel dimension to achieve skip connection, where i is an integer from 1 to 5.

[0011] Furthermore, the adversarial training process between the generator module and the Markov discriminator is driven by a composite loss function, which includes a reconstruction loss term, an adversarial loss term, and a gradient penalty term. The reconstruction loss term is a weighted sum of the mean squared error loss and the structural similarity loss, and its calculation formula is as follows: ,in, For mean square error loss, For structural similarity loss, As the first preset weight, As the second preset weight, and + =1; where the second preset weight The value of is between 0.3 and 0.4 to enhance the constraint of structural similarity loss in filtering pseudo-defects caused by periodic textures in patterned sapphire substrates; the adversarial loss term is calculated based on the M×M probability matrix output by the Markov discriminator, where M is 16 or 30. The gradient penalty term is applied to the Markov discriminator to constrain the gradient norm of random interpolated samples between the real good image and the reconstructed good image.

[0012] Furthermore, the step of calculating a local adaptive segmentation threshold based on the statistical mean and standard deviation of each local sub-block in the structural residual map specifically includes: The structural residual map is divided into multiple 32×32 pixel sub-blocks, and the local mean of each sub-block is calculated separately. With local standard deviation , and according to the formula = +K Calculate the local adaptive segmentation threshold corresponding to this sub-block. ; The baseline value of the predetermined multiplier coefficient k is between 3 and 6, and the dynamic correction is based on the following rules: when the yield of the current production batch is more than 5 percentage points lower than the preset target yield, the value of k is reduced to expand the defect detection range and reduce the missed detection rate; when the yield of the current production batch is higher than the preset target yield and the false alarm rate exceeds the preset threshold, the value of k is increased to tighten the defect judgment standard and reduce the false alarm rate.

[0013] Furthermore, the defect location steps also include: The binarized image is subjected to morphological post-processing consisting of opening and closing operations; the structuring element of the opening operation is a 3×3 pixel rectangular kernel, and the structuring element of the closing operation is a 5×5 pixel rectangular kernel. Connectivity analysis is performed on the morphologically post-processed binary image to extract each independent connected component, and morphological feature parameters of the independent connected components are calculated. The morphological feature parameters include at least one of area, major axis length, minor axis length, perimeter, and compactness.

[0014] Furthermore, by optimizing the scheduling and memory reuse strategies of the neural network computation graph on the graphics processor, the processing time of a single image's complete process, including the reconstruction step, residual generation, and defect localization steps, is controlled within the range of 30 to 80 milliseconds. The optimizations include: fusing consecutive convolutional layers, batch normalization layers, and activation function layers in the generator module into a single computational kernel, and reusing feature map memory in-situ during forward propagation.

[0015] Furthermore, in the self-diagnosis monitoring step, the time-frequency domain analysis of the reconstructed loss signal specifically involves performing a short-time Fourier transform or wavelet transform on the time series composed of the reconstructed loss values ​​in the continuous detection process to extract the energy distribution of different frequency components as frequency domain distribution features. The pre-calibrated frequency domain feature library includes: the low-frequency energy continuously rising feature corresponding to the light source attenuation mode, and the high-frequency energy attenuation and mid-frequency component broadening feature corresponding to the lens defocus mode.

[0016] A deep learning-based LED chip defect detection system includes: The image acquisition module is equipped with an industrial charge-coupled device camera with a pixel count between 12 million and 25 million, a telecentric lens with a magnification between 1x and 5x, and a coaxial illumination device that emits blue light in the wavelength range of 450 nm to 470 nm or green light in the wavelength range of 520 nm to 535 nm, for acquiring images of the LED chip under test. The magnification of the telecentric lens and the pixel count of the industrial charge-coupled device camera are configured together to meet the requirements for analyzing chip surface defects such as chipping or scratches with dimensions at the micrometer level. The good image reconstruction module has a built-in trained generator module, which receives the test image and generates the corresponding reconstructed good image. The generator module has a symmetrical encoding and decoding architecture. Its encoding path uses convolutional layers with a stride of 2 instead of pooling layers to achieve downsampling, and includes a substrate texture suppression operator based on an attention mechanism. There are skip connections between corresponding layers of the encoding and decoding paths. During the training phase, the generator module is adversarially trained with a Markov discriminator that outputs a probability matrix. The residual generation module is used to perform pixel-by-pixel comparison between the image to be tested and the reconstructed good image to generate a structural residual map. The adaptive threshold segmentation module is used to divide the structural residual map into multiple sub-blocks of preset size, and calculate the local adaptive segmentation threshold based on the local mean and local standard deviation of each sub-block; the multiplier k of the segmentation threshold is dynamically adjusted according to the yield of the current production batch. The morphological processing and feature extraction module is used to perform opening and closing operations on the binarized segmentation results to optimize connectivity, and to perform quantitative analysis of morphological feature parameters on the extracted connected components. The results are output to the database module, which is used to visually mark defect areas on the original image to be tested and write defect information into the production quality database. The self-diagnostic monitoring module is used to calculate the average reconstruction loss of a preset number of continuous LED chips in real time during continuous detection, and to identify the frequency domain offset mode of the average reconstruction loss signal through time-frequency domain analysis in order to distinguish between generating a light source attenuation warning signal or a lens defocus warning signal.

[0017] In summary, this application includes at least one of the following beneficial technical effects: 1. This invention constructs a generative adversarial network architecture based on unsupervised learning, which only requires collecting images of readily available good LED chips from the production line to complete model training. This completely solves the problem of traditional supervised learning detection methods relying on large-scale defect sample annotation, and greatly reduces the manual cost and time cycle of data preparation.

[0018] 2. This invention adopts the principle of reconstruction failure detection. The generator model only learns and masters the feature distribution of good chips. For unknown defect forms generated by new processes that have never appeared in the training set, the model will show extremely high reconstruction error. Thus, through residual analysis, all non-good defect abnormalities can be reliably intercepted, effectively avoiding the problem of missed detection of new types of defects that traditional methods are prone to.

[0019] 3. This invention achieves accurate preservation of multi-scale spatial information, ensuring the texture delicacy of reconstructed high-quality images; at the same time, the introduction of structural similarity loss in the composite loss function makes the detection algorithm exhibit strong robustness to ambient light fluctuations, and with the adaptive dynamic threshold segmentation technology, the false alarm rate is controlled at an extremely low level, ensuring the absolute accuracy of chip quality judgment.

[0020] 4. This invention can achieve automatic defect location and morphological parameter extraction in a very short time. Combined with the automatic docking of the production quality database, it provides timely and accurate data feedback for chip manufacturing companies to optimize their processes, and significantly improves the overall yield and reliability of semiconductor lighting products. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall technical solution for a deep learning-based LED chip defect detection method. Figure 2 This is a schematic diagram illustrating the core principles of a symmetric encoding / decoding architecture and skip connections. Figure 3 A flowchart illustrating the logical process of learning the distribution of good product features and training the model. Figure 4 A schematic diagram illustrating the interaction and data flow between image reconstruction and multi-dimensional structural residual generation; Figure 5 This is a flowchart illustrating the logic for precise localization of abnormal regions based on statistical adaptive thresholding and morphological processing. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 5 The present invention will be further described in detail below with reference to specific embodiments.

[0023] The first aspect concerns the LED chip defect detection method based on deep learning disclosed in this application, which is implemented according to the following steps: First, in step S1, a high-quality benchmark database containing only the surface morphology of qualified LED chips is established. This database provides a data source for the subsequent generative adversarial network to understand and learn the feature distribution of normal chips. The specific implementation process can be broken down into the following sequential sub-steps.

[0024] Step S101: Configure the image acquisition hardware system and select an industrial charge-coupled device camera with a pixel scale between 12 million and 25 million. This resolution range can meet the resolution requirements of micron-level texture details and potential anomalies on the LED chip surface.

[0025] The camera is equipped with a telecentric lens with a magnification between 1x and 5x, the specific magnification being determined based on the actual size of the chip under test. The telecentric lens is used to eliminate perspective distortion, ensuring that the geometric dimensions in the chip image maintain a strictly linear relationship with its physical dimensions. This relationship provides a metrological basis for subsequent accurate quantification of chip size or scratch length.

[0026] The imaging light source employs a coaxial illumination device with a predetermined wavelength range. The light source wavelength is set to the blue light band of 450 nm to 470 nm, or the green light band of 520 nm to 535 nm. The short-wavelength light source exhibits differentiated scattering characteristics on the surfaces of metal electrodes and semiconductor epitaxial layers, which can enhance the reflection contrast of fine features such as chipping, scratches, and electrode contamination on the chip surface, thereby increasing the initial grayscale difference between the background area and the potential defect area.

[0027] Step S102 involves preprocessing the acquired raw image by applying a median filter algorithm with a window size of 3×3 or 5×5 pixels to smooth the image. This algorithm sorts the pixel values ​​within the window and selects the median to replace the center pixel, effectively filtering out salt-and-pepper noise caused by electromagnetic interference in industrial environments.

[0028] The nonlinear characteristics of median filtering allow it to retain the sharpness of key features such as chip edges and electrode contours while denoising, avoiding edge blurring that may be introduced by Gaussian filtering.

[0029] Subsequently, a linear stretching algorithm is used to adjust the dynamic range of the image. The gray values ​​corresponding to the 10-bit or 12-bit sampling depth of the original image are linearly mapped to a standard 8-bit grayscale space of 0 to 255, or normalized to a floating-point range of 0.0 to 1.0. This step is used to standardize the overall contrast of the image, making the texture distribution in the good-quality areas more consistent, thereby stabilizing the model's judgment benchmark for normal features.

[0030] Step S103: Expand the training sample set by sliding window slicing and data augmentation strategies. Perform sliding window slicing sampling on the large-size original chip image. The window size is set to 256×256 or 512×512 pixels according to the scale of the generative adversarial network input layer. The overlap ratio between adjacent windows is set between 20% and 50%.

[0031] This redundant sampling method covers the chip edges and corners with multiple slice sub-images, providing the model with more complete contextual information during the training phase, while increasing the number of effective training samples.

[0032] Data augmentation operations are performed on each obtained good product slice sub-image, including random rotations of 0, 90, 180, and 270 degrees, as well as random flips in the horizontal and vertical directions. These geometric transformations simulate the actual placement posture of LED chips in carriers such as tapes, blue films, or trays, enabling the subsequent model to acquire rotational and translational invariance during the learning process, ensuring that the detection algorithm is not affected by the material placement angle during actual operation.

[0033] Step S104: Complete the construction of the training set and verify the data characteristics. All good-quality image sub-images after preprocessing, slicing, and enhancement are compiled to form the image training set for unsupervised learning. After construction, statistical analysis can be performed on the pixel mean and variance distribution of the dataset to confirm the consistency level of each sub-image.

[0034] In summary, the training set obtained in step S1 is characterized by containing no manually labeled defective samples. This dataset serves as the sole input to the generative adversarial network in subsequent steps S2 and S3. The training objective is to enable the generator module in the network to learn and reproduce the texture distribution, electrode structure, and background features of good chips, while simultaneously enabling the discriminator module to distinguish between genuine good chip images and generator-forged good chip images.

[0035] Based on the model's deep understanding of normal patterns, during the detection phase, the system can effectively identify various defect morphologies that were not trained by analyzing the residuals between the image to be tested and the reconstructed image.

[0036] For step S2, a generative adversarial network (GAN) architecture is constructed and adversarial training is performed: The GAN constructed in step S2 is the core component for implementing the reconstruction failure detection principle, consisting of a generator module and a discriminator module. The generator module is responsible for learning and reproducing the texture distribution and structural features of the good chip images, while the discriminator module evaluates the authenticity of the generated images. Both modules iteratively optimize during adversarial training. The training data comes from the good chip image training set constructed in step S1. The specific construction and training methods are explained in the following sub-steps.

[0037] Step S201: Construct the encoding path for the generator module. The generator module adopts a symmetrical encoding and decoding architecture. The encoding path consists of 5 layers of convolutional units stacked sequentially. Each convolutional unit contains a convolutional layer, a batch normalization layer, and a modified linear unit activation function layer. The kernel size of each convolutional layer is fixed at 3×3 or 4×4, and the stride of each convolution is set to 2.

[0038] This step size configuration halves the spatial resolution of the feature map layer by layer, achieving downsampling without using pooling layers. This avoids the information loss that may be caused by max pooling or mean pooling, and preserves the integrity of spatial features to the maximum extent.

[0039] The number of convolutional kernels in each layer increases progressively according to a preset rule. Specifically, the first layer has 64 kernels, the second layer has 128, the third layer has 256, the fourth layer has 512, and the fifth layer has 1024. As the network depth increases, the number of feature map channels increases while the spatial size gradually decreases, ultimately compressing the input image into a low-dimensional latent space vector. This vector carries the high-dimensional semantic features in the input image related to the structure of the good chip.

[0040] The batch normalization layer configured after each convolutional layer is used to normalize the distribution of feature maps, which helps to accelerate network convergence and suppress the gradient vanishing problem during training.

[0041] Step S202: Construct the decoding path of the generator module. The decoding path consists of 5 layers of transposed convolutional units, and their layer order is mirror-symmetric to that of the encoding path. The stride of each transposed convolutional layer is also set to 2, so that the spatial resolution of the feature map is restored to the original input image size layer by layer.

[0042] The transposed convolution kernel size is consistent with the corresponding coding layer convolution kernel size. The number of transposed convolution kernels in each layer is set as follows: 512 for layer 5, 256 for layer 4, 128 for layer 3, 64 for layer 2, and layer 1 outputs the same number of channels as the input image.

[0043] A skip connection mechanism is introduced between corresponding layers of the encoding and decoding paths. Specifically, after the transposed convolution operation at layer i is completed, the feature map output from layer 6-i in the encoding path is concatenated with the feature map of the current decoding layer along the channel dimension, where i takes values ​​from 1 to 5.

[0044] Through this connection method, the shallow positional information and edge contour information retained in the encoding stage can be directly transmitted to the corresponding level in the decoding stage, effectively compensating for the loss of spatial details in the deep network, so that the reconstructed image finally output by the generator is closer to the real good chip image in terms of texture fineness and structural integrity.

[0045] Step S203: Construct the discriminator module, which adopts a Markov discriminator design based on a fully convolutional architecture. This discriminator does not output a single scalar probability value to determine the authenticity of the entire image, but instead outputs a probability matrix of a predetermined dimension, which can be set to 16×16 or 30×30.

[0046] Each element in the matrix represents the probability estimate that the corresponding local receptive field region in the image under test belongs to the true good product distribution.

[0047] This local discrimination mechanism imposes stronger constraints on the generator, requiring it to produce image content that conforms to the characteristics of genuine high-quality products at both the global statistical distribution level and the local texture detail level. Because the discriminator independently judges the authenticity of each local region in the image, the generator cannot evade the discriminator's identification simply by mimicking the overall brightness and color distribution. This forces the generator to meticulously restore the texture features of each subtle local area during reconstruction, significantly improving the local fidelity of the reconstructed image.

[0048] Step S204: Perform adversarial training between the generator and the discriminator. The training process employs an alternating update strategy. In each iteration, the generator parameters are fixed first to update the discriminator, and then the discriminator parameters are fixed to update the generator. The training objective of the discriminator is: for real good images from the training set in step S1, the values ​​of each element in its output probability matrix should approach 1; for images reconstructed by the generator, the values ​​of each element in its output probability matrix should approach 0. The training objective of the generator is: to make the values ​​of each element in the probability matrix output by the discriminator for the reconstructed image approach 1.

[0049] The loss function used in training is a composite loss function, the specific composition and weight allocation of which will be explained in step S3. The optimizer uses the adaptive moment estimation algorithm, with an initial learning rate set to 0.0002, a first-order momentum decay coefficient set to 0.5, and a second-order momentum decay coefficient set to 0.999. The total number of training epochs is set to 200 to 500. After half of the total number of training epochs has been completed, the learning rate decreases sequentially using a linear decay strategy until training ends.

[0050] In summary, through step S2, the generative adversarial network completes its architecture construction and learns the feature distribution of the good chip image through adversarial training. During this process, the generator gradually masters the ability to map any input image to the manifold of the good chip image, while the discriminator simultaneously improves its accuracy in distinguishing between real good chip images and generated images.

[0051] The generator module, once trained, will be used for image reconstruction in step S4, and the reconstruction quality directly determines the accuracy of residual analysis in locating defect areas in step S5.

[0052] Next, in step S3, a composite loss function is defined and generator training is performed. Based on the network architecture established in step S2, step S3 defines the composite loss function used in the training process to drive the generator to learn the feature distribution of the good chip. The composite loss function consists of a weighted sum of reconstruction loss and adversarial loss, and a gradient penalty term is introduced to stabilize training. Specifically, it includes the following implementation sub-steps: Step S301: Construct the reconstruction loss term. The reconstruction loss term is used to constrain the consistency between the generator's output image and the input image, and includes two parts: mean squared error loss and structural similarity loss. Mean squared error loss Calculate using the following formula: Where I is the input image, G is the generator-reconstructed image, N is the total number of pixels, and j is the pixel index. This loss term ensures that the reconstructed image is consistent with the original good image in terms of brightness and color.

[0053] Structural similarity loss Image similarity is evaluated based on three dimensions: brightness, contrast, and structure, and is defined as follows: In the formula, K is the number of local windows in the whole image. The structural similarity index value for the k-th window is calculated using a method known in the art. This loss enhances the generator's robustness to illumination fluctuations, stably maintaining the electrode contour and texture structure.

[0054] Reconstructing the loss term The weighted sum of the above two items is as follows: in The first preset weight is set to a value between 0.6 and 0.8. The second preset weight takes a value between 0.2 and 0.4, and + =1.

[0055] Step S302, construct the adversarial loss term. To enhance the realism of local textures in the reconstructed image, a least-squares approach is used, calculated based on the M×M probability matrix output by the discriminator: In the formula M represents the value of the element in the u-th row and v-th column of the probability matrix output by the discriminator for the reconstructed image, where M is 16 or 30.

[0056] Step S303: Introduce a gradient penalty term. To prevent mode collapse during training, a gradient penalty term is added to the discriminator loss. In the formula, X represents a random interpolation sample between the real image I and the reconstructed image G. It is a random number. ; The output of the discriminator for X; This is the penalty coefficient, with a value of 10. This term constrains the discriminator gradient norm, making the training process more stable.

[0057] Step S304: Define the total loss and complete training; generator total loss. ,in The third preset weight, ranging from 0.01 to 0.1, is used to balance reconstruction accuracy and texture realism. (Discriminator total loss) Loss by adversarial discrimination and The summation results in an adversarial discriminative loss that penalizes the degree to which the output deviates from 1 for the real image and from 0 for the reconstructed image.

[0058] The training data comes from the good-quality image training set constructed in step S1. The optimizer uses the adaptive moment estimation algorithm with an initial learning rate of 0.0002 and momentum coefficients of 0.5 and 0.999. The total number of training rounds is 200 to 500, and the learning rate decays linearly after half of the training is completed. Each round of training is performed using an alternating strategy of updating the discriminator first and then updating the generator.

[0059] In summary, through the loss function configuration and training process in step S3, the generator gradually learns the feature distribution of the good chips. After training convergence, the generator achieves high-fidelity reconstruction of the good regions, while the defective regions exhibit significant reconstruction errors due to deviations from the learned distribution, providing a basis for residual analysis and defect localization in step S5. The trained generator is directly used for the reconstruction of the test image in step S4.

[0060] Next, in step S4, the image under test is reconstructed and a good product image is generated. This step applies the generator module trained in step S3 to the actual inspection process, performing a reconstruction operation on the image of the LED chip under test to generate a corresponding good product reference image. The reconstruction process follows the distribution law of good product features learned by the generator, providing a comparison benchmark for subsequent residual analysis.

[0061] Step S401: Image Preprocessing and Normalization. After acquiring the image of the LED chip to be tested, the same preprocessing procedure as in Step S1 (training set construction) is first executed. A median filter algorithm with a window size of 3×3 or 5×5 pixels is applied to the original image to filter out salt-and-pepper noise caused by electromagnetic interference in the industrial environment. Median filtering preserves the sharpness of the chip edges and electrode contours while removing noise.

[0062] Subsequently, a linear stretching algorithm is used to adjust the dynamic range of the image. If the original image has a 10-bit or 12-bit sampling depth, its grayscale values ​​are linearly mapped to a standard 8-bit grayscale space of 0 to 255; if the original image has been normalized, it is scaled to a floating-point range of -1 to 1 or 0 to 1. The normalization target range is strictly consistent with the range used in the training phase of step S1. This preprocessing operation ensures that the data distribution of the image under test and the data distribution of the training set images are under the same dimensions and statistical characteristics, eliminating reconstruction bias caused by differences in the scale of the input data.

[0063] In step S402, the image to be tested is input into the generator for reconstruction. The preprocessed image to be tested is input into the generator module jointly trained in steps S2 and S3. After receiving the image to be tested, the generator compresses the input image to a low-dimensional latent space vector through its encoding path. Then, the decoding path combines the shallow position information and edge information passed by skip connections to gradually restore the spatial resolution of the image. Finally, it outputs a reconstructed good image with the same dimensions and number of channels as the image to be tested.

[0064] If a single-channel grayscale image is used as input during the training phase, the reconstructed output will also be a single-channel grayscale image; if a three-channel color image is used during the training phase, the reconstructed output will also be a color image with the corresponding number of channels.

[0065] Step S403: Differentiated Reconstruction Performance. During the training phase, the generator only interacts with images of good-quality chips. Its network parameters have converged to a distribution state that can accurately reproduce the texture, electrode structure, and background features of the good-quality region. When the image to be tested contains a good-quality region, the features of that region highly match the distribution learned by the generator. The generator outputs an image content that is almost identical to the original input, achieving high-fidelity reconstruction.

[0066] When defective regions exist in the image under test, such as scratches, chipped edges, electrode contamination, or epitaxial layer anomalies, the visual features of these regions have not appeared in the training set constructed in step S1, exceeding the range of good product feature distribution already known to the generator. When processing such abnormal regions, the generator replaces the abnormal pixel pattern with the closest good product pattern based on the statistical regularity of its internal parameterization.

[0067] Specifically, this manifests as follows: the scratched area is reconstructed into a smooth background texture, the chipped edge area is restored to a complete chip edge outline, and the electrode contamination area is restored to a normal electrode morphology. As a result, a significant pixel-level difference occurs between the image under test and the reconstructed good image at the corresponding defect locations. This difference constitutes the signal source for the residual analysis in step S5.

[0068] In summary, step S4 yields the image to be tested and the corresponding reconstructed good image, which together form the input pair for residual analysis. The reconstructed good image provides a reference benchmark for the normal morphology that the location should have under defect-free conditions, while the image to be tested reflects the actual imaging result. Subsequent step S5 calculates the structural residual based on the difference between the two images and uses adaptive thresholding and morphological processing to achieve quantitative localization and feature extraction of the defect region.

[0069] Finally, in step S5, the structural residual is calculated and the abnormal region is extracted using an adaptive threshold. This step, based on the obtained image to be tested and the reconstructed good image, generates a structural residual map through pixel-by-pixel comparison, and uses statistical adaptive thresholding and morphological post-processing techniques to achieve precise localization and feature quantification of the defect region. The specific implementation method is described below through sub-steps.

[0070] Step S501, residual image preprocessing and Gaussian smoothing: To eliminate the interference of sub-pixel-level image shifts caused by slight fluctuations in industrial lighting or mechanical vibrations on the residual calculation, a small amount of Gaussian smoothing is first performed on the reconstructed good product image output in step S4. The standard deviation of the Gaussian smoothing kernel is set between 0.5 and 1.0. This smoothing amplitude can filter out high-frequency noise and registration errors caused by small displacements, while not significantly blurring the boundary gradient between defective and good product areas in the residual image.

[0071] Step S502: Generate a structural residual map. The image to be tested is compared pixel-by-pixel with the reconstructed good image after Gaussian smoothing. The difference in pixel values ​​at corresponding coordinate positions between the two images is calculated to generate the structural residual map. The difference is measured using the absolute value of the pixel value difference. For a single-channel grayscale image, the formula for calculating the residual value R at coordinate positions x,y is:

[0072] Where I represents the image to be tested, G represents the reconstructed image, and x and y are the coordinate indices of the pixels in the width and height directions of the image, respectively. For multi-channel color images, the Euclidean distance between corresponding pixels in each channel of the two images can be calculated as the residual value. The generated residual map has the same dimension as the image to be tested; a higher residual value indicates a greater deviation from the expected reconstructed image at that location.

[0073] Step S503: An adaptive segmentation threshold is calculated based on statistical parameters. To accurately separate defect areas from the normal fluctuation background in the residual map, an adaptive dynamic thresholding technique is employed. The threshold T is calculated based on the statistical distribution characteristics of the residual map, using the following formula: in The arithmetic mean of the pixel values ​​in the residual image. is the standard deviation of pixel values ​​in the residual map, and k is a predetermined multiplier. The multiplier k ranges from 3 to 6, and the specific value can be adjusted according to the production line's need to balance the false negative rate and false positive rate. When k is 3, the threshold covers approximately 99.7% of the normal distribution's normal fluctuation range, and pixels exceeding this threshold are judged as candidate defect points.

[0074] To address the uneven residual background caused by the morphological difference between the thinned edge region and the flat central region on the chip surface, a local window sliding method can be used for threshold calculation. The residual map is divided into 32×32 pixel sub-blocks, and the local mean is calculated for each sub-block. With local standard deviation The local threshold corresponding to the sub-block is obtained by substituting the local statistics into the threshold formula above. This local adaptive strategy can improve the detection sensitivity of subtle defects with low contrast.

[0075] Step S504: Binarization segmentation and morphological post-processing. Based on the threshold determined in step S503, the residual image is binarized. Pixels with residual values ​​greater than the threshold are marked as foreground candidate defect points, and the rest are marked as background. Morphological post-processing is performed on the binarization result to optimize the connectivity of the defect region.

[0076] First, an opening operation is applied to remove isolated noise points. The opening operation consists of erosion followed by dilation. The erosion operation eliminates scattered pixels caused by minor reconstruction errors, while the dilation operation restores the size of the eroded main area. The structuring element size for both erosion and dilation can be a 3×3 pixel rectangular kernel.

[0077] The broken defect regions are then connected using a closing operation. The closing operation consists of dilation followed by erosion. The dilation operation merges adjacent candidate defect points, connecting continuous defects such as scratches or chipped edges into complete connected regions. A 5×5 pixel rectangular kernel can be used as the structuring element size.

[0078] Step S505, Connected Component Analysis and Morphological Feature Extraction: Perform connected component analysis on the morphologically processed binary image to extract each independent connected region. Calculate the morphological feature parameters of each connected region, including area (total number of pixels), major axis length (major axis of the smallest circumscribed ellipse), minor axis length (minor axis of the smallest circumscribed ellipse), perimeter (number of boundary pixels), and compactness (ratio of area to the square of perimeter).

[0079] By setting an area threshold, minute fluctuations that have no engineering significance are filtered out. The area threshold can be set according to the chip size and defect detection accuracy requirements, for example, filtering connected regions with an area smaller than 5 pixels. The retained connected regions are the finally confirmed defect areas.

[0080] Step S506 involves defect localization, visualization, and data recording. The coordinates of the connected components of the defects extracted in step S505 are mapped back from the slice sub-image coordinate system to the original large-size chip image coordinate system. The location and extent of each defect region are marked on the original image using red or yellow rectangular bounding boxes, achieving visual labeling.

[0081] Simultaneously, defect information is written into the production quality database, recording information including defect center coordinates, defect type preliminarily determined based on morphological characteristics, defect area, major axis length, minor axis length, and severity assessment value. This database provides quantitative feedback for subsequent parameter adjustments and yield analysis in chip manufacturing processes.

[0082] In summary, step S5 transforms the pixel-level differences resulting from reconstruction failures into quantitatively describable defect detection results through residual calculation, adaptive threshold segmentation, and morphological post-processing. This step is directly connected to the reconstructed image generation stage in step S4, together forming a complete detection chain from image input to defect output.

[0083] In this embodiment, hardware acceleration is achieved using a graphics processing unit (GPU) with multiple computational cores. This is achieved by optimizing the scheduling order and memory reuse strategy of the neural network computation graph on the GPU. Specifically, this includes merging consecutive convolutional layers, batch normalization layers, and activation function layers in the generator encoding and decoding paths into a single computational kernel to reduce kernel startup overhead, and reusing feature map memory in-situ during forward propagation, retaining only the intermediate gradient tensors required for backpropagation.

[0084] After the above optimization, the time taken for the entire process of a single LED chip image from input preprocessing, generator reconstruction to residual calculation and defect extraction is controlled within the range of 30 to 80 milliseconds, which meets the real-time requirement of the production line to inspect thousands of chips per minute.

[0085] In addition, this method also includes a real-time monitoring step. This step continuously calculates the average reconstruction loss of a preset number of chips during the detection process. The reconstruction loss can be the mean square error loss defined in step S301. Or generator total loss The default quantity is usually 100 pieces.

[0086] The average reconstruction loss of consecutive chips of the same quantity recorded during the initial stable operation of the production line is used as a preset benchmark value. When the average reconstruction loss calculated in real time subsequently continues to rise above this benchmark value and the increase exceeds 20%, the system generates an early warning signal, prompting operators to check whether the camera focus is loose, whether the light source brightness is attenuated, or whether there is contamination in the optical path, thus realizing the online self-diagnosis function of the detection system.

[0087] On the other hand, the LED chip defect detection system based on deep learning disclosed in this application includes: The image acquisition module is equipped with an industrial charge-coupled device camera with a pixel count between 12 million and 25 million, a telecentric lens with a magnification between 1x and 5x, and a coaxial illumination device that emits blue light in the wavelength range of 450 nm to 470 nm or green light in the wavelength range of 520 nm to 535 nm, for acquiring images of the LED chip under test. The magnification of the telecentric lens and the pixel count of the industrial charge-coupled device camera are configured together to meet the requirements for analyzing chip surface defects such as chipping or scratches with dimensions at the micrometer level. The good image reconstruction module has a built-in trained generator module, which receives the test image and generates the corresponding reconstructed good image. The generator module has a symmetrical encoding and decoding architecture. Its encoding path uses convolutional layers with a stride of 2 instead of pooling layers to achieve downsampling, and includes a substrate texture suppression operator based on an attention mechanism. There are skip connections between corresponding layers of the encoding and decoding paths. During the training phase, the generator module is adversarially trained with a Markov discriminator that outputs a probability matrix. The residual generation module is used to perform pixel-by-pixel comparison between the image to be tested and the reconstructed good image to generate a structural residual map. The adaptive threshold segmentation module is used to divide the structural residual map into multiple sub-blocks of preset size, and calculate the local adaptive segmentation threshold based on the local mean and local standard deviation of each sub-block; the multiplier k of the segmentation threshold is dynamically adjusted according to the yield of the current production batch. The morphological processing and feature extraction module is used to perform opening and closing operations on the binarized segmentation results to optimize connectivity, and to perform quantitative analysis of morphological feature parameters on the extracted connected components. The results are output to the database module, which is used to visually mark defect areas on the original image to be tested and write defect information into the production quality database. The self-diagnostic monitoring module is used to calculate the average reconstruction loss of a preset number of continuous LED chips in real time during continuous detection, and to identify the frequency domain offset mode of the average reconstruction loss signal through time-frequency domain analysis in order to distinguish between generating a light source attenuation warning signal or a lens defocus warning signal.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0089] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting defects in LED chips based on deep learning, characterized in that, Includes the following steps: A coaxial illumination device with a wavelength range of 450 to 470 nanometers blue light or 520 to 535 nanometers green light, and a telecentric lens with a magnification of 1 to 5 times are used to acquire the image of the LED chip under test. The image to be tested is input into a trained generator module to generate a reconstructed good image; The generator module employs a symmetrical encoding and decoding path architecture. The encoding path consists of multiple convolutional units with a stride greater than 1 stacked together without pooling layers, and includes an attention-based substrate texture suppression operator to suppress periodic texture interference from the patterned sapphire substrate during compression into latent space vectors. The decoding path consists of symmetrical transposed convolutional units, and skip connections are provided between corresponding layers of the encoding and decoding paths. During the training phase, the generator module undergoes adversarial training against a fully convolutional Markov discriminator that outputs a local receptive field probability matrix. A structural residual map is generated by comparing the image to be tested with the reconstructed good product image pixel by pixel. The local adaptive segmentation threshold is calculated based on the statistical mean and standard deviation of each local sub-block in the image, and the structural residual map is binarized to extract abnormal regions. The multiplier k used in the calculation of the local adaptive segmentation threshold is dynamically corrected according to the deviation between the current production batch yield and the preset target yield. During continuous detection, the average reconstruction loss of a preset number of consecutive chips is calculated in real time, and the reconstruction loss signal is analyzed in the time and frequency domains. When the frequency domain distribution characteristics match the first offset mode, a light source attenuation warning signal is generated, and when they match the second offset mode, a lens defocus warning signal is generated.

2. The LED chip defect detection method based on deep learning according to claim 1, characterized in that, The substrate texture suppression operator is specifically implemented as follows: at the intermediate level of the encoding path, the channel descriptor of the feature map is obtained through a global average pooling branch. After being processed by the gating mechanism, an attention weight vector is generated. This attention weight vector is then weighted with the original feature map in the channel dimension to enhance or suppress the feature response of a specific channel, thereby achieving background suppression of the periodic texture of the patterned sapphire substrate.

3. The LED chip defect detection method based on deep learning according to claim 1, characterized in that, The encoding path of the generator module consists of 5 layers of convolutional units stacked sequentially, with the stride of each convolutional unit set to 2, so as to halve the spatial resolution of the feature map after each convolution operation. The number of convolutional kernels in each layer of the encoding path increases progressively, with configurations of 64, 128, 256, 512, and 1024 kernels respectively. The substrate texture suppression operator is set after the convolutional units with 256 or 512 kernels in the encoding path.

4. The LED chip defect detection method based on deep learning according to claim 3, characterized in that, The decoding path consists of 5 layers of transposed convolutional units, with the stride of each transposed convolutional unit set to 2, so that the spatial resolution of the feature map is restored to the original size of the image to be tested layer by layer. After the transpose convolution operation of the i-th layer is completed, the feature map output of the (6-i)-th layer in the encoding path, which has been processed by the substrate texture suppression operator, is concatenated and spliced ​​with the feature map of the current decoding layer in the channel dimension to achieve skip connection, where i is an integer from 1 to 5.

5. The LED chip defect detection method based on deep learning according to claim 1, characterized in that, The adversarial training process between the generator module and the Markov discriminator is driven by a composite loss function, which includes a reconstruction loss term, an adversarial loss term, and a gradient penalty term. The reconstruction loss term is a weighted sum of the mean squared error loss and the structural similarity loss, and its calculation formula is as follows: ,in, For mean square error loss, For structural similarity loss, As the first preset weight, As the second preset weight, and + =1; where the second preset weight The value ranges from 0.3 to 0.4 to enhance the constraint of structural similarity loss in filtering pseudo-defects caused by periodic textures in patterned sapphire substrates; The adversarial loss term is calculated based on the M×M probability matrix output by the Markov discriminator, where M takes the value of 16 or 30. The gradient penalty term is applied to the Markov discriminator to constrain the gradient norm of random interpolated samples between the real good image and the reconstructed good image.

6. The LED chip defect detection method based on deep learning according to claim 1, characterized in that, The steps for calculating a local adaptive segmentation threshold based on the statistical mean and standard deviation of each local sub-block in the structural residual map specifically include: The structural residual map is divided into multiple 32×32 pixel sub-blocks, and the local mean of each sub-block is calculated separately. With local standard deviation , and according to the formula = +K Calculate the local adaptive segmentation threshold corresponding to this sub-block. ; The baseline value of the predetermined multiplier coefficient k is between 3 and 6, and the dynamic correction is based on the following rules: when the yield of the current production batch is more than 5 percentage points lower than the preset target yield, the value of k is reduced to expand the defect detection range and reduce the missed detection rate; when the yield of the current production batch is higher than the preset target yield and the false alarm rate exceeds the preset threshold, the value of k is increased to tighten the defect judgment standard and reduce the false alarm rate.

7. The LED chip defect detection method based on deep learning according to claim 6, characterized in that, The defect location steps also include: The binarized image is subjected to morphological post-processing consisting of opening and closing operations; the structuring element of the opening operation is a 3×3 pixel rectangular kernel, and the structuring element of the closing operation is a 5×5 pixel rectangular kernel. Connectivity analysis is performed on the morphologically post-processed binary image to extract each independent connected component, and morphological feature parameters of the independent connected components are calculated. The morphological feature parameters include at least one of area, major axis length, minor axis length, perimeter, and compactness.

8. The LED chip defect detection method based on deep learning according to claim 1, characterized in that, By optimizing the scheduling and memory reuse strategy of the neural network computation graph on the graphics processor, the processing time of a single image's complete process, including the reconstruction step, residual generation, and defect localization steps, is controlled within the range of 30 to 80 milliseconds. The optimizations include: fusing consecutive convolutional layers, batch normalization layers, and activation function layers in the generator module into a single computational kernel, and reusing feature map memory in-situ during forward propagation.

9. The LED chip defect detection method based on deep learning according to claim 1, characterized in that, In the self-diagnosis monitoring step, the time-frequency domain analysis of the reconstructed loss signal is specifically performed by performing a short-time Fourier transform or wavelet transform on the time series composed of the reconstructed loss values ​​in the continuous detection process to extract the energy distribution of different frequency components as frequency domain distribution features. The pre-calibrated frequency domain feature library includes: the low-frequency energy continuously rising feature corresponding to the light source attenuation mode, and the high-frequency energy attenuation and mid-frequency component broadening feature corresponding to the lens defocus mode.

10. A deep learning-based LED chip defect detection system, characterized in that, The system employs the deep learning-based LED chip defect detection method as described in any one of claims 1 to 9, wherein the system comprises: The image acquisition module is equipped with an industrial charge-coupled device camera with a pixel count between 12 million and 25 million, a telecentric lens with a magnification between 1x and 5x, and a coaxial illumination device that emits blue light in the wavelength range of 450 nm to 470 nm or green light in the wavelength range of 520 nm to 535 nm, for acquiring images of the LED chip under test. The magnification of the telecentric lens and the pixel count of the industrial charge-coupled device camera are configured together to meet the requirements for analyzing chip surface defects such as chipping or scratches with dimensions at the micrometer level. The good image reconstruction module has a built-in trained generator module, which receives the test image and generates the corresponding reconstructed good image. The generator module has a symmetrical encoding and decoding architecture. Its encoding path uses convolutional layers with a stride of 2 instead of pooling layers to achieve downsampling, and includes a substrate texture suppression operator based on an attention mechanism. There are skip connections between corresponding layers of the encoding and decoding paths. During the training phase, the generator module is adversarially trained with a Markov discriminator that outputs a probability matrix. The residual generation module is used to perform pixel-by-pixel comparison between the image to be tested and the reconstructed good image to generate a structural residual map. The adaptive threshold segmentation module is used to divide the structural residual map into multiple sub-blocks of preset size, and calculate the local adaptive segmentation threshold based on the local mean and local standard deviation of each sub-block; the multiplier k of the segmentation threshold is dynamically adjusted according to the yield of the current production batch. The morphological processing and feature extraction module is used to perform opening and closing operations on the binarized segmentation results to optimize connectivity, and to perform quantitative analysis of morphological feature parameters on the extracted connected components. The results are output to the database module, which is used to visually mark defect areas on the original image to be tested and write defect information into the production quality database. The self-diagnostic monitoring module is used to calculate the average reconstruction loss of a preset number of continuous LED chips in real time during continuous detection, and to identify the frequency domain offset mode of the average reconstruction loss signal through time-frequency domain analysis in order to distinguish between generating a light source attenuation warning signal or a lens defocus warning signal.

Images