Micro-defect recognition method and system of die AOI detection equipment

Abstract

The present application relates to a micro-defect recognition method and system of a crystal grain AOI detection device, comprising the following steps: step S1: collecting a two-dimensional reflection image of a to-be-detected crystal grain through a bright field and a dark field optical module of the crystal grain AOI detection device, simultaneously acquiring height distribution data of the crystal grain surface by using a laser automatic focusing module, and recording real-time suction force feedback signals of a mechanical hand suction cup during the process of sucking the crystal grain; acquiring a nominal size of the current to-be-detected crystal grain; step S2: performing pixel-level registration on the two-dimensional reflection image and the height distribution data to form a multi-channel input feature map; step S3: inputting the multi-channel input feature map into a shared feature extraction network, and outputting a feature vector of each pixel position by the shared feature extraction network; step S4: dynamically adjusting an abnormality detection sensitivity according to the real-time suction force feedback signals and the nominal size, that is, adjusting the abnormality detection sensitivity according to a sensitivity adjustment coefficient; the present application can realize better micro-defect recognition.

Description

Technical Field

[0001] This invention relates to the field of grain inspection, and more specifically to a method and system for identifying micro-defects in grain AOI inspection equipment. Background Technology

[0002] In the field of semiconductor silicon wafer dicing and grain sorting, automated optical inspection (AOI) equipment is the core equipment for identifying micro-defects in silicon grains. However, traditional grain AOI micro-defect detection technology has many technical shortcomings that make it difficult to adapt to industrial production: existing detection relies solely on two-dimensional reflection images, without incorporating grain surface height distribution data, making it unable to distinguish between two-dimensional surface defects such as scratches and dirt and three-dimensional structural defects such as chipping, detachment, and bulging, resulting in insufficient accuracy in micro-defect morphology determination; the detection sensitivity uses fixed parameters, without combining real-time suction feedback from the robotic arm picking up the grains with dynamic adjustment of the grain nominal size, making it easy to miss micro-defects when abnormal suction causes grain posture deviation or unstable adhesion, and easy to over-detect when suction is normal; the anomaly detection models are mostly trained offline with fixed parameters, lacking a semi-supervised incremental update mechanism, making it unable to adapt to fine-tuning of silicon wafer dicing processes and minor differences in the surface characteristics of silicon grains from different batches, resulting in model aging and poor generalization; defects The judgment threshold, morphological parameters of connected component analysis, and area screening threshold are all fixed values, which cannot match the detection characteristics of silicon grains of different sizes, resulting in missed detection of micro-defects in small-sized grains and false judgment of false defects in large-sized grains. There is a feature domain offset problem between different detection batches, and there is a lack of domain adaptive correction and rapid adaptation mechanism, resulting in distorted defect confidence output. Adaptation of new batches of grains requires full model retraining, which is inefficient. The core detection parameters rely on manual adjustment and there is no adaptive closed-loop optimization mechanism, which cannot dynamically balance the missed detection rate and the over-detection rate. The detection performance is difficult to continuously evolve and cannot meet the industrial detection requirements of high precision, high robustness, and adaptability of silicon grain micro-defects after silicon wafer cutting. Therefore, a micro-defect identification method and system for grain AOI inspection equipment is proposed. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for identifying micro-defects in grain AOI inspection equipment, comprising the following steps: Step S1: Acquire two-dimensional reflection images of the grain to be inspected using the bright-field and dark-field optical modules of the grain AOI inspection equipment. At the same time, use the laser autofocus module to obtain the height distribution data of the grain surface and record the real-time suction feedback signal of the robotic arm suction cup during the grain picking process; obtain the nominal size of the current grain to be inspected. Step S2: Perform pixel-level registration between the two-dimensional reflection image and the height distribution data to form a multi-channel input feature map; Step S3: Input the multi-channel input feature map into the shared feature extraction network, and the shared feature extraction network outputs the feature vector at each pixel location; Step S4: Dynamically adjust the anomaly detection sensitivity based on the real-time suction feedback signal and nominal size, i.e., adjust the anomaly detection sensitivity according to the sensitivity adjustment coefficient: First, calculate the theoretical suction value of the current grain based on the nominal size, which is proportional to the grain area; then, determine the upper and lower limits of the normal suction fluctuation range based on the theoretical suction value; finally, calculate the relative deviation of the real-time suction feedback signal from the normal range, and generate a continuously changing sensitivity adjustment coefficient based on the relative deviation. Step S5: Calculate the anomaly score for each feature vector using a pre-trained Gaussian Mixture Model (GM). The GM contains multiple Gaussian components, each with a mean vector and a covariance matrix. For the current feature vector, calculate its Mahalanobis distance with each Gaussian component. The Mahalanobis distance is calculated as follows: first, calculate the difference vector between the feature vector and the mean vector of the corresponding Gaussian component; then, multiply the transpose of the difference vector by the inverse of the covariance matrix of the corresponding Gaussian component, multiply by the difference vector again, and finally take the square root of the result. Take the minimum Mahalanobis distance among all Gaussian components as the anomaly score. The calculation process for the anomaly score is as follows: ; in Let x be the feature vector at pixel x. and Let T and φ be the mean vector and covariance matrix of the k-th Gaussian component, respectively. The superscript T indicates the vector transpose, and the superscript -1 indicates the matrix inversion. Step S6: Multiply the anomaly score by the sensitivity adjustment coefficient to obtain the weighted anomaly score; then dynamically determine a benchmark threshold based on the statistical characteristics of the weighted anomaly scores of all pixels in the current detection batch; compare the weighted anomaly score with the dynamic benchmark threshold; if the weighted anomaly score exceeds the dynamic benchmark threshold, the pixel is determined to be a suspected defect pixel; generate a binary image of the suspected defect pixel based on the determination result, where the pixel determined to be a suspected defect has a value of 1, and the other pixels have a value of 0; Step S7: Perform connected component analysis on the binary image of suspected defective pixels to form defect candidate regions; wherein, the morphological operation parameters and area screening threshold in the connected component analysis are dynamically adjusted according to the nominal size: when the grain size is larger than the preset size, increase the size of the morphological structural element and raise the lower and upper area thresholds; when the grain size is within the preset size, decrease the size of the structural element and lower the area threshold. Step S8: Use the domain adaptive correction module to perform confidence correction on each defect candidate region. The domain adaptive correction module aligns the feature distribution between the current detection batch and the historical normal batch through adversarial training. Step S9: Based on the corrected defect confidence level and the confidence threshold determined by the statistical distribution of confidence levels of normal and defective samples in historical batches, output the final defect category and location.

[0004] Furthermore, the process of generating the sensitivity adjustment coefficient in step S4 is as follows: First, the area A of the grain is calculated based on its nominal size, and then the theoretical suction value is calculated according to a preset linear relationship. ,in and These are coefficients calibrated experimentally. Then, determine the upper limit of the normal suction fluctuation range. and lower limit ,in This is the preset allowable fluctuation ratio; Next, the current real-time suction feedback value F is compared with the normal range. If the current real-time suction feedback value F is within the normal range, the relative deviation is 0; otherwise, the relative deviation is calculated. ,in The value that is closest to the normal range boundary value of F; Finally, the relative deviation is converted into a sensitivity adjustment coefficient α through a monotonically increasing nonlinear mapping function. This function outputs a reference value of 1 when the relative deviation is zero, and the output value gradually increases and approaches the preset maximum value as the relative deviation increases. ; The calculation process for the sensitivity adjustment coefficient is as follows: ; Where k is a preset gain coefficient, this formula makes The value is 1 when the suction power is normal, and greater than 1 when the suction power deviates, increasing monotonically with the degree of deviation. .

[0005] Furthermore, the parameters of the Gaussian mixture model in step S5 are obtained through semi-supervised learning and continuous update mechanisms, specifically including the following steps: Step S51: In the initial training phase, multiple batches of defect-free grain samples that have been re-inspected are collected. These samples are then input into a shared feature extraction network to obtain the feature vector of each pixel position of each sample, forming a feature vector set. Step S52: Fit the initial Gaussian mixture model using the expectation-maximization algorithm. The number of Gaussian components is automatically determined by the Bayesian information criterion. The expectation-maximization algorithm includes the following iterative process: First, calculate the posterior probability of each feature vector belonging to each component based on the current Gaussian component parameters. Then, re-estimate the mean vector, covariance matrix, and mixing coefficients of each component based on the posterior probability until convergence. Step S53: After each subsequent inspection task is completed, if the current batch of grains is confirmed to be defect-free after re-inspection, the feature vector of the batch is used as incremental data; the weighted expectation-maximization algorithm is used to incrementally update the Gaussian mixture model, where the cumulative weight of historical data decays exponentially with time, that is, after each update, the historical weight is multiplied by a decay factor to maintain the sensitivity of the Gaussian mixture model to the current process. Step S54: During incremental update, the mean vector of the j-th Gaussian component is updated according to the following weighted average method: the sum of the historical cumulative weight and the current batch weight is used as the denominator, and the numerator is the historical cumulative weight multiplied by the mean vector before the update plus the current batch weight multiplied by the mean vector of the feature vectors belonging to the j-th component in the current batch. The formula for updating the mean vector is: ; in The cumulative weight of historical data, As the weight of the current batch of data, The mean vector before the update. It is the mean vector of the feature vectors belonging to the j-th component in the current batch; Step S55: Covariance matrix of the j-th Gaussian component The incremental update method is as follows: First, calculate the covariance matrix of the eigenvectors belonging to the j-th component in the current batch. Then, a weighted average method, similar to the mean update, is used to merge the historical covariance matrix with the current batch covariance matrix, and a small positive regularization coefficient is added multiplied by the identity matrix to ensure that the updated covariance matrix is ​​positive definite; the update formula for the covariance matrix is: ; in The covariance matrix before the update is the regularization coefficient, and I is the identity matrix; Step S56: To prevent drastic changes in the parameters of the Gaussian mixture model, a parameter change constraint term is introduced during the incremental update process. This constraint term is based on the product of the sum of squares of the differences between the old and new parameters and the Fisher information matrix, and is added to the objective function of the expectation maximization algorithm as an elastic weight consolidation loss.

[0006] Furthermore, the process of determining the dynamic benchmark threshold in step S6 is as follows: The weighted anomaly scores of all pixels in the current detection batch are statistically analyzed, and their arithmetic mean and standard deviation are calculated. Then, based on the arithmetic mean and standard deviation, and introducing a configurable sensitivity coefficient that balances historical pass and miss rates, a dynamic baseline threshold is calculated. A larger sensitivity coefficient results in a higher threshold, increasing the risk of misses but decreasing pass rates; conversely, a smaller sensitivity coefficient results in a lower threshold, increasing the detection rate but increasing pass rates. The calculation process for the dynamic baseline threshold is as follows: ; in This is the arithmetic mean of the weighted anomaly scores for all pixels in the current batch. The standard deviation of the weighted outlier scores for all pixels in the current batch. This is a configurable sensitivity coefficient.

[0007] Furthermore, the connected component analysis process in step S7 is as follows: First, using the nominal size and combining it with the spatial resolution parameters of the image acquisition system, the physical size of the die is converted into the expected pixel area in the image. ; Then, based on the expected pixel area, the size of the structuring element for the morphological closing operation is dynamically determined: the square root of the expected pixel area is calculated and multiplied by a preset scaling factor. Then, round down to the nearest integer, and then limit the resulting value to a preset minimum of 3 and a maximum of 21, which is the structural element size. ; Next, set the lower and upper thresholds for area filtering based on the expected pixel area: Lower threshold Equal to the preset percentage coefficient Multiply by the expected pixel area Upper limit threshold Equal to the preset percentage coefficient Multiply by the expected pixel area ,Right now ; Then, morphological closing operations are performed on the binary image of the suspected defective pixels using the determined structuring element size r to connect neighboring isolated defective pixels; Finally, the connected components after the closing operation are filtered by area, retaining those with areas within the range of 1000. and Connected regions between them are considered as candidate regions for defects.

[0008] Furthermore, the specific calculation process of the domain adaptive correction module in step S8 is as follows: The domain adaptive correction module includes a feature extractor, a domain discriminator, and a defect classifier; The feature extractor extracts feature vectors from defect candidate regions; The domain discriminator is connected to the feature extractor via a gradient inversion layer to determine whether the feature vector originates from the current detection batch or a historical normal batch. The defect classifier receives feature vectors and outputs initial defect confidence scores; During training, the loss function of the domain discriminator is the cross-entropy loss. This loss is backpropagated to the feature extractor after passing through the gradient reversal layer, so that the features learned by the feature extractor cannot be distinguished by the domain discriminator, thereby achieving domain alignment. The confidence score of the defect after domain adaptive correction is obtained through the following steps: First, the feature vector of the defect candidate region is input into the feature extractor to obtain the domain-invariant features; Then, the domain-invariant features are input into the defect classifier to obtain the initial confidence level; Finally, the initial confidence is calibrated using a confidence scaling calibration method. A confidence scaling factor is introduced, which is obtained by minimizing the negative log-likelihood loss on the validation set. After calibration, the calibrated confidence is obtained. The confidence level calculation process after calibration is as follows: ; in Let i be the logical value of the i-th category output by the defect classifier. This is the confidence scaling factor. When the confidence level is greater than or equal to 1, the peak confidence level is reduced; when it is less than 1, the confidence level discrimination is enhanced.

[0009] Furthermore, the domain adaptive correction module also includes an online rapid adaptation mechanism: When the detection equipment switches to a new batch of grains, a preset number of defect candidate regions with the highest abnormal scores and non-overlapping spatial distributions are automatically selected from the batch as representative samples. For representative samples, gradient updates are performed on the parameters of the last fully connected layer of the defect classifier in a limited number of steps. During the update, an optimizer with weight decay is used, and the weight decay coefficient is adaptively adjusted according to the number of samples. During the update process, the parameters of the feature extractor and the domain discriminator are frozen; After each update, calculate the confidence distribution entropy of the detected areas in the current batch, and stop updating when the entropy value no longer decreases after several consecutive iterations; The gradient update formula is: ; Where t is the iteration step index. These are the parameters of the final layer of the defect classifier at step t. For learning rate, For cross-entropy loss, is the weight decay coefficient, and is the gradient of the loss with respect to the parameters.

[0010] Furthermore, step S9 is followed by a defect 3D morphology determination step based on height distribution data: For each defective region in the final output, extract the height value matrix of that region in the height distribution data; A ring-shaped background area is formed by extending a predetermined number of pixels outward from the boundary of the defective area. Outliers with significantly deviated height values ​​in the background area are removed. Outliers are identified using a method based on interquartile range. After removing outliers, the average height of the background area is calculated. Then calculate the median of the height values ​​of all pixels within the defect area. And the arithmetic mean of the height values ​​of all pixels within the defective area, i.e., the average height of the background area. ; Median height of the defect area Average height of the background area Subtracting the two values ​​gives the relative height difference; simultaneously, the standard deviation of the height values ​​within the defect area is calculated. ; If the absolute value of the relative height difference is greater than the first threshold determined based on the grain surface roughness statistics, and the height standard deviation is greater than the second threshold determined based on the height standard deviation statistics of the defect-free area on the grain surface, then the defect area is determined to be a three-dimensional structural defect, including chipping, detachment, or bulging. Otherwise, it is judged as a two-dimensional surface defect, which includes scratches, dirt, or ink spots; after the judgment is completed, the three-dimensional morphological information of the defect is output as an additional attribute. The calculation process for the relative height difference is as follows: ; in The average height of the background area after removing outliers; The calculation process for the height standard deviation is as follows: ; in This represents the number of pixels within the defective area. This represents the height value of pixel p.

[0011] Furthermore, it also includes an adaptive closed-loop optimization step for the detection parameters: After each testing task is completed, collect the actual defect labels confirmed by re-inspection equipment or manual inspection, and calculate the recall and precision of this test. Construct a comprehensive performance index based on the recall and precision; this index is the harmonic average of recall and precision. Use the following parameter combination used in the current testing task as the variables to be optimized: the gain coefficient k from step S4, and the sensitivity coefficient from step S6. The morphological structural element size and area screening threshold in step S7, and the confidence scaling factor in step S8. The confidence threshold in step S9; Record each parameter combination and its corresponding comprehensive performance index to build a performance database; when the number of records in the database reaches the preset minimum sample size, use a Gaussian process regression model to fit the mapping relationship between the parameter combination and the comprehensive performance index. The kernel function of the Gaussian process is the sum of the radial basis function and the white noise kernel. Then, based on the prediction results of the Gaussian process regression model, the expected improvement acquisition function in Bayesian optimization is used to guide the search for the next set of parameter combinations; The proposed improved acquisition function takes into account both the current optimal performance and prediction uncertainty, and its calculation process is as follows: ; in For parameter combination vectors, and Gaussian processes in The predicted mean and predicted standard deviation at the location, This represents the current optimal overall performance index value. The standard normal cumulative distribution function is... It is the standard normal probability density function; The optimal parameter combination found is automatically deployed to subsequent batches of detection tasks, enabling continuous evolution of detection performance.

[0012] The micro-defect identification system for grain AOI inspection equipment includes: The data acquisition module is used to acquire two-dimensional reflection images of the grains to be inspected through the bright field and dark field optical modules of the grain AOI inspection equipment. At the same time, it uses the laser autofocus module to obtain the height distribution data of the grain surface, records the real-time suction feedback signal of the robotic arm suction cup during the process of picking up the grains, and obtains the nominal size of the grains to be inspected. The registration module is used to perform pixel-level registration between the two-dimensional reflectance image and the height distribution data to form a multi-channel input feature map. The feature extraction module is used to input the multi-channel input feature map into the shared feature extraction network and output the feature vector at each pixel location; The sensitivity dynamic adjustment module is used to dynamically adjust the anomaly detection sensitivity based on the real-time suction feedback signal and nominal size. Specifically, it adjusts the anomaly detection sensitivity according to the sensitivity adjustment coefficient: First, it calculates the theoretical suction value of the current grain based on the nominal size, which is proportional to the grain area; then, it determines the upper and lower limits of the normal suction fluctuation range based on the theoretical suction value; finally, it calculates the relative deviation of the real-time suction feedback signal from the normal range and generates a continuously changing sensitivity adjustment coefficient based on the relative deviation. The anomaly score calculation module is used to calculate the anomaly score for each feature vector using a pre-trained Gaussian mixture model. The Gaussian mixture model contains multiple Gaussian components, each of which has a mean vector and a covariance matrix. The weighted comparison module is used to multiply the anomaly score by the sensitivity adjustment coefficient to obtain the weighted anomaly score; then, a benchmark threshold is dynamically determined based on the statistical characteristics of the weighted anomaly scores of all pixels in the current detection batch; the weighted anomaly score is compared with the dynamic benchmark threshold; if the weighted anomaly score exceeds the dynamic benchmark threshold, the pixel is determined to be a suspected defect pixel; a binary image of the suspected defect pixel is generated based on the determination result, where the pixel determined to be a suspected defect has a value of 1, and the other pixels have a value of 0; The connected component analysis module is used to perform connected component analysis on the binary image of suspected defective pixels to form defect candidate regions. The morphological operation parameters and area screening thresholds in the connected component analysis are dynamically adjusted according to the nominal size: when the grain size is larger than the preset size, the size of the morphological structural element is increased and the lower and upper area thresholds are raised; when the grain size is within the preset size, the size of the structural element is decreased and the area threshold is lowered. The domain adaptive correction module is used to perform confidence correction on each defect candidate region. The domain adaptive correction module aligns the feature distribution between the current detection batch and the historical normal batch through adversarial training. The output module is used to output the final defect category and location based on the corrected defect confidence level and the confidence threshold determined by the statistical distribution of confidence levels of normal and defective samples in historical batches.

[0013] The present invention has the following advantages over the prior art: By fusing two-dimensional reflection images of grains, surface height distribution data, and real-time suction feedback signals from robotic arms and completing pixel-level registration to construct multi-channel feature inputs, the anomaly detection sensitivity is dynamically adjusted based on the nominal grain size and suction deviation. A Gaussian mixture model with a semi-supervised learning and incremental update mechanism is used to accurately calculate pixel anomaly scores. Defect candidate regions are efficiently screened by relying on dynamic benchmark thresholds and connected component analysis that is adaptive to grain size. Domain adaptive correction and online rapid adaptation mechanisms are used to eliminate feature distribution offsets between batches to improve the accuracy of defect confidence. Furthermore, the three-dimensional morphology of defects can be determined based on height distribution data to distinguish between two-dimensional surface defects and three-dimensional structural defects. Finally, closed-loop Bayesian optimization is used to achieve automatic iterative upgrades of detection parameters. Overall, the accuracy, robustness, and scene adaptability of grain micro-defect identification are significantly improved, effectively reducing the false negative rate and false positive rate. It adapts to different grain sizes and process batch variations, allowing the detection performance to continuously evolve. Attached Figure Description

[0014] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0015] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0016] like Figure 1 As shown, the micro-defect identification method of the grain AOI inspection equipment includes the following steps: Step S1: Acquire two-dimensional reflection images of the grain to be inspected using the bright-field and dark-field optical modules of the grain AOI inspection equipment. At the same time, use the laser autofocus module to obtain the height distribution data of the grain surface and record the real-time suction feedback signal of the robotic arm suction cup during the grain picking process; obtain the nominal size of the current grain to be inspected. Step S2: Perform pixel-level registration between the two-dimensional reflection image and the height distribution data to form a multi-channel input feature map; Step S3: Input the multi-channel input feature map into the shared feature extraction network, and the shared feature extraction network outputs the feature vector at each pixel location; Step S4: Dynamically adjust the anomaly detection sensitivity based on the real-time suction feedback signal and nominal size, i.e., adjust the anomaly detection sensitivity according to the sensitivity adjustment coefficient: First, calculate the theoretical suction value of the current grain based on the nominal size, which is proportional to the grain area; then, determine the upper and lower limits of the normal suction fluctuation range based on the theoretical suction value; finally, calculate the relative deviation of the real-time suction feedback signal from the normal range, and generate a continuously changing sensitivity adjustment coefficient based on the relative deviation. Step S5: Calculate the anomaly score for each feature vector using a pre-trained Gaussian Mixture Model (GM). The GM contains multiple Gaussian components, each with a mean vector and a covariance matrix. For the current feature vector, calculate its Mahalanobis distance with each Gaussian component. The Mahalanobis distance is calculated as follows: first, calculate the difference vector between the feature vector and the mean vector of the corresponding Gaussian component; then, multiply the transpose of the difference vector by the inverse of the covariance matrix of the corresponding Gaussian component, multiply by the difference vector again, and finally take the square root of the result. Take the minimum Mahalanobis distance among all Gaussian components as the anomaly score. The calculation process for the anomaly score is as follows: ; in Let x be the feature vector at pixel x. and Let T and φ be the mean vector and covariance matrix of the k-th Gaussian component, respectively. The superscript T indicates the vector transpose, and the superscript -1 indicates the matrix inversion. Step S6: Multiply the anomaly score by the sensitivity adjustment coefficient to obtain the weighted anomaly score; then dynamically determine a benchmark threshold based on the statistical characteristics of the weighted anomaly scores of all pixels in the current detection batch; compare the weighted anomaly score with the dynamic benchmark threshold; if the weighted anomaly score exceeds the dynamic benchmark threshold, the pixel is determined to be a suspected defect pixel; generate a binary image of the suspected defect pixel based on the determination result, where the pixel determined to be a suspected defect has a value of 1, and the other pixels have a value of 0; Step S7: Perform connected component analysis on the binary image of suspected defective pixels to form defect candidate regions; wherein, the morphological operation parameters and area screening threshold in the connected component analysis are dynamically adjusted according to the nominal size: when the grain size is larger than the preset size, increase the size of the morphological structural element and raise the lower and upper area thresholds; when the grain size is within the preset size, decrease the size of the structural element and lower the area threshold. Step S8: Use the domain adaptive correction module to perform confidence correction on each defect candidate region. The domain adaptive correction module aligns the feature distribution between the current detection batch and the historical normal batch through adversarial training. Step S9: Based on the corrected defect confidence level and the confidence threshold determined by the statistical distribution of confidence levels of normal and defective samples in historical batches, output the final defect category and location.

[0017] The specific process for generating the sensitivity adjustment coefficient in step S4 is as follows: First, the area A of the grain is calculated based on its nominal size, and then the theoretical suction value is calculated according to a preset linear relationship. ,in and These are coefficients calibrated experimentally. Then, determine the upper limit of the normal suction fluctuation range. and lower limit ,in This is the preset allowable fluctuation ratio; Next, the current real-time suction feedback value F is compared with the normal range. If the current real-time suction feedback value F is within the normal range, the relative deviation is 0; otherwise, the relative deviation is calculated. ,in The value that is closest to the normal range boundary value of F; Finally, the relative deviation is converted into a sensitivity adjustment coefficient α through a monotonically increasing nonlinear mapping function. This function outputs a reference value of 1 when the relative deviation is zero, and the output value gradually increases and approaches the preset maximum value as the relative deviation increases. ; The calculation process for the sensitivity adjustment coefficient is as follows: ; Where k is a preset gain coefficient, this formula makes The value is 1 when the suction power is normal, and greater than 1 when the suction power deviates, increasing monotonically with the degree of deviation. ; The theoretical suction value is calculated linearly based on the nominal size of the silicon wafer diced grains. Combined with a preset allowable fluctuation ratio, the normal suction fluctuation range is defined. The deviation of the real-time suction feedback is generated into a continuously changing sensitivity adjustment coefficient through a monotonically increasing nonlinear mapping. When the suction force of the robotic arm is normal when picking up silicon diced grains, the baseline detection sensitivity is maintained and over-detection is avoided. When the suction force is abnormal, the detection sensitivity is adaptively improved according to the degree of deviation. It is adapted to the picking conditions of silicon diced grains of different sizes after silicon wafer cutting, accurately matches the silicon diced grain grasping state and detection requirements, enhances the ability to detect micro-defects when silicon diced grains are deviated in posture or have abnormal adhesion due to suction fluctuations, and significantly improves the stability and detection rate of silicon diced grain micro-defect identification.

[0018] At the AOI inspection station of the semiconductor silicon wafer cutting production line, when the robotic arm suction cup picks up 0402 type (1.0mm×0.5mm) and 0603 type (1.6mm×0.8mm) silicon granules after silicon wafer cutting, slight leakage in the vacuum pipeline and uneven adhesion between the suction cup and the silicon granules cause fluctuations in suction force, requiring dynamic adjustment of the detection sensitivity.

[0019] Preset parameters: Experimental calibration coefficients Permissible fluctuation ratio Gain coefficient Maximum value of sensitivity adjustment coefficient .

[0020] Calculate silicon grain area: 0402 silicon grain 0603 silicon grains ; According to the formula Theoretical suction force calculated for 0402 silicon grains. 0603 silicon grains ; According to the formula , Determine the normal suction range: 0402 silicon grains , 0603 silicon grains ; Real-time suction value: 0402 silicon grains (Below the lower limit); 0603 silicon grains (Above the upper limit); According to the formula Calculate the relative deviation: 0402 silicon grain relative deviation 0603 silicon grain relative deviation ; According to the formula Calculate the sensitivity adjustment coefficient: 0402 silicon grains 0603 silicon grains ; In this scenario, when the suction is normal, the sensitivity coefficient is 1 to maintain the baseline detection state. The 0402 silicon grain has a greater deviation in suction and a higher sensitivity coefficient, which can accurately detect micro-chipping at the edge of the silicon grain and micro-cracks left over from cutting caused by abnormal suction. The 0603 silicon grain has a smaller deviation and a moderate sensitivity coefficient, avoiding meaningless over-detection. It is suitable for the suction conditions and detection needs of silicon grains of different sizes after silicon wafer cutting.

[0021] In step S5, the parameters of the Gaussian mixture model are obtained through semi-supervised learning and continuous update mechanisms, specifically including the following steps: Step S51: In the initial training phase, multiple batches of defect-free grain samples that have been re-inspected are collected. These samples are then input into a shared feature extraction network to obtain the feature vector of each pixel position of each sample, forming a feature vector set. Step S52: Fit the initial Gaussian mixture model using the expectation-maximization algorithm. The number of Gaussian components is automatically determined by the Bayesian information criterion. The expectation-maximization algorithm includes the following iterative process: First, calculate the posterior probability of each feature vector belonging to each component based on the current Gaussian component parameters. Then, re-estimate the mean vector, covariance matrix, and mixing coefficients of each component based on the posterior probability until convergence. Step S53: After each subsequent inspection task is completed, if the current batch of grains is confirmed to be defect-free after re-inspection, the feature vector of the batch is used as incremental data; the weighted expectation-maximization algorithm is used to incrementally update the Gaussian mixture model, where the cumulative weight of historical data decays exponentially with time, that is, after each update, the historical weight is multiplied by a decay factor to maintain the sensitivity of the Gaussian mixture model to the current process. Step S54: During incremental update, the mean vector of the j-th Gaussian component is updated according to the following weighted average method: the sum of the historical cumulative weight and the current batch weight is used as the denominator, and the numerator is the historical cumulative weight multiplied by the mean vector before the update plus the current batch weight multiplied by the mean vector of the feature vectors belonging to the j-th component in the current batch. The formula for updating the mean vector is: ; in The cumulative weight of historical data, As the weight of the current batch of data, The mean vector before the update. It is the mean vector of the feature vectors belonging to the j-th component in the current batch; Step S55: Covariance matrix of the j-th Gaussian component The incremental update method is as follows: First, calculate the covariance matrix of the eigenvectors belonging to the j-th component in the current batch. Then, a weighted average method, similar to the mean update, is used to merge the historical covariance matrix with the current batch covariance matrix, and a small positive regularization coefficient is added multiplied by the identity matrix to ensure that the updated covariance matrix is ​​positive definite; the update formula for the covariance matrix is: ; in The covariance matrix before the update is the regularization coefficient, and I is the identity matrix; Step S56: To prevent drastic changes in the parameters of the Gaussian mixture model, a parameter change constraint term is introduced during the incremental update process. This constraint term is based on the product of the sum of squares of the differences between the old and new parameters and the Fisher information matrix, and is added to the objective function of the expectation maximization algorithm as an elastic weight consolidation loss. The Gaussian mixture model is initially trained using semi-supervised learning. The model is incrementally updated by using a weighted expectation-maximization algorithm combined with exponentially decaying weights based on historical data. The mean vector and covariance matrix are weighted and fused, and regularization is added to ensure that the covariance matrix is ​​positive definite. At the same time, elastic weights are introduced to consolidate constraints and avoid drastic parameter fluctuations. The model can continuously adapt to the surface features of silicon grains in different batches after silicon wafer cutting and the slight changes in the cutting process without the need for a large number of labeled samples. It always maintains the sensitivity to the detection of micro-defects such as micro-chipping, micro-cracks, and cutting residues in silicon grains, and solves the problems of missed detection and over-detection caused by model aging and insufficient generalization. This improves the long-term stability and recognition accuracy of silicon grain AOI detection.

[0022] The AOI inspection station for semiconductor silicon wafer cutting and sorting continuously inspects the 0402 type silicon granules (1.0mm×0.5mm) obtained from silicon wafer cutting. After 100 batches, the 101st batch has slight changes in the surface reflection and height characteristics of the silicon granules due to slight wear of the cutting tool and minor adjustments to the surface coating process. A semi-supervised training and continuous model update mechanism is used to adapt to the process changes.

[0023] Preset parameters: Cumulative weight of historical data (Exponential decay factor 0.9, obtained after the previous weight iteration), current batch data weights. ; Regularization coefficient identity matrix ; The j-th Gaussian component history parameter ; The characteristic parameters of this component in the current batch .

[0024] Mean vector incremental update: According to the formula calculate: ; Incremental update of covariance matrix: According to the formula calculate: Weighted fusion term: ; Add regular expression: ; In the initial stage, the model was fitted using the EM algorithm with defect-free 0402 silicon grain samples. After the silicon wafer cutting process was fine-tuned in the 101st batch, the model was updated only with incremental defect-free samples from the current batch. Historical weights were exponentially decayed to avoid old data dominating the model. Covariance matrix regularization ensured numerical stability. Elastic weights reinforced constraints to prevent parameter mutations. The model quickly adapted to the characteristics of new batches of silicon grains and accurately detected micro-chipping defects and surface micro-cracks on silicon grain edges caused by cutting tool wear, with no missed or over-detection.

[0025] The process of determining the dynamic benchmark threshold in step S6 is as follows: The weighted anomaly scores of all pixels in the current detection batch are statistically analyzed, and their arithmetic mean and standard deviation are calculated. Then, based on the arithmetic mean and standard deviation, and introducing a configurable sensitivity coefficient that balances historical pass and miss rates, a dynamic baseline threshold is calculated. A larger sensitivity coefficient results in a higher threshold, increasing the risk of misses but decreasing pass rates; conversely, a smaller sensitivity coefficient results in a lower threshold, increasing the detection rate but increasing pass rates. The calculation process for the dynamic baseline threshold is as follows: ; in This is the arithmetic mean of the weighted anomaly scores for all pixels in the current batch. The standard deviation of the weighted outlier scores for all pixels in the current batch. This is a configurable sensitivity coefficient; By statistically analyzing the arithmetic mean and standard deviation of the weighted anomaly scores of all pixels in the current detection batch, and combining this with a configurable sensitivity coefficient to dynamically determine the baseline detection threshold, the system can adaptively match the detection data distribution of different batches of silicon dies after wafer cutting. This flexibly balances the false negative rate and false positive rate of micro-defect detection, avoiding detection failures caused by fixed thresholds failing to adapt to batch process differences, and significantly improving the adaptability and detection accuracy of silicon die AOI micro-defect detection.

[0026] In the AOI inspection station for semiconductor silicon wafer cutting and sorting, the 0402 type silicon grains (1.0mm×0.5mm) obtained from silicon wafer cutting are continuously inspected. In the 101st batch, due to slight wear of the cutting tool, the weighted abnormal score distribution on the surface of the silicon grains has slightly shifted. The dynamic benchmark threshold method is used to determine the detection threshold.

[0027] Preset parameter: The weighted arithmetic mean of the anomaly scores of all pixels in the current batch. Standard deviation ; High-precision detection requires a sensitivity coefficient γ=2.5, routine balancing requires γ=3, and low over-detection requires γ=3.5; The dynamic baseline threshold calculation process is as follows: .

[0028] High-precision detection scenarios (focusing on reducing missed detections): ; With a lower threshold, it can accurately detect minute defects such as tiny cutting cracks and edge micro-chipping on the surface of silicon grains, significantly reducing the false negative rate.

[0029] Typical balancing scenario (accommodating both missed and over-detections): ; With a moderate threshold, it balances the detection of micro-defects in silicon grains with the filtering of false defects, making it suitable for the routine needs of mass production testing.

[0030] Low-pass detection scenarios (focusing on reducing false positives); ; With a higher threshold, it effectively filters out false defects caused by normal textures and slight cutting marks on the silicon grain surface, significantly reducing the over-detection rate.

[0031] The dynamic threshold is adjusted in real time according to the batch data distribution of silicon grains, and the sensitivity coefficient can be configured as needed. It perfectly adapts to the data changes caused by process fluctuations after silicon wafer cutting, solves the core problem that fixed thresholds cannot adapt to batch differences, and ensures the stability and accuracy of detection.

[0032] The connected component analysis process in step S7 is as follows: First, using the nominal size and combining it with the spatial resolution parameters of the image acquisition system, the physical size of the die is converted into the expected pixel area in the image. ; Then, based on the expected pixel area, the size of the structuring element for the morphological closing operation is dynamically determined: the square root of the expected pixel area is calculated and multiplied by a preset scaling factor. Then, round down to the nearest integer, and then limit the resulting value to a preset minimum of 3 and a maximum of 21, which is the structural element size. ; Next, set the lower and upper thresholds for area filtering based on the expected pixel area: Lower threshold Equal to the preset percentage coefficient Multiply by the expected pixel area Upper limit threshold Equal to the preset percentage coefficient Multiply by the expected pixel area ,Right now ; Then, morphological closing operations are performed on the binary image of the suspected defective pixels using the determined structuring element size r to connect neighboring isolated defective pixels; Finally, the connected components after the closing operation are filtered by area, retaining those with areas within the range of 1000. and The connected regions between them are used as candidate regions for defects; Based on the nominal size of silicon grains after silicon wafer cutting and the spatial resolution of image acquisition, the expected pixel area is calculated. The size of the morphological closed operation structure element is dynamically determined and the upper and lower limits of area screening are adaptively set. This can accurately match the defect pixel distribution characteristics of silicon grains of different sizes, effectively connect adjacent isolated defect pixels and filter out invalid small noise and excessively large false defect areas. This avoids the failure to detect micro-defects in small silicon grains and the misjudgment of false defects in large silicon grains due to fixed parameters, and significantly improves the accuracy and robustness of silicon grain defect candidate region extraction.

[0033] The AOI inspection station for semiconductor silicon wafer cutting and sorting performs micro-defect detection on 0402 type silicon granules (1.0mm×0.5mm) and 0603 type silicon granules (1.6mm×0.8mm) obtained from silicon wafer cutting. The dynamic connected domain analysis method is adapted to the defect extraction requirements of silicon granules of different sizes.

[0034] Preset parameters: Spatial resolution of the image acquisition system Morphological structural element ratio coefficient Area screening lower limit coefficient Upper limit coefficient ;Structural element size constraints .

[0035] Calculate the physical area of ​​the silicon die and the expected pixel area. : Pixel area calculation process: ; 0402 silicon grain: physical area ; ; 0603 silicon grain: physical area ; ; Dynamically determine the size r of the morphological closing operation structuring element: Calculation process: ; 0402 silicon grains: ; 0603 silicon grains: ; Dynamically calculate the area filtering threshold: Calculation process: ; 0402 silicon grains: ; 0603 silicon grains: ; Morphological closing operations are performed on suspected defective pixels using structuring elements of corresponding sizes to connect isolated defective pixels; then, effective defective connected regions are retained by filtering according to a dynamic area threshold. The 0402 silicon grain uses small structuring elements and a narrow area range to accurately capture small defects such as edge micro-chipping and micro-cracks; the 0603 silicon grain uses large structuring elements and a wide area range to effectively filter noise and completely extract the real defective region, with no missed or over-detected defects.

[0036] The specific calculation process of the domain adaptive correction module in step S8 is as follows: The domain adaptive correction module includes a feature extractor, a domain discriminator, and a defect classifier; The feature extractor extracts feature vectors from defect candidate regions; The domain discriminator is connected to the feature extractor via a gradient inversion layer to determine whether the feature vector originates from the current detection batch or a historical normal batch. The defect classifier receives feature vectors and outputs initial defect confidence scores; During training, the loss function of the domain discriminator is the cross-entropy loss. This loss is backpropagated to the feature extractor after passing through the gradient reversal layer, so that the features learned by the feature extractor cannot be distinguished by the domain discriminator, thereby achieving domain alignment. The confidence score of the defect after domain adaptive correction is obtained through the following steps: First, the feature vector of the defect candidate region is input into the feature extractor to obtain the domain-invariant features; Then, the domain-invariant features are input into the defect classifier to obtain the initial confidence level; Finally, the initial confidence is calibrated using a confidence scaling calibration method. A confidence scaling factor is introduced, which is obtained by minimizing the negative log-likelihood loss on the validation set. After calibration, the calibrated confidence is obtained. The confidence level calculation process after calibration is as follows: ; in Let i be the logical value of the i-th category output by the defect classifier. This is the confidence scaling factor. When the confidence level is greater than or equal to 1, the peak confidence level is reduced; when it is less than 1, the confidence level discrimination is enhanced. By using a domain adaptive correction module with adversarial training and a gradient inversion layer to align the feature distribution of silicon grains in the current detection batch with those in historical normal batches and extract domain-invariant features, and then correcting the defect confidence through confidence scaling calibration, the feature domain offset problem caused by the fine-tuning of the cutting process and the difference in surface state of different batches of silicon grains after silicon wafer cutting can be effectively eliminated. This improves the accuracy and stability of the defect confidence output, avoids misjudgment and missed judgment of silicon grain micro-defects caused by domain differences, and greatly enhances the generalization and adaptation ability of AOI detection to different batches of silicon grains.

[0037] In the AOI inspection station for semiconductor silicon wafer cutting and sorting, the 0402 type silicon grains (1.0mm×0.5mm) obtained from the cutting of silicon wafers are inspected. Due to wear of the cutting tool and slight adjustment of the coating thickness, the silicon grain characteristics of the 101st batch have a domain offset from those of the historical normal batches. The defect confidence is corrected by domain adaptive correction and confidence scaling calibration method.

[0038] Preset parameters: Defect classifier output logical value (defect category) Normal category ); confidence scaling factor ; Confidence calibration formula: .

[0039] Domain-adaptive feature alignment: The feature extractor of the domain adaptive correction module extracts the features of candidate regions for silicon grain defects. The domain discriminator backpropagates the domain discrimination loss through the gradient inversion layer, forcing the feature extractor to learn domain-invariant features and eliminating the feature distribution offset between the 101st batch and historical batches.

[0040] Initial confidence level calculation: The defect classifier outputs a logical value for the defect category based on domain-invariant features. Normal category logical value .

[0041] Confidence scaling calibration calculation; Defect category numerator: ; Normal category numerator: ; Summation term in denominator: ; Defect confidence level after calibration: ; Normal confidence level after calibration: ; Uncalibrated raw confidence levels are susceptible to distortion due to domain offset. After scaling and calibration, the defect confidence level closely matches the actual defect probability of silicon grains, accurately identifying micro-chipping defects at the edges of silicon grains and micro-cracks on the surface. This effectively avoids over- and under-detection caused by batch domain offset and meets the detection needs of process fluctuations after silicon wafer cutting.

[0042] Furthermore, the domain adaptive correction module also includes an online rapid adaptation mechanism: When the detection equipment switches to a new batch of grains, a preset number of defect candidate regions with the highest abnormal scores and non-overlapping spatial distributions are automatically selected from the batch as representative samples. For representative samples, gradient updates are performed on the parameters of the last fully connected layer of the defect classifier in a limited number of steps. During the update, an optimizer with weight decay is used, and the weight decay coefficient is adaptively adjusted according to the number of samples. During the update process, the parameters of the feature extractor and the domain discriminator are frozen; After each update, calculate the confidence distribution entropy of the detected areas in the current batch, and stop updating when the entropy value no longer decreases after several consecutive iterations; The gradient update formula is: ; Where t is the iteration step index. These are the parameters of the final layer of the defect classifier at step t. For learning rate, For cross-entropy loss, This is the weight decay coefficient. The gradient of the loss with respect to the parameters; Through the online rapid adaptation mechanism of the domain adaptive correction module, representative defect candidate region samples are automatically selected when switching to a new batch of silicon grains after silicon wafer dicing. Only a limited number of gradient updates are performed on the last fully connected layer of the defect classifier, and the parameters of the feature extractor and domain discriminator are frozen. With the adaptive weight decay and confidence distribution entropy stopping condition, the feature distribution of the new batch of silicon grains can be quickly aligned without full model retraining. This greatly reduces the adaptation time and computational resource consumption of the new batch, avoids model overfitting, and quickly and stably improves the confidence accuracy of micro-defect detection in the new batch of silicon grains. It also efficiently adapts to batch feature changes caused by fluctuations in silicon wafer dicing process.

[0043] At the semiconductor silicon wafer cutting and sorting AOI inspection station, the equipment was switched to a new batch of 0402 type silicon grains (1.0mm×0.5mm). Due to the change of cutting tools and batch differences in silicon wafer material, the surface features of the silicon grains were slightly off from those of historical batches. An online rapid adaptation mechanism was used to complete the rapid adaptation of the model.

[0044] Preset parameters: learning rate η = 0.01; initial defect classifier last layer parameters. Cross-entropy loss gradient Adaptive weight decay coefficient ; Iteration stopping condition: The confidence distribution entropy shows no decrease for three consecutive iterations; Gradient update formula: .

[0045] Representative sample selection: Five candidate defect regions with the highest anomaly scores and non-overlapping spatial distributions are automatically selected from the new batch of 0402 silicon grains as online adaptation samples, covering two typical defects: micro-fractures and micro-cracks.

[0046] Parameter freeze and update range settings: Freeze all parameters of the feature extractor and domain discriminator, and perform gradient updates only on the parameters of the last fully connected layer of the defect classifier.

[0047] Online gradient update calculation (1st iteration, t=0): Substitute the parameters into the update formula: ; Step-by-step calculation: Items in parentheses: ; Multiplication term: ; Updated parameters: ; Iteration stopping criteria: After each iteration, the confidence distribution entropy of the current detection region is calculated. If the entropy value decreases by less than 0.001 for three consecutive iterations, the stopping condition is met, and the update is terminated.

[0048] It takes only 10 seconds and 5 samples to complete the adaptation of the new batch without the need for full model retraining. The model quickly aligns with the feature distribution of the new batch of 0402 silicon grains, accurately identifies micro-chipping at the edge of the silicon grains and micro-cracks on the surface, with no over-detection or under-detection, and fully adapts to the batch process changes after silicon wafer cutting.

[0049] Step S9 is followed by a defect 3D morphology discrimination step based on height distribution data: For each defective region in the final output, extract the height value matrix of that region in the height distribution data; A ring-shaped background area is formed by extending a predetermined number of pixels outward from the boundary of the defective area. Outliers with significantly deviated height values ​​in the background area are removed. Outliers are identified using a method based on interquartile range. After removing outliers, the average height of the background area is calculated. Then calculate the median of the height values ​​of all pixels within the defect area. And the arithmetic mean of the height values ​​of all pixels within the defective area, i.e., the average height of the background area. ; Median height of the defect area Average height of the background area Subtracting the two values ​​gives the relative height difference; simultaneously, the standard deviation of the height values ​​within the defect area is calculated. ; If the absolute value of the relative height difference is greater than the first threshold determined based on the grain surface roughness statistics, and the height standard deviation is greater than the second threshold determined based on the height standard deviation statistics of the defect-free area on the grain surface, then the defect area is determined to be a three-dimensional structural defect, including chipping, detachment, or bulging. Otherwise, it is judged as a two-dimensional surface defect, which includes scratches, dirt, or ink spots; after the judgment is completed, the three-dimensional morphological information of the defect is output as an additional attribute. The calculation process for the relative height difference is as follows: ; in The average height of the background area after removing outliers; The calculation process for the height standard deviation is as follows: ; in This represents the number of pixels within the defective area. The height value of pixel p; Based on the height distribution data of silicon grains after silicon wafer dicing, this method extracts height features of defect areas, removes outliers in the annular background area, and calculates the baseline height. Combining the relative height difference and height standard deviation as dual thresholds, it can accurately distinguish between two-dimensional surface defects and three-dimensional structural defects of silicon grains. It clarifies whether the defects are surface problems such as scratches and dirt or structural damage such as chipping, detachment, and bulging. It also supplements the three-dimensional morphological attributes of defects, solving the problem that pure two-dimensional image detection cannot determine the depth and structural morphology of defects. This significantly improves the accuracy of silicon grain micro-defect classification and the completeness of detection information, meeting the needs of refined defect judgment for silicon wafer dicing.

[0050] The AOI inspection station for semiconductor silicon wafer cutting and sorting performs micro-defect detection on the 0402 type silicon grains (1.0mm×0.5mm) obtained from silicon wafer cutting, and distinguishes between two-dimensional surface defects and three-dimensional structural defects through a three-dimensional morphology discrimination method.

[0051] Preset parameters: Number of pixels in the defect area Average height of the background area after removing outliers The first threshold for relative height difference ; Second threshold of height standard deviation ; Median height of defect area ; Average height of defect area ; The height values ​​of the 20 pixels in the defect area are: 6.3, 6.4, 6.5, 6.6, 6.7, 6.3, 6.4, 6.5, 6.6, 6.7, 6.3, 6.4, 6.5, 6.6, 6.7, 6.3, 6.4, 6.5, 6.6, 6.8.

[0052] Calculate the relative height difference: Formula used: ; ; absolute value The first condition for judgment is met.

[0053] Calculate the standard deviation of the defect area height: Formula used: ; Step-by-step calculation: Sum of squares of height differences: ; Sum of squares and mean: ; Height standard deviation: ; The second condition is met.

[0054] If both threshold conditions are met, the defect is determined to be a three-dimensional structural defect (bulge); if it is contamination on the silicon grain surface, the relative height difference is only 0.2μm and the height standard deviation is 0.1μm, which does not meet the two threshold conditions, and it is determined to be a two-dimensional surface defect.

[0055] This method accurately distinguishes between surface defects and structural damage in silicon grains, providing precise three-dimensional morphological basis for defect classification and sorting of silicon wafer dicing grains, with no judgment errors.

[0056] It also includes an adaptive closed-loop optimization step for the detection parameters: After each testing task is completed, collect the actual defect labels confirmed by re-inspection equipment or manual inspection, and calculate the recall and precision of this test. Construct a comprehensive performance index based on the recall and precision; this index is the harmonic average of recall and precision. Use the following parameter combination used in the current testing task as the variables to be optimized: the gain coefficient k from step S4, and the sensitivity coefficient from step S6. The morphological structural element size and area screening threshold in step S7, and the confidence scaling factor in step S8. The confidence threshold in step S9; Record each parameter combination and its corresponding comprehensive performance index to build a performance database; when the number of records in the database reaches the preset minimum sample size, use a Gaussian process regression model to fit the mapping relationship between the parameter combination and the comprehensive performance index. The kernel function of the Gaussian process is the sum of the radial basis function and the white noise kernel. Then, based on the prediction results of the Gaussian process regression model, the expected improvement acquisition function in Bayesian optimization is used to guide the search for the next set of parameter combinations; The proposed improved acquisition function takes into account both the current optimal performance and prediction uncertainty, and its calculation process is as follows: ; in For parameter combination vectors, and Gaussian processes in The predicted mean and predicted standard deviation at the location, This represents the current optimal overall performance index value. The standard normal cumulative distribution function is... It is the standard normal probability density function; The optimal parameter combination found is automatically deployed to subsequent batches of detection tasks, enabling continuous evolution of detection performance; By collecting real defect labels from silicon wafer re-inspection, recall and precision are calculated, and the harmonic average of the two is used as a comprehensive performance index. Key parameters of the entire inspection process are used as optimization variables. Gaussian process regression is used to fit the mapping relationship between parameters and performance. Combined with the expectation of Bayesian optimization, the acquisition function is improved to automatically search for the optimal parameter combination and deploy in a closed loop. This achieves adaptive and continuous optimization of AOI inspection parameters for silicon wafer cutting, dynamically balancing the false negative rate and the false positive rate without manual parameter tuning. It continuously adapts to the fluctuations in cutting process and changes in batch characteristics, allowing the inspection accuracy and stability to continuously evolve, maximizing the long-term effectiveness of silicon wafer micro-defect detection.

[0057] The semiconductor silicon wafer cutting and sorting AOI inspection station continuously inspects the 0402 type silicon grains (1.0mm×0.5mm) obtained from silicon wafer cutting. It adopts an adaptive closed-loop optimization mechanism to automatically iterate and optimize the inspection parameters, continuously improving the micro-defect detection performance.

[0058] Preset parameters: Optimization variables are gain coefficient k, sensitivity coefficient γ, and confidence scaling factor. Confidence threshold T; overall performance metric is F1 score (harmonic average of recall R and precision P); current best overall performance ; Combination of parameters to be evaluated Gaussian process regression prediction mean Prediction standard deviation Standard normal cumulative distribution function probability density function ; Desired improvement to the acquisition function formula: ; Calculate the overall performance index F1: Collect genuine defect labels confirmed by manual re-inspection. The recall rate (R) for this test was 0.95, and the precision (P) was 0.9. The results were calculated using the harmonic mean formula. ; Store the parameter combination and F1 value in the performance database.

[0059] Gaussian process regression model fitting: After the database sample size reached the required level, a Gaussian process regression with radial basis function and white noise kernel was used to fit the mapping relationship between the parameter combination and the F1 value, resulting in... The predicted mean and standard deviation.

[0060] Desired improvements in EI value calculation: The first step is to calculate the standardized difference: ; From the standard normal distribution table, we find: ; Substitute into the EI formula to calculate: ; Parameter optimization and deployment: The optimal parameter combination is obtained by searching using the EI function. It is automatically deployed to subsequent batches for testing, improving the F1 value to 0.94 and further reducing the false negative and true positive rates, continuously adapting to fluctuations in silicon wafer cutting processes.

[0061] The micro-defect identification system for grain AOI inspection equipment includes: The data acquisition module is used to acquire two-dimensional reflection images of the grains to be inspected through the bright field and dark field optical modules of the grain AOI inspection equipment. At the same time, it uses the laser autofocus module to obtain the height distribution data of the grain surface, records the real-time suction feedback signal of the robotic arm suction cup during the process of picking up the grains, and obtains the nominal size of the grains to be inspected. The registration module is used to perform pixel-level registration between the two-dimensional reflectance image and the height distribution data to form a multi-channel input feature map. The feature extraction module is used to input the multi-channel input feature map into the shared feature extraction network and output the feature vector at each pixel location; The sensitivity dynamic adjustment module is used to dynamically adjust the anomaly detection sensitivity based on the real-time suction feedback signal and nominal size. Specifically, it adjusts the anomaly detection sensitivity according to the sensitivity adjustment coefficient: First, it calculates the theoretical suction value of the current grain based on the nominal size, which is proportional to the grain area; then, it determines the upper and lower limits of the normal suction fluctuation range based on the theoretical suction value; finally, it calculates the relative deviation of the real-time suction feedback signal from the normal range and generates a continuously changing sensitivity adjustment coefficient based on the relative deviation. The anomaly score calculation module is used to calculate the anomaly score for each feature vector using a pre-trained Gaussian mixture model. The Gaussian mixture model contains multiple Gaussian components, each of which has a mean vector and a covariance matrix. The weighted comparison module is used to multiply the anomaly score by the sensitivity adjustment coefficient to obtain the weighted anomaly score; then, a benchmark threshold is dynamically determined based on the statistical characteristics of the weighted anomaly scores of all pixels in the current detection batch; the weighted anomaly score is compared with the dynamic benchmark threshold; if the weighted anomaly score exceeds the dynamic benchmark threshold, the pixel is determined to be a suspected defect pixel; a binary image of the suspected defect pixel is generated based on the determination result, where the pixel determined to be a suspected defect has a value of 1, and the other pixels have a value of 0; The connected component analysis module is used to perform connected component analysis on the binary image of suspected defective pixels to form defect candidate regions. The morphological operation parameters and area screening thresholds in the connected component analysis are dynamically adjusted according to the nominal size: when the grain size is larger than the preset size, the size of the morphological structural element is increased and the lower and upper area thresholds are raised; when the grain size is within the preset size, the size of the structural element is decreased and the area threshold is lowered. The domain adaptive correction module is used to perform confidence correction on each defect candidate region. The domain adaptive correction module aligns the feature distribution between the current detection batch and the historical normal batch through adversarial training. The output module is used to output the final defect category and location based on the corrected defect confidence level and the confidence threshold determined by the statistical distribution of confidence levels of normal and defective samples in historical batches.

[0062] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for identifying micro-defects in a grain AOI inspection device, characterized in that, Includes the following steps: Step S1: Acquire two-dimensional reflection images of the grain to be inspected using the bright-field and dark-field optical modules of the grain AOI inspection equipment. At the same time, use the laser autofocus module to obtain the height distribution data of the grain surface and record the real-time suction feedback signal of the robotic arm suction cup during the grain picking process; obtain the nominal size of the current grain to be inspected. Step S2: Perform pixel-level registration between the two-dimensional reflection image and the height distribution data to form a multi-channel input feature map; Step S3: Input the multi-channel input feature map into the shared feature extraction network, and the shared feature extraction network outputs the feature vector at each pixel location; Step S4: Dynamically adjust the anomaly detection sensitivity based on the real-time suction feedback signal and nominal size, that is, adjust the anomaly detection sensitivity according to the sensitivity adjustment coefficient; Step S5: Calculate the anomaly score for each feature vector using a pre-trained Gaussian mixture model. The Gaussian mixture model contains multiple Gaussian components, each of which has a mean vector and a covariance matrix. Step S6: Multiply the anomaly score by the sensitivity adjustment coefficient to obtain the weighted anomaly score; then dynamically determine a benchmark threshold based on the statistical characteristics of the weighted anomaly scores of all pixels in the current detection batch; compare the weighted anomaly score with the dynamic benchmark threshold; if the weighted anomaly score exceeds the dynamic benchmark threshold, the pixel is determined to be a suspected defective pixel; and generate a binary image of the suspected defective pixel based on the determination result. Step S7: Perform connected component analysis on the binary image of suspected defective pixels to form defect candidate regions; wherein, the morphological operation parameters and area screening threshold in the connected component analysis are dynamically adjusted according to the nominal size: when the grain size is larger than the preset size, increase the size of the morphological structural element and raise the lower and upper area thresholds; when the grain size is within the preset size, decrease the size of the structural element and lower the area threshold. Step S8: Use the domain adaptive correction module to perform confidence correction on each defect candidate region. The domain adaptive correction module aligns the feature distribution between the current detection batch and the historical normal batch through adversarial training. Step S9: Based on the corrected defect confidence level and the confidence threshold determined by the statistical distribution of confidence levels of normal and defective samples in historical batches, output the final defect category and location.

2. The micro-defect identification method of the grain AOI inspection equipment according to claim 1, characterized in that: The specific process for generating the sensitivity adjustment coefficient in step S4 is as follows: First, the area of ​​the grain is calculated based on the nominal size of the grain, and the theoretical suction value is calculated according to a preset linear relationship; Then, determine the upper and lower limits of the normal suction fluctuation range; Next, the current real-time suction feedback value is compared with the normal range. If the current real-time suction feedback value is within the normal range, the relative deviation is 0; otherwise, the relative deviation is calculated. Finally, the relative deviation is converted into a sensitivity adjustment coefficient through a monotonically increasing nonlinear mapping function. The nonlinear mapping function outputs a reference value of 1 when the relative deviation is zero, and the output value gradually increases and approaches the preset maximum value as the relative deviation increases.

3. The micro-defect identification method of the grain AOI inspection equipment according to claim 2, characterized in that: In step S5, the parameters of the Gaussian mixture model are obtained through semi-supervised learning and continuous update mechanisms, specifically including the following steps: Step S51: In the initial training phase, multiple batches of defect-free grain samples that have been re-inspected are collected. These samples are then input into a shared feature extraction network to obtain the feature vector of each pixel position of each sample, forming a feature vector set. Step S52: Fit the initial Gaussian mixture model using the expectation-maximization algorithm. The number of Gaussian components is automatically determined by the Bayesian information criterion. The expectation-maximization algorithm includes the following iterative process: First, calculate the posterior probability of each feature vector belonging to each component based on the current Gaussian component parameters. Then, re-estimate the mean vector, covariance matrix, and mixing coefficients of each component based on the posterior probability until convergence. Step S53: After each subsequent inspection task is completed, if the current batch of grains is confirmed to be defect-free after re-inspection, the feature vector of the batch is used as incremental data; the weighted expectation-maximization algorithm is used to incrementally update the Gaussian mixture model, where the cumulative weight of historical data decays exponentially with time, that is, after each update, the historical weight is multiplied by a decay factor to maintain the sensitivity of the Gaussian mixture model to the current process. Step S54: During incremental update, the mean vector of the j-th Gaussian component is updated according to the following weighted average method: the sum of the historical cumulative weight and the current batch weight is used as the denominator, and the numerator is the historical cumulative weight multiplied by the mean vector before the update plus the current batch weight multiplied by the mean vector of the feature vectors belonging to the j-th component in the current batch. Step S55: The incremental update method for the covariance matrix of the j-th Gaussian component is as follows: First, calculate the covariance matrix of the eigenvectors belonging to the j-th component in the current batch. Then, use a weighted average method similar to the mean vector update to merge the historical covariance matrix with the current batch covariance matrix, and add a preset positive regularization coefficient multiplied by the identity matrix to ensure that the updated covariance matrix is ​​positive definite. Step S56: To prevent drastic changes in the parameters of the Gaussian mixture model, a parameter change constraint term is introduced during the incremental update process. The parameter change constraint term is based on the product of the sum of squares of the differences between the old and new parameters and the Fisher information matrix, and is added to the objective function of the expectation maximization algorithm as an elastic weight consolidation loss.

4. The micro-defect identification method of the grain AOI inspection equipment according to claim 3, characterized in that: The process of determining the dynamic benchmark threshold in step S6 is as follows: The weighted anomaly scores of all pixels in the current detection batch are statistically analyzed, and their arithmetic mean and standard deviation are calculated. Then, based on the arithmetic mean and standard deviation, and by introducing a sensitivity coefficient that can be configured according to the balance requirements of historical pass rate and miss rate, a dynamic baseline threshold is calculated. The larger the sensitivity coefficient, the higher the threshold, the greater the risk of miss detection but the lower the pass rate; the smaller the sensitivity coefficient, the lower the threshold, the higher the detection rate but the more miss rates.

5. The micro-defect identification method of the grain AOI inspection equipment according to claim 4, characterized in that: The connected component analysis process in step S7 is as follows: First, using the nominal size and combined with the spatial resolution parameters of the image acquisition system, the physical size of the grain is converted into the expected pixel area in the image; Then, based on the expected pixel area, the size of the structuring element for morphological closing operation is dynamically determined. Specifically, the square root of the expected pixel area is calculated, multiplied by a preset scaling factor, and then rounded down. The value obtained after rounding down is then restricted to a preset minimum value of 3 and a maximum value of 21, thus obtaining the structuring element size. Next, set the lower and upper thresholds for area filtering based on the expected pixel area: The lower threshold is equal to a preset percentage coefficient multiplied by the expected pixel area, and the upper threshold is equal to a preset percentage coefficient multiplied by the expected pixel area. Then, morphological closing operations are performed on the binary image of the suspected defective pixels using the determined structuring element size to connect neighboring isolated defective pixels; Finally, the connected regions after the closing operation are filtered by area, and connected regions with areas between the lower and upper thresholds are retained as candidate defect regions.

6. The micro-defect identification method of the grain AOI inspection equipment according to claim 5, characterized in that: The specific calculation process of the domain adaptive correction module in step S8 is as follows: The domain adaptive correction module includes a feature extractor, a domain discriminator, and a defect classifier; The feature extractor extracts feature vectors from defect candidate regions; The domain discriminator is connected to the feature extractor via a gradient inversion layer to determine whether the feature vector originates from the current detection batch or a historical normal batch. The defect classifier receives feature vectors and outputs initial defect confidence scores; During training, the loss function of the domain discriminator is the cross-entropy loss. This cross-entropy loss is backpropagated to the feature extractor after passing through the gradient reversal layer, so that the features learned by the feature extractor cannot be distinguished by the domain discriminator, thereby achieving domain alignment. The confidence score of the defect after domain adaptive correction is obtained through the following steps: First, the feature vector of the defect candidate region is input into the feature extractor to obtain the domain-invariant features; Then, the domain-invariant features are input into the defect classifier to obtain the initial confidence level; Finally, the initial confidence level is calibrated using a confidence scaling calibration method. A confidence scaling factor is introduced, which is obtained by minimizing the negative log-likelihood loss on the validation set. After calibration, the calibrated confidence level is obtained.

7. The micro-defect identification method of the grain AOI inspection equipment according to claim 6, characterized in that: The domain adaptive correction module also includes an online fast adaptation mechanism: When the detection equipment switches to a new batch of grains, a preset number of defect candidate regions with the highest abnormal scores and non-overlapping spatial distributions are automatically selected from the batch as representative samples. For representative samples, gradient updates are performed on the parameters of the last fully connected layer of the defect classifier in a limited number of steps. During the update, an optimizer with weight decay is used, and the weight decay coefficient is adaptively adjusted according to the number of samples. During the update process, the parameters of the feature extractor and the domain discriminator are frozen; After each update, calculate the confidence distribution entropy of the detected regions in the current batch. Stop updating when the entropy value no longer decreases after several consecutive iterations.

8. The micro-defect identification method of the grain AOI inspection equipment according to claim 7, characterized in that: Step S9 is followed by a defect 3D morphology discrimination step based on height distribution data: For each defect region in the final output, extract the height value matrix of the defect region in the height distribution data; A ring-shaped background area is formed by extending a predetermined number of pixels outward from the boundary of the defective area. Outliers with significantly deviated height values ​​in the ring-shaped background area are removed. Outliers are identified using a method based on interquartile range. After removing outliers, the average height of the ring-shaped background area is calculated. Then calculate the median of all pixel height values ​​within the defect area and the arithmetic mean of all pixel height values ​​within the defect area, which is the average height of the annular background area; The relative height difference is obtained by subtracting the median height of the defect area from the average height of the annular background area. Simultaneously calculate the standard deviation of the height values ​​within the defect area; If the absolute value of the relative height difference is greater than the first threshold determined based on the grain surface roughness statistics, and the height standard deviation is greater than the second threshold determined based on the height standard deviation statistics of the defect-free area on the grain surface, then the defect area is determined to be a three-dimensional structural defect, including chipping, detachment, or bulging. Otherwise, it is judged as a two-dimensional surface defect, which includes scratches, dirt, or ink spots; after the judgment is completed, the three-dimensional morphological information of the defect is output as an additional attribute.

9. A micro-defect identification system for a grain AOI inspection device, wherein the system is applied in the method described in any one of claims 1-8, characterized in that: The system includes; The data acquisition module is used to acquire two-dimensional reflection images of the grains to be inspected through the bright field and dark field optical modules of the grain AOI inspection equipment. At the same time, it uses the laser autofocus module to obtain the height distribution data of the grain surface, records the real-time suction feedback signal of the robotic arm suction cup during the process of picking up the grains, and obtains the nominal size of the grains to be inspected. The registration module is used to perform pixel-level registration between the two-dimensional reflectance image and the height distribution data to form a multi-channel input feature map. The feature extraction module is used to input the multi-channel input feature map into the shared feature extraction network and output the feature vector at each pixel location; The sensitivity dynamic adjustment module is used to dynamically adjust the anomaly detection sensitivity based on the real-time suction feedback signal and nominal size. That is, the anomaly detection sensitivity is adjusted according to the sensitivity adjustment coefficient: First, the theoretical suction value of the current grain is calculated based on the nominal size. This theoretical suction value is proportional to the grain area. Then, the upper and lower limits of the normal suction fluctuation range are determined based on this theoretical suction value. Finally, the relative deviation of the real-time suction feedback signal from the normal range is calculated, and a continuously changing sensitivity adjustment coefficient is generated based on the relative deviation. The anomaly score calculation module is used to calculate the anomaly score for each feature vector using a pre-trained Gaussian mixture model. The Gaussian mixture model contains multiple Gaussian components, each of which has a mean vector and a covariance matrix. The weighted comparison module is used to multiply the anomaly score by the sensitivity adjustment coefficient to obtain the weighted anomaly score; then, a benchmark threshold is dynamically determined based on the statistical characteristics of the weighted anomaly scores of all pixels in the current detection batch; the weighted anomaly score is compared with the dynamic benchmark threshold; if the weighted anomaly score exceeds the dynamic benchmark threshold, the pixel is determined to be a suspected defective pixel; and a binary image of the suspected defective pixel is generated based on the determination result. The connected component analysis module is used to perform connected component analysis on the binary image of suspected defective pixels to form defect candidate regions. The morphological operation parameters and area screening thresholds in the connected component analysis are dynamically adjusted according to the nominal size: when the grain size is larger than the preset size, the size of the morphological structural element is increased and the lower and upper area thresholds are raised; when the grain size is within the preset size, the size of the structural element is decreased and the area threshold is lowered. The domain adaptive correction module is used to perform confidence correction on each defect candidate region. The domain adaptive correction module aligns the feature distribution between the current detection batch and the historical normal batch through adversarial training. The output module is used to output the final defect category and location based on the corrected defect confidence level and the confidence threshold determined by the statistical distribution of confidence levels of normal and defective samples in historical batches.

Images