Image preprocessing data processing method and system for small defects on surface of semiconductor equipment component

Abstract

The invention provides a semiconductor device part surface micro defect image preprocessing data processing method and system, and relates to the technical field of data processing, and the method comprises the steps: carrying out the image quality normalization processing of an obtained high-magnification amplification surface original image, and generating a quality enhancement image; performing defect feature selective enhancement processing on the quality enhanced image to generate a feature optimized image; performing three-dimensional shape reconstruction on the feature optimization image to generate surface three-dimensional shape topological data containing height information; on the basis of the surface three-dimensional topography topological data, a topography reference origin is determined at the highest point of a local topography protrusion of the preliminary defect candidate area, and a first feature radiation vector and a second feature radiation vector are generated in a fitting mode from the topography reference origin in the main direction and the normal change gradient direction of the surface texture respectively; according to the invention, precise characterization of micro defects in a high-magnification magnification scene can be realized, and the accuracy and real-time performance of defect detection are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for preprocessing images of minute defects on the surface of semiconductor device components. Background Technology

[0002] In the field of semiconductor manufacturing equipment, semiconductor ferrule connectors, as core components for gas path connections, directly determine the operational stability of the equipment due to their surface airtightness. To ensure component performance, micron-level surface defects must be detected under 150x high magnification. The first step in this detection process is image preprocessing, and the accuracy of the preprocessing results directly determines the accuracy of subsequent defect identification.

[0003] Current mainstream image preprocessing methods for minute defects generally suffer from the following technical limitations: relying solely on two-dimensional grayscale information and edge features for defect characterization fails to eliminate the shape distortion caused by the combined effects of microscopic morphological fluctuations in components and inherent optical system distortions during high-magnification imaging. This limitation leads to the following consequences. For example, in a 150x magnification inspection scenario for semiconductor ferrule connectors, shape distortion causes deviations between the two-dimensional image contours of tiny scratches and pinholes and the actual three-dimensional shape. Components that meet the size standards may be misjudged as defective, and actual micron-level defects may be missed due to contour distortion, directly affecting the accuracy of component quality control. Moreover, the detection errors caused by shape distortion force the production line to add manual re-inspection steps. This not only reduces the real-time performance of online inspection but also increases inspection costs, failing to meet the efficiency requirements of large-scale production.

[0004] In existing technologies, some preprocessing schemes attempt to optimize image quality by improving the accuracy of filtering algorithms, while others enhance defect edge features by increasing contrast. However, these schemes still cannot fundamentally solve the problem of accurately representing tiny defects in high-magnification scenes. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for preprocessing images of minute defects on the surface of semiconductor equipment components, so as to achieve accurate characterization of minute defects in high magnification scenarios, ensure the accuracy and real-time performance of defect detection, and adapt to the large-scale production and testing needs of precision components such as semiconductor ferrule connectors.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] A first aspect is a method for preprocessing image data of minute defects on the surface of semiconductor device components, the method comprising:

[0008] The acquired high-magnification magnified original surface image is subjected to image quality normalization processing to generate a quality-enhanced image;

[0009] Selective enhancement of defect features is performed on the quality-enhanced image to generate a feature-optimized image;

[0010] Three-dimensional topography reconstruction is performed on the feature-optimized image to generate surface three-dimensional topography data containing height information;

[0011] Based on the surface three-dimensional topological data, a topological reference origin is established at the highest point of the local topological protrusion in the preliminary defect candidate region. Starting from the topological reference origin, the first feature radiation vector and the second feature radiation vector are respectively fitted and generated along the main direction of the surface texture and the normal gradient direction.

[0012] Based on the first and second feature radiation vectors, a multi-directional feature analysis sector is defined. Within the adjacent reference area inside and outside the multi-directional feature analysis sector, a set of topographic sampling points arranged according to polar coordinates are deployed.

[0013] Based on the three-dimensional spatial coordinates of the topography sampling points, a closed-loop evaluation trajectory is constructed around the origin of the topography reference point. The micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory are calculated.

[0014] By integrating the micro-surface curvature distribution matrix with the local topological variability, a morphology distortion quantification factor characterizing the degree of local morphology distortion is generated.

[0015] By using a shape distortion quantization factor, the geometric representation of defects in the corresponding region of the feature-optimized image is compensated and calibrated, and the final preprocessed image data after shape distortion correction is output.

[0016] Secondly, a data processing system for preprocessing images of minute defects on the surface of semiconductor device components includes:

[0017] The optimization processing module is used to perform image quality normalization processing on the acquired high-magnification magnified surface original image to generate a quality-enhanced image; and to perform defect feature selective enhancement processing on the quality-enhanced image to generate a feature-optimized image.

[0018] The 3D reconstruction module is used to reconstruct the 3D topography of the feature-pair optimized image, generating 3D topographic data of the surface topography containing height information.

[0019] The module for establishing and fitting is used to establish a topographic reference point at the highest point of the local topographic protrusion in the preliminary defect candidate region based on the surface three-dimensional topographic data. Starting from the topographic reference point, the first feature radiation vector and the second feature radiation vector are respectively fitted and generated along the main direction of the surface texture and the normal gradient direction.

[0020] The definition and deployment module is used to define a multi-directional feature analysis sector based on the first feature radiation vector and the second feature radiation vector, and to deploy a set of topographic sampling points arranged according to polar coordinates within the adjacent reference area inside and outside the multi-directional feature analysis sector.

[0021] The construction and calculation module is used to construct a closed-loop evaluation trajectory around the origin of the topography reference point based on the three-dimensional spatial coordinates of the topography sampling points, and to calculate the micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory.

[0022] The fusion generation module is used to fuse the micro-surface curvature distribution matrix and the local topological variability to generate a morphology distortion quantification factor that characterizes the degree of local morphology distortion.

[0023] The compensation and calibration module is used to compensate and calibrate the defect geometric representation of the corresponding region in the feature-optimized image by using the shape distortion quantization factor, and output the final preprocessed image data after shape distortion correction.

[0024] The above-described solution of the present invention has at least the following beneficial effects:

[0025] Because it adopts a preprocessing workflow that integrates three-dimensional topography reconstruction, it obtains optimized images through image quality normalization and selective enhancement of defect features. It then generates three-dimensional topographic data by combining structured light phase analysis and photometric stereo vision. After establishing the reference origin, setting up polar coordinate sampling points, and analyzing micro-surface curvature and topological variability to generate a topography distortion quantification factor, it finally achieves defect geometric characterization calibration. Therefore, it effectively overcomes the technical problem that traditional two-dimensional preprocessing cannot eliminate topographic distortion in high-magnification imaging. This allows for accurate characterization of micron-level defects, ensuring the accuracy and real-time performance of subsequent inspections, reducing the cost of manual re-inspection, and adapting to the large-scale production inspection needs of precision components such as semiconductor ferrule connectors. Attached Figure Description

[0026] Figure 1 This is a schematic flowchart of a method for preprocessing image data of minute defects on the surface of semiconductor device components provided in an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of a semiconductor device component surface micro-defect image preprocessing data processing system provided in an embodiment of the present invention. Detailed Implementation

[0028] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0029] like Figure 1 As shown, embodiments of the present invention propose a method for preprocessing image data of minute defects on the surface of semiconductor device components. The method includes the following steps:

[0030] Step 1: Perform image quality normalization processing on the acquired high-magnification original image of the surface to generate a quality-enhanced image;

[0031] Step 2: Perform selective enhancement of defect features on the quality-enhanced image to generate a feature-optimized image;

[0032] Step 3: Perform 3D topography reconstruction on the feature-optimized image to generate surface 3D topography data containing height information;

[0033] Step 4: Based on the surface three-dimensional topology data, establish a topology reference origin at the highest point of the local topology protrusion in the preliminary defect candidate region. Starting from the topology reference origin, fit and generate the first feature radiation vector and the second feature radiation vector along the main direction of the surface texture and the normal gradient direction, respectively.

[0034] Step 5: Based on the first feature radiation vector and the second feature radiation vector, define a multi-directional feature analysis sector. Within the adjacent reference area inside and outside the multi-directional feature analysis sector, set up a set of topographic sampling points arranged according to polar coordinates.

[0035] Step 6: Based on the three-dimensional spatial coordinates of the topography sampling points, construct a closed-loop evaluation trajectory around the topography reference origin, and calculate the micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory.

[0036] Step 7: Combine the micro-surface curvature distribution matrix with the local topological variability to generate a morphological distortion quantification factor that characterizes the degree of local morphological distortion.

[0037] Step 8: Using the shape distortion quantization factor, compensate and calibrate the defect geometric representation of the corresponding region in the feature optimization image, and output the final preprocessed image data after shape distortion correction.

[0038] In this embodiment of the invention, the basic quality of the original image is improved by image quality normalization, and the distinction between defects and background is enhanced by selective enhancement of defect features. The surface height information of the component is obtained by combining three-dimensional topography reconstruction. The defect analysis area is accurately located by establishing a reference origin, fitting feature radiation vectors, and setting up polar coordinate sampling points. The micro-surface curvature distribution matrix and local topological variability calculated based on the closed-loop evaluation trajectory can accurately quantify the degree of topographic distortion. Then, a distortion quantification factor is used to achieve targeted calibration of the defect geometric characterization. This method effectively solves the problem that traditional two-dimensional preprocessing cannot eliminate topographic distortion in high-magnification imaging, improves the accuracy of micro-defect characterization, provides reliable image data support for defect detection, ensures detection accuracy, adapts to the online detection needs of semiconductor precision components, and helps improve the efficiency of production quality control.

[0039] In a preferred embodiment of the present invention, step 1 above may include:

[0040] Step 1.1 involves performing a global illumination intensity analysis on the acquired high-magnification original surface image to generate a background illumination distribution model. Specifically, this includes receiving a 150x high-magnification original image of a semiconductor ferrule connector. Addressing practical application scenarios such as differences in illuminance between the center and edge of the optical lens, uneven reflection between the polished and processed surfaces of the connector, and shifts in the angle of the light source at different workstations—all issues that easily arise during high-magnification imaging of precision semiconductor components—a background illumination distribution model based on an improved lightweight U-Net architecture is used to complete the global illumination intensity analysis. This model is an improvement upon the traditional U-Net segmentation architecture. While conventional methods are often used for medical image segmentation, their symmetrical encoding and decoding structure can efficiently extract multi-scale features. However, they suffer from problems such as a large number of parameters, slow inference speed, and weak ability to capture features of local illumination changes, making them unsuitable for the real-time and accuracy requirements of online detection of semiconductor components. This invention addresses these shortcomings by replacing the standard convolution at the encoding end with a depthwise separable convolution, reducing the number of parameters by 70% while maintaining feature extraction capabilities. Simultaneously, a pixel-level attention module is embedded after each upsampling module at the decoding end to strengthen the feature extraction weights in areas of illumination change and suppress feature redundancy in defect-free, uniformly illuminated areas.

[0041] The specific construction and training process of the model is as follows: In the model construction stage, firstly, 150x magnified images of the defect-free areas of semiconductor ferrule connectors from different batches, different production stations, and different lighting conditions are collected to build an initial sample library. Blurred samples caused by lens contamination and mechanical vibration are removed, and the effective sample size is finally determined to be 8000 images. The sample images are divided into several non-overlapping local image blocks according to the principle of taking into account both the stability of local lighting features and the capture of subtle lighting changes. Four types of lighting feature parameters, namely gray-level mean, variance, gray-level distribution entropy, and local gradient magnitude, are extracted from each image block and used as the input features of the model. The output layer of the model is designed as a single-channel gray-level prediction matrix with the same size as the input image. Each element in the matrix corresponds to the predicted value of the lighting intensity of the original image pixel, thereby realizing pixel-level lighting distribution representation. At the same time, two cross-scale feature fusion branches are added between the encoding and decoding ends to fuse shallow detailed lighting features with deep global lighting features, thereby improving the model's ability to perceive subtle lighting differences.

[0042] During the model training phase, 8000 samples were divided into training, validation, and test sets in a 7:2:1 ratio. The training set was used for iterative optimization of model parameters, the validation set was used to monitor overfitting during training, and the test set was used for final model performance evaluation. A noise-free illumination baseline image obtained by fusion of multiple defect-free images of the same joint and location was used as a label. This label can accurately represent the true illumination distribution on the joint surface and avoid the interference of accidental noise from single images. A weighted combination of the L1 loss function and the structural similarity loss function (SSIM) was used as the loss function. The L1 loss function was used to constrain the pixel-level illumination prediction value from the label. The absolute error of the signature and the structural similarity loss function are used to constrain the spatial structural consistency of the illumination distribution. The weighting coefficients of the two are set to 0.7:0.3. The Adam optimizer is used for parameter iteration, with an initial learning rate of 0.001. A cosine annealing strategy is used to dynamically adjust the learning rate to avoid the model getting trapped in local optima. The training batch size is set to 16. After each training round, the illumination prediction error and structural similarity index of the validation set are calculated. When the illumination prediction error of the validation set is below 0.8% for 10 consecutive rounds and the structural similarity index is above 0.98, training is stopped, the optimal model parameters are saved, and the construction of the background illumination distribution model is completed.

[0043] After training, the high-magnification original image of the semiconductor card connector to be processed is input into the model. The model extracts multi-scale illumination features through the encoder, integrates deep and shallow features through a cross-scale feature fusion branch, and then focuses on the uneven illumination area by the pixel-level attention module of the decoder, outputting an illumination intensity prediction matrix with the same size as the original image. Subsequently, a bilinear interpolation algorithm is used for post-processing. The specific process is as follows: first, the boundary regions of adjacent image blocks in the model's output prediction matrix are located, and the illumination prediction values ​​of 3 to 5 pixels on each side of the boundary are extracted as constraint benchmarks. Then, for the pixels in the boundary transition region, the illumination completion value of the transition region is calculated by bilinear interpolation with the distance between the pixels and the boundary pixels of the adjacent blocks as the weight, ensuring that the difference between the completion value and the pixel values ​​of the two boundary sides is controlled within a preset threshold to avoid gray-scale abrupt discontinuities between blocks. After all adjacent block boundaries have been interpolated and completed, a complete global background illumination distribution model is obtained. This model has both pixel-level prediction accuracy and global illumination transition smoothness.

[0044] The advantages of using this improved lightweight U-Net architecture background illumination distribution model are as follows: First, the lightweight design significantly improves inference speed, with the model's inference speed increased by more than 45% compared to the traditional U-Net. The inference time for a single 2048×2048 pixel image is controlled within 20ms, meeting the frame rate requirements of more than 50fps for online detection of semiconductor components. Second, the pixel-level attention module and cross-scale feature fusion branch at the decoding end can accurately capture local illumination abrupt changes under high-magnification imaging. Compared with traditional statistical illumination models, the accuracy of characterizing illumination differences caused by optical distortion and uneven material reflection is improved by more than 38%. Third, the model is trained on a large number of industrial samples under different working conditions, possessing strong generalization ability. It does not require retraining the model for connectors of different batches and workstations, reducing the debugging cost of field applications. Fourth, the model trained by combining L1 and structural similarity loss functions outputs illumination distribution that meets pixel-level accuracy requirements while maintaining spatial structural consistency, providing a reliable benchmark for subsequent pixel-by-pixel illumination compensation.

[0045] Step 1.2: Based on the background illumination distribution model, perform pixel-by-pixel illumination intensity compensation on the original surface image to obtain an illumination homogenized image. Specifically, this includes: retrieving a 150x magnified image of a defect-free semiconductor ferrule connector from the same batch as a standard sample; calculating the global grayscale mean of the standard sample and setting it as a preset standard illumination intensity value; extracting the coordinate information of each pixel in the original image according to the line-by-line scanning order; retrieving the predicted illumination intensity value of the corresponding coordinate from the generated background illumination distribution model; calculating the difference between the predicted illumination intensity value and the preset standard illumination intensity value; using this difference as the illumination compensation amount for the current pixel; when the predicted illumination intensity value is higher than the standard value, the compensation amount is negative; when the predicted illumination intensity value is lower than the standard value, the compensation amount is positive; threshold clipping of the compensation amount to avoid the grayscale value of the compensated pixel exceeding the normal display range; superimposing the clipped compensation amount onto the original grayscale value of the corresponding pixel to complete single-pixel illumination calibration; traversing all pixels of the original image to complete the calibration operation; and outputting an illumination homogenized image to eliminate the interference of local overbrightness or underbrightness in high-magnification imaging on the observation of minute defects.

[0046] Step 1.3: Based on the gray-level statistical characteristics of each local region in the homogenized illumination image, dynamically calculate and apply an adaptive contrast stretching coefficient to generate a contrast-normalized image. Specifically, this includes: combining the size characteristics of minute defects such as ≥3μm pinholes and ≥5μm scratches on the surface of the semiconductor card connector, dividing the local regions of the homogenized illumination image according to the principle of completely covering a single minute defect without diluting local contrast features. The region size must match the defect scale to avoid excessive statistical noise due to an excessively small region, or the gray-level difference between the defect and the background being averaged due to an excessively large region. Perform a global gray-level traversal on each local region, collecting core feature parameters such as maximum, minimum, mean, variance, skewness, and kurtosis. Variance is used to quantify the degree of gray-level difference between the defect and the background, while skewness and kurtosis help determine the asymmetry and steepness of the gray-level distribution: if the skewness deviates significantly from 0, it indicates a local gray-level abrupt change within the region, which is likely due to a defect. A high kurtosis indicates that the grayscale is concentrated in a specific range, either in the background or a defect. A difference judgment threshold is set based on the grayscale variance. When the variance is less than the threshold, the grayscale difference between the defect and the background is considered weak. In this case, a stretching coefficient is dynamically calculated based on the grayscale range of the region; the smaller the grayscale range, the larger the stretching coefficient, ensuring that the difference is effectively amplified. When the variance is greater than or equal to the threshold, the difference is considered sufficiently significant, and a gentle stretching coefficient close to 1 is set to avoid excessive stretching that could distort the details of the defect edges. The corresponding stretching coefficient is applied pixel-by-pixel to the grayscale values ​​within the region. After adjustment, grayscale continuity is verified for each region to ensure no abnormal grayscale jumps. Finally, all regions are integrated to generate a contrast-normalized image, significantly improving the grayscale difference between the small defect and the joint substrate, providing a clear feature basis for subsequent defect edge localization. The division size and grayscale statistical characteristics of the aforementioned local regions can be adjusted according to the defect type or imaging conditions.

[0047] Step 1.4 involves applying guided smoothing filtering to the contrast-normalized image. This smooths the inherent texture background while preserving the integrity of structural edges in the image, resulting in an enhanced image. Specifically, this includes: using an edge-aware guided smoothing filtering algorithm to process the contrast-normalized image. The core objective is to separate inherent texture noise such as processing lines and material graininess from minor defect edges. First, edge detection is performed on the image, calculating the gray-level gradient values ​​of each pixel in the horizontal and vertical directions. Then, the comprehensive gradient magnitude is obtained using the gradient magnitude formula, constructing a gray-level gradient guidance map with the same size as the original image. A larger gradient magnitude indicates a more drastic gray-level change at that location, and a higher probability of it being a defect edge. The selection of the filtering kernel needs to balance noise smoothing with edge preservation. For high accuracy, small rectangular kernels are preferred to avoid edge blurring caused by large kernels. During the filtering process, the gray-level gradient guide map is used as the basis for weight allocation: for regions with gradient magnitudes less than a preset threshold, they are identified as inherent texture backgrounds, and the weight of the filtering kernel is increased, with noise weakened through multiple smoothing operations; for regions with gradient magnitudes greater than or equal to the preset threshold, they are identified as potential defect edges, and the weight of the filtering kernel is reduced, with only light smoothing or preservation of the original gray-level values ​​to ensure that the edge structure is not eroded; after filtering, the image edge integrity is checked. By comparing the gradient magnitude changes of defect edges before and after filtering, it is confirmed that the edge contours are without missing or broken parts, and finally, a quality-enhanced image with low noise and clear edges is output, providing a high-quality 2D image input foundation for subsequent 3D topography reconstruction.

[0048] In a preferred embodiment of the present invention, step 2 above may include:

[0049] Step 2.1 involves multi-scale gradient feature extraction of the enhanced image to generate gradient feature maps containing edge information at different scales. Specifically, this includes receiving the enhanced image and initiating a multi-scale gradient feature extraction process to detect the edges of small defects of different sizes, such as pinholes (3-5 micrometers) and scratches (5-10 micrometers), on the surface of the semiconductor card connector. First, the gradient extraction scale range is automatically matched based on the defect size. Small scales correspond to the fine edges of 3-5 micrometer pinholes, medium scales to the edges of medium-sized scratches (5-10 micrometers), and large scales to the extended areas of defects larger than 10 micrometers. Each scale is precisely matched with an appropriate gradient extraction operator. Small-sized operators are selected at low scales to preserve fine edge details, while slightly larger-sized operators are selected at medium and large scales to capture complete edge contours. The image is traversed pixel by pixel by operators at each scale, and the gradient response in the horizontal and vertical directions is calculated to generate a gradient magnitude map containing edge strength information and a gradient direction map containing edge direction information. Then, multi-scale feature fusion is performed, and weights are assigned based on the defect scale priority. Small-scale gradients are given 60% weight to highlight small defect details, medium-scale gradients are given 30% weight to ensure the integrity of medium-sized defect edges, and large-scale gradients are given 10% weight to suppress noise interference. A fused gradient feature map is generated by weighted superposition to ensure that no defect edge information is missed at all scales.

[0050] Step 2.2: Based on the gradient feature map, locate the edge structure of potential defects on the quality-enhanced image and generate the corresponding edge structure mask. Specifically, this includes: loading the gradient feature map; locating the peak range of the gradient amplitude by statistically analyzing the frequency distribution curve of the gradient amplitude; using half of the upper limit of the peak range as the initial edge discrimination threshold; and then adjusting the threshold through iterative verification to ensure that the threshold can accurately distinguish between real defect edges and residual texture noise; performing binarization processing on the gradient feature map based on this threshold; marking pixels with gradient amplitudes higher than the threshold as 1 (potential edge pixels) and pixels with gradient amplitudes lower than the threshold as 0 (background pixels); and addressing potential defects after binarization. To address edge breakage and isolated noise issues, a morphological processing flow is initiated in the order of dilation followed by erosion. First, one to two dilation operations are performed to fill edge gaps and connect broken fragments with small-sized structuring elements. Then, one erosion operation is performed to remove edge redundancy and isolated micro-noise introduced during the dilation process, resulting in a complete and clean binary edge image. Based on the pixel coordinate mapping of this binary image, the contour range of potential defect edges is accurately located on the quality-enhanced image, generating a single-channel edge structure mask with the same size as the quality-enhanced image. The marked regions in the mask strictly correspond to the potential defect edges, providing accurate pixel-level region localization for subsequent targeted enhancement.

[0051] Step 2.3: Based on the edge structure mask, perform mask-guided local contrast enhancement on the quality-enhanced image to strengthen the grayscale difference between defects and the background in the mask-covered area, generating a locally contrast-enhanced image. Specifically, this includes: calling the generated edge structure mask and dividing the quality-enhanced image into a defect-concerned region (mask 1 marked area) and a background region (mask 0 marked area) according to the mask pixel markings; for the defect-concerned region, automatically dividing it into tiny sub-blocks based on edge connectivity. The sub-block size is set according to the standard of covering a single connected edge plus 5 to 8 surrounding background pixels, ensuring that each sub-block can completely contain the local defect and a small amount of background, avoiding... To avoid insufficient enhancement targeting due to excessively large sub-blocks, we first statistically analyze features such as grayscale mean and variance for each sub-block, and then dynamically adjust the contrast enhancement intensity based on these features. Sub-blocks with small grayscale differences receive slightly higher enhancement intensity, while sub-blocks with moderate grayscale differences receive moderate enhancement intensity. We maintain the original grayscale features of the background area without processing to avoid over-enhancing and amplifying texture noise. After enhancement, we perform grayscale linear interpolation processing on the 3-5 pixel width of the transition band between the defect area of ​​interest and the background area to eliminate grayscale jump traces after region fusion. Finally, we generate a locally contrast-enhanced image to ensure a natural transition between the defect area and the background and to make the defect features more prominent.

[0052] Step 2.4 involves performing adaptive edge sharpening on the local contrast-enhanced image to improve the clarity and continuity of the defect contour within the edge structure mask, and outputting a feature-optimized image. Specifically, this includes: receiving the local contrast-enhanced image; using the edge structure mask as a guide, performing adaptive edge sharpening only on the defect contour region marked by mask 1; firstly, constructing a 3×3 pixel local neighborhood within the defect contour region; calculating the gradient magnitude of each pixel within the neighborhood; dividing the region into strong and weak regions based on the gradient magnitude; regions with gradient magnitudes higher than a preset threshold are considered clear edge regions, while regions with gradient magnitudes lower than the threshold are considered blurred edge regions; and then further processing the image for different... The sharpening intensity is dynamically adjusted within each region. A low-intensity sharpening kernel is used in clear edge areas to only slightly enhance edge contrast and avoid sharpening artifacts such as white edges. A medium-intensity sharpening kernel is used in blurred edge areas to increase the difference in edge grayscale gradient to enhance clarity. No sharpening operation is performed on the background area outside the mask to prevent amplification of texture noise. After sharpening, the continuity of the defect contour is verified by edge connectivity detection, and the integrity is verified by comparison with the original edge structure mask. After confirming that there are no edge breaks, distortions, or artifacts, the feature-optimized image is output, providing a high-quality 2D input with clear edges and complete features for subsequent 3D shape reconstruction.

[0053] In a preferred embodiment of the present invention, step 3 above may include:

[0054] Step 3.1 involves performing structured light phase analysis on the feature-optimized image to extract the phase gradient field characterizing the micro-undulations of the surface. Specifically, this includes: receiving the output feature-optimized image; synchronously calling the synchronization signals of the structured light projection module and the image acquisition module to ensure precise timing matching between structured light projection and image acquisition, thereby obtaining a structured light modulated image of the semiconductor ferrule connector surface; for the detection requirements of micro-undulation features such as 3-5 micrometer pinholes and 5-10 micrometer scratches on the semiconductor ferrule connector surface, initiating a structured light phase analysis process adapted to the micro-scale; automatically extracting corner feature points from the feature-optimized image and the structured light modulated image, then performing registration and alignment operations on the two images to correct displacement deviations to the sub-pixel level, completely eliminating displacement interference caused by mechanical vibration during image acquisition; and for fine stripes... By analyzing the grayscale variation patterns of structured light modulated images, the modulation phase carrying surface undulation information is separated. Considering that microscopic defects can cause abrupt phase changes, a quality assessment criterion characterizing phase reliability is first constructed. Using the phase difference between adjacent pixels as a constraint, the existing phase ambiguity is gradually corrected to eliminate the 2π periodic ambiguity problem of the modulation phase, resulting in a continuous and distortion-free absolute phase distribution. The obtained absolute phase distribution is then subjected to local neighborhood smoothing. The neighborhood window size is set according to the needs of adapting to microscopic undulations, and high-frequency phase noise introduced by uneven reflection of the surface metal material is eliminated. Finally, the phase gradient field characterizing the microscopic undulation features of the joint surface is extracted. The variation trend of this phase gradient field is linearly correlated with the surface height undulation, providing accurate phase basis data for subsequent 3D reconstruction.

[0055] Step 3.2: Based on the extracted phase gradient field, an initial 3D point cloud seed set for the surface height is generated through integral reconstruction calculation. Specifically, this includes: loading the extracted phase gradient field characterizing the microscopic undulations of the semiconductor ferrule surface; simultaneously calling a preset high-precision calibration parameter package, which includes camera intrinsic parameters, structured light projection angle parameters, and phase height mapping coefficients. The camera intrinsic parameters are used to correct for lens distortion during imaging; the structured light projection angle parameters relate the fringe projection direction to the surface phase change; and the phase height mapping coefficients are obtained through multiple calibration experiments using a standard height template (covering standard undulation scales from 0 to 50 micrometers). All parameters are stored in batches to ensure compatibility with the detection requirements of different specifications of semiconductor ferrules. The initial 3D reconstruction process is then initiated, first performing a consistency check on the phase gradient field in both horizontal and vertical directions, and setting the gradient direction change rate threshold and gradient amplitude. The system employs a dual-criteria system for determining abrupt changes. When the gradient direction change of adjacent pixels exceeds the direction threshold, or the gradient magnitude abruptly exceeds the magnitude threshold, it is identified as an abnormal region. After automatically marking the range of the abnormal region, the system uses the weighted average of the effective gradients within a 3×3 neighborhood around the abnormal region for completion. Gradients closer to the abnormal point have higher weights, ensuring that the completed gradient field has no abrupt changes and conforms to the overall trend of surrounding micro-undulations. A serpentine integration path is selected from the upper left to the lower right corner of the image. Specifically, odd-numbered rows integrate from left to right, and even-numbered rows integrate from right to left. This path can offset the cumulative error generated by single-row unidirectional integration through reverse integration. When performing integration on the phase gradient along the path, the camera intrinsic parameters are simultaneously called to correct the pixel coordinate distortion, ensuring the accuracy of the integration reference. Then, combined with the phase height mapping coefficient, the phase value is converted to the physical height value, yielding the initial micron-level surface height value corresponding to each pixel.

[0056] The initial height values ​​undergo multi-level validity screening. The first level, based on statistical data of surface heights from a large batch of normal semiconductor ferrule connectors, sets a dynamic height threshold range to avoid false screening due to differences in normal heights between different batches. The second level performs local neighborhood verification on the pre-screened height values, determining whether the deviation of each height value from its 3×3 neighborhood is within the allowable range, and eliminating isolated extreme points caused by local phase noise. For valid height data that passes both levels of screening, its corresponding sub-pixel-level two-dimensional coordinates are associated, inheriting the sub-pixel accuracy from the registration in step 3.1, forming a bond between sub-pixel two-dimensional coordinates and micrometer-level height. The key-value pairs are processed, and a reliability tag is added to each key-value pair. The deviation is determined based on the magnitude of the neighborhood verification. All valid key-value pairs are sorted in order of row priority and column priority of pixel coordinates. Normal areas with gentle gradient changes are sparsely sampled at certain intervals, while all valid data are retained for potential defect areas with drastic gradient changes. Finally, an initial 3D point cloud seed set containing key undulation features of the joint surface is generated. Although the seed set is sparsely distributed overall, it maintains a high data density in the defect area, fully preserving the core undulation features of the defect area and the normal area. The attached reliability tag can assist in the weight allocation of subsequent iterative optimization.

[0057] Step 3.3, based on the principle of photometric stereo vision, uses feature-optimized images and the initial 3D point cloud seed set for iterative optimization to correct height data errors and fill missing areas, generating a dense 3D point cloud model. Specifically, this includes: using the principle of photometric stereo vision as the core technology support, the core logic of which is to acquire multiple grayscale images of the same object under a fixed camera viewpoint and different light source angles, and use the reflection mapping relationship between the grayscale value of the object surface and the light source direction and surface normal vector to infer the normal vector distribution of the object surface, and then combine the normal vector integral to obtain the surface height information; in the semiconductor card sleeve connector inspection scenario, since the connector surface is made of metal, its grayscale value will show significant differences with the light source angle, and tiny defects such as 3 to 5 micrometer pinholes and 5 to 10 micrometer scratches will disrupt the continuity of surface normal vectors, resulting in a significant difference in the grayscale change pattern between the defect area and the normal area. The principle of photometric stereo vision can accurately capture this difference, providing a core basis for the 3D morphology reconstruction of tiny defects.

[0058] Based on this principle, the system calls upon the parameter information (including the direction vector and light intensity parameters of each light source) of the preset multi-angle light source module and the image data of the semiconductor ferrule connector surface collected at the corresponding angles. An improved PointNet architecture-based dense 3D point cloud generation model is used to complete the densification optimization and error correction of the initial 3D point cloud seed set. This model is an improvement on the traditional PointNet point cloud processing architecture. Traditional PointNet can only extract the spatial coordinate features of the point cloud, and has shortcomings such as insufficient ability to extract sparse point cloud features and failure to integrate the 2D grayscale variation features generated by the photometric stereo vision principle, making it unable to accurately adapt to the 3D reconstruction needs of minute defects. Therefore, this invention specifically adds a grayscale and height correlation feature branch and a neighborhood feature fusion module. The grayscale and height correlation feature branch specifically extracts the grayscale variation features under multi-angle light sources and establishes a mapping with the point cloud height features. The neighborhood feature fusion module strengthens the local feature correlation of the defect area, ultimately achieving accurate reconstruction of the 3D morphology of minute defects on the semiconductor ferrule connector surface. The specific construction and training process of the model is as follows:

[0059] In the model building phase, a training dataset containing over 10,000 sets of semiconductor ferrule connectors under different operating conditions was constructed. Each dataset covers three core elements: first, surface grayscale images under three or more different light sources, covering common light source angle deviations in production lines; second, an initial sparse point cloud seed set obtained through integral reconstruction; and third, high-precision real surface height data calibrated by a laser interferometer as label data to ensure height accuracy benchmarks. The dataset comprehensively covers typical defects such as 3-5 micrometer pinholes and 5-10 micrometer scratches, as well as material reflectivity differences between different batches of connectors. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for iterative optimization of model parameters, the validation set was used to monitor overfitting during training, and the test set was used for final model performance evaluation. Based on the traditional PointNet architecture, targeted improvements are made by adding a dual-input branch. The first branch takes into account the spatial coordinate features of the point cloud seed set and extracts the global spatial distribution features of the point cloud through a one-dimensional convolutional layer. The second branch takes into account the gray-level gradient features of the optimized image and the gray-level difference features corresponding to multiple light sources, and extracts the gray-level abrupt change features of defect areas through a two-dimensional convolutional layer. A neighborhood feature fusion module is added at the dual-branch feature output end. This module uses the pixel coordinates of the point cloud as the link to fuse the spatial features within the neighborhood of each point cloud with the gray-level change features of the corresponding region pixel by pixel, enhancing the feature representation capability of small defect areas and avoiding the distortion of defect reconstruction caused by feature fragmentation. The model output layer is designed as a joint prediction layer, which simultaneously outputs two types of core data: the height correction of each point in the sparse point cloud seed set and the grid height filling value of missing areas (areas without data due to shadow occlusion or excessive reflection), realizing the integrated output of error correction and region filling, and improving processing efficiency.

[0060] During model training, a weighted combined loss function was used. The height error loss weight was set to 0.6, constraining the model's height reconstruction accuracy by calculating the absolute deviation between the predicted height and the true height calibrated by the laser interferometer. The grayscale prediction error loss weight was set to 0.4, constraining the deviation between the theoretical grayscale value generated based on the predicted height and light source parameters and the actual acquired grayscale value, ensuring that the predicted height conforms to the reflection law of photometric stereo vision, balancing height accuracy and grayscale feature consistency. The Adam optimizer was used for parameter iteration, with an initial learning rate of 0.0005. A cosine annealing strategy was adopted, with the learning rate decaying to 50% of its current value every 50 iterations to effectively avoid the model getting trapped in local optima. The training batch size was set to 32. After each iteration, two core metrics of the validation set were calculated: the average height error and the edge matching degree of the defect region, i.e., the degree of overlap between the predicted defect edge and the actual defect edge. When the average height error of the validation set is less than 0.2 micrometers for 10 consecutive rounds and the edge matching degree is higher than 98%, training is stopped and the optimal model parameters are saved. The early stopping mechanism avoids the overfitting problem and ensures the generalization ability of the model in the new batch of joint detection.

[0061] After training, the initial 3D point cloud seed set and feature-optimized image generated in step 3.2 are input into the model. First, sub-pixel-level pixel coordinate mapping is performed. The bilinear interpolation algorithm is used to accurately match the image grayscale information and multi-angle grayscale difference features corresponding to each point cloud data. The model extracts the spatial features and grayscale change features of the point cloud simultaneously through a dual-branch feature extraction module. After neighborhood fusion, the height correction amount of each point cloud is output. The initial point cloud height data is adjusted according to the correction amount to correct the cumulative error introduced by integral reconstruction. For the missing regions in the initial point cloud seed set caused by shadow occlusion and excessive reflection, the model predicts the height value of the missing regions and completes dense filling based on the feature distribution of the effective point clouds in the neighborhood and the grayscale change law of photometric stereo vision. The above feature extraction, height correction and region filling process is repeated until the average height error of the global point cloud in two adjacent iterations is less than the preset 0.1 micrometer threshold, generating a dense 3D point cloud model with complete coverage and high accuracy.

[0062] The advantages of using this improved PointNet architecture dense 3D point cloud model are as follows: First, the dual-branch feature fusion design, which combines the principles of photometric stereo vision, fully utilizes the correlation information between grayscale changes and surface undulations. Compared with traditional pure point cloud iterative optimization algorithms, it improves the height reconstruction accuracy of small defect areas by more than 40%, and can accurately restore the 3D morphology of 3 to 5 micrometer pinholes. Second, the model is trained on a large number of industrial samples under different working conditions, which has strong generalization ability and does not require retraining for joints of different batches and different defect types, thus adapting to the needs of online inspection on the production line. Third, the joint prediction layer enables height correction and region filling to be performed simultaneously, which improves the iteration efficiency by more than 35% compared with the step-by-step processing algorithm, meeting the real-time requirements of online inspection. Fourth, the neighborhood feature fusion module strengthens the feature representation of defect edge areas, avoids the problem of defect edge blurring during the filling process, and ensures that the dense point cloud model completely preserves the 3D contour features of small defects.

[0063] Step 3.4 involves triangulating and constructing topological relationships for the dense 3D point cloud model to form the surface 3D topological data containing continuous height information and spatial connectivity. Specifically, this includes: loading the generated dense 3D point cloud model; firstly, performing point cloud deduplication by setting a very small spatial distance threshold, traversing all point clouds, and removing duplicate point clouds with spatial coordinate distances less than the threshold; then, using a statistical filtering algorithm to perform noise filtering, statistically analyzing the distance distribution of other point clouds within the fixed neighborhood of each point cloud, and removing noise points with distances exceeding several times the standard deviation of the mean, significantly improving the purity of the point cloud data; next, based on the spatial coordinate distribution of the point clouds, constructing triangular mesh elements according to the constraint of maximizing the minimum angle, prioritizing the construction of triangles from point clouds with similar spatial distances to avoid narrow and long triangles, ensuring that the mesh elements are uniform in size and closely conform to the microscopic undulations of the joint surface; simultaneously constructing topological relationships during triangulation, establishing a triangular element index table, recording the coordinates of the three vertices of each triangular element and the index numbers of adjacent triangular elements, and calculating and storing the normal vector direction of each triangular element.

[0064] After meshing, a topological integrity check is performed. Through connectivity analysis, all triangular elements are traversed to check for isolated elements without adjacent elements and whether the vertex relationships between elements are continuous. If isolated elements exist, they are corrected by adding adjacent elements. If topological breaks exist, the mesh of the broken areas is reconstructed. Finally, a three-dimensional surface topological data containing continuous height information and complete spatial connectivity is formed. This data can intuitively and accurately reflect the microscopic undulations of the semiconductor ferrule connector surface. Its mesh density and height accuracy meet the requirements of subsequent defect identification and micron-level size measurement, providing a core spatial structural foundation for subsequent inspection processes.

[0065] In a preferred embodiment of the present invention, step 4 above may include:

[0066] Step 4.1: Based on the height field of the surface 3D topology data, calculate the preliminary topology boundary of the defect candidate region. Specifically, this involves accurately locating the defect candidate region and extracting the preliminary topology boundary from the height field of the surface 3D topology data. The machine first loads the surface 3D topology data output in Step 3 and parses the complete height field information from it. This height field contains the 3D coordinates of all triangular facet vertices, where the Z-axis data is the surface height value. Defect regions, such as pinholes or scratches, will show significant deviations in height from normal regions. Next, the machine calls the previously stored statistical data on the surface height of normal semiconductor ferrule connectors from the same batch, calculates the mean and standard deviation of the normal height, and sets a dynamic height threshold range, for example, mean ± 3 times the standard deviation. This threshold accurately avoids the micro-processing texture undulations of normal surfaces, targeting only abnormal height regions exceeding the normal range. The height of a pinhole is below the lower threshold limit, while the height of a scratch edge may be above the upper threshold limit. Subsequently, the machine traverses all triangular faces in the entire height field, comparing the height value of each facet vertex with a dynamic threshold, marking all triangular faces whose vertex heights exceed the threshold range, and initially identifying these facet sets as candidate defect regions. To eliminate noise interference in the candidate regions, such as isolated abnormal faces, the machine performs connectivity analysis on the marked faces, retaining only continuous regions as valid candidate regions. Then, the boundary extraction process begins: based on topological relationships, each triangular facet in the valid candidate region is traversed, determining whether its adjacent faces are normal height faces. If adjacent faces are normal regions, the intersection of these two faces is marked as a boundary segment. After collecting all boundary segments, the machine smooths the segments, removing tiny burrs such as 3D line segments shorter than 3 pixels, and connects broken boundary fragments through linear interpolation, ultimately forming a closed, continuous preliminary morphological boundary of the candidate defect region.

[0067] Step 4.2: Within the area enclosed by the initial topographic boundary, locate the vertex with the maximum height value and establish it as the topographic reference origin. Specifically, the core step is to lock down the vertex with the maximum height value within the area enclosed by the initial topographic boundary and establish it as the topographic reference origin. The machine first precisely delineates the area to be traversed based on the coordinate range of the initial topographic boundary to avoid invalid calculations due to exceeding the boundary range. Then, the machine traverses all vertices of all triangular faces within the enclosed area, including the three vertices of each face and internal interpolation vertices, reading the Z-axis height value of each vertex one by one, while simultaneously recording the corresponding X and Y two-dimensional coordinates, establishing a temporary coordinate-height data list. Next, the machine selects the vertex with the largest height value from this list. It is important to note that the location of the vertex with the maximum height differs for different defect types: the vertex with the maximum height for pinhole-type dent defects is usually at the boundary between the defect edge and the normal area; the vertex with the maximum height for scratch-type defects is at the protruding edge on both sides of the scratch. To avoid misidentifying isolated high points caused by noise as the benchmark, the machine performs reliability verification on the selected highest vertex: A verification neighborhood is formed by selecting 3×3 adjacent triangular faces centered on this vertex. The average height of all vertices within the neighborhood is calculated, and the deviation between the highest-height vertex and the average is checked to see if it is within 0.2 micrometers, thus adapting to the precision requirements of minute defects. If the deviation exceeds the threshold, the next highest vertex is re-selected until a vertex meeting the deviation requirement is found. After successful verification, the machine records the precise 3D coordinates of this vertex, retaining micrometer-level decimal precision, and establishes it as the origin of the topographic reference, providing a unique and accurate reference core for subsequent feature vector fitting.

[0068] Step 4.3: Centered on the topographic reference origin, perform statistical analysis of the normal vectors of the surface triangular facets in its neighborhood to determine the main direction of the texture direction, and fit and generate the first feature radiation vector along this main direction. Specifically, this includes: centering on the topographic reference origin, determining the main direction of the surface texture and fitting the first feature radiation vector by statistically analyzing the normal vectors of the neighboring triangular facets. First, the neighborhood range is set according to the scale of the micro-defects: a spherical neighborhood is defined with a radius of 3 to 5 micrometers centered on the reference origin, ensuring that the neighborhood can cover the continuous triangular facets around the reference origin, while not exceeding the defect-related area, avoiding the inclusion of irrelevant normal textures; this neighborhood usually contains 10 to 20 continuous triangular facets, which is sufficient to provide stable normal vector samples. Then, the normal vector of each triangular facet in the neighborhood is calculated. The normal vector is obtained by vector cross product based on the three-dimensional coordinates of the three vertices of each facet, while unifying the direction of the normal vectors to ensure that all normal vectors point to the outside of the surface; after that, the direction distribution of all normal vectors is statistically analyzed: the normal vector direction in three-dimensional space is mapped to a two-dimensional plane, based on the reference origin. XYPlanar projection is used to divide the angle range into intervals of 5 degrees, and the number of normal vectors in each angle interval is counted. Since the surface of a normal semiconductor ferrule connector has a fixed processing texture, such as turning texture or grinding texture, the corresponding normal vectors will form obvious clusters in specific angle intervals. The angle interval with the most clusters and the most concentrated distribution of normal vectors is the main direction of the surface texture.

[0069] To confirm the reliability of the main direction, the variance of the normal vector direction within the cluster interval is first calculated. If the variance is less than a preset threshold, it indicates good directional consistency, and the direction is ultimately determined as the main direction. The preset threshold is determined based on the statistical value of the variance of the normal vector direction of the surface texture of normal semiconductor card sleeve connectors in the same batch, and its value range is typically [5 deg]. 2 10 deg 2 The variance can be expressed as 5 square degrees to 10 square degrees; if the variance is too large, the neighborhood range is expanded and the statistics are recalculated, and the threshold can be adjusted according to the defect type or imaging conditions; after determining the main direction, taking the origin of the topography reference as the starting point, select more than 3 vertices in the same direction in the neighborhood along this main direction, and generate a radial vector through linear fitting, that is, the first feature radiation vector. This vector can accurately characterize the processing texture direction of the normal surface around the defect.

[0070] Step 4.4: Calculate the surface curvature in each direction within the neighborhood of the topographic reference origin. Determine the direction with the largest absolute value of curvature as the characteristic direction of normal change, and fit a second characteristic radiation vector along this characteristic direction. Specifically, this includes: calculating the surface curvature in each direction within the neighborhood of the reference origin, finding the characteristic direction of the most drastic normal change, and fitting a second characteristic radiation vector; using the spherical neighborhood range set in Step 4.3 to ensure consistency of the analysis scenario, first, at uniform intervals of 5 degrees from 0 to 360 degrees, at the reference origin... XY Thirty-six sampling directions are set on the planar projection to ensure comprehensive coverage of all possible directions. For each sampling direction, continuous vertices in the neighborhood are selected along that direction to form a straight sampling path. The ratio of the height change of adjacent vertices on this path to the spatial distance is calculated. By accumulating the local curvature, the surface curvature value of that direction is obtained. The larger the absolute value of the curvature value, the more severe the curvature of the surface in that direction, and the more likely it is to be the direction of change of the defect profile. Then, the absolute values ​​of curvature of all sampling directions are compared, and the direction with the largest absolute value is selected as the characteristic direction of normal change. For pinhole defects, this direction is usually the radial direction of the pinhole, pointing from the reference origin to the center of the pinhole. For scratch defects, this direction is usually the direction perpendicular to the scratch direction, that is, the direction of change of scratch depth.

[0071] To verify the validity of the feature direction, the continuity of the curvature value in that direction needs to be checked: if the absolute value of the curvature of three consecutive sampling points in that direction is greater than that of the surrounding direction, then the direction is confirmed as the true feature direction; if there is a sudden change, the sampling interval is adjusted and recalculated; after confirming the feature direction, starting from the origin of the morphology reference, select three or more vertices of the defect edge along the feature direction, such as the concave vertices of the pinhole edge or the vertices of the scratch sidewall, and generate another radial vector through linear fitting, namely the second feature radiation vector; this vector can accurately characterize the morphological change features of the defect itself, such as the concave direction and depth change direction of the defect, and together with the first feature radiation vector, it can completely outline the core morphological features of the defect.

[0072] In a preferred embodiment of the present invention, step 5 above may include:

[0073] Step 5.1: Using the morphological reference origin as the vertex and the first and second feature radiation vectors as the boundaries, an initial analysis sector is constructed. This specifically includes: using the established morphological reference origin's three-dimensional coordinates as the core reference, and combining the direction parameters of the first and second feature radiation vectors, the initial analysis sector is accurately constructed; firstly, the unit direction vectors of the two feature radiation vectors are extracted, and the angle between them is calculated through vector dot product operation. During the calculation, direction data with micron-level precision is retained to ensure the accuracy of the angle result; this angle needs to be dynamically adapted in combination with the defect type: for 3 to 5 micron pinhole defects, the angle is controlled between 60 and 90 degrees to ensure coverage of the core recessed area of ​​the pinhole's radial direction; for 5 to 10 micron scratch defects, the angle is adjusted according to the scratch extension direction, usually between 45 and 70 degrees, to ensure that the main undulations and edge features of the scratch are framed.

[0074] Next, using the topographic reference origin as the vertex, two rays are drawn along the unit direction vectors of the two characteristic radiation vectors, serving as the left and right boundaries of the sector. When generating the rays, it is necessary to associate them with the surface three-dimensional topographic data to ensure that the boundary rays always fall on the effective triangular facets and avoid crossing the gaps between facets. At the same time, an initial radial length is set, which is determined based on the maximum statistical size of defects in the same batch of semiconductor ferrule connectors, and is set according to the principle of the maximum possible size of the defect × 1.5: for example, for a 5-micrometer pinhole defect, the initial radial length is set to 7.5 micrometers; for a 10-micrometer scratch defect, the initial radial length is set to 15 micrometers, which ensures complete coverage of the core area of ​​the defect while avoiding premature inclusion of too much normal surface data. Finally, the two boundary rays, the initial radial length, and the reference origin are used to enclose a closed fan-shaped initial analysis sector, and the triangular facet index corresponding to the sector boundary is recorded to provide data support for subsequent edge continuity checks.

[0075] Step 5.2: Based on the continuity of the surface 3D topology data at the edge of the initial analysis sector, adaptively expand the boundary of the initial analysis sector to form the final multi-directional feature analysis sector. Specifically, this includes: first, locating the triangular facet set corresponding to the two boundaries of the initial sector; traversing all facets on each boundary in order from the reference origin to the radial end; extracting the vertex height data of each facet and the topological connection relationship with adjacent facets, and whether they share edges; setting dual continuity judgment indicators: first, the vertex height difference between adjacent facets is ≤0.2 micrometers, matching the height accuracy requirements of small defects, avoiding misjudging normal texture undulations as discontinuities; second, adjacent facets share edges and have no topological breaks. If all facets on the two boundaries meet both indicators, it means that the initial sector has completely covered the defect area and no expansion is needed; if any boundary has a sudden change in height difference >0.2 micrometers, or has a topological break, it means that the sector boundary is exactly stuck at the defect edge, and the expansion process needs to be started.

[0076] The expansion strategy employs angular widening combined with synchronous radial extension: the angle between the two boundaries is gradually increased in 5-degree increments; after each widening, the radial length is synchronously extended according to the ratio of (widening angle / initial angle) × initial radial length to ensure sector shape coordination; after each expansion, the triangular facets on the new boundary are re-traversed to verify the dual continuity index; the expansion logic is optimized for different defect types: pinhole defects are widened symmetrically to ensure the sector uniformly covers the pinhole circumference; scratch defects are widened first along the direction of scratch extension to avoid over-inclusion of irrelevant normal areas; the verification and expansion process is repeated until all facets on both boundaries meet the conditions of high continuity and topological integrity, ultimately forming a multi-directional feature analysis sector with smooth edges and complete coverage of the defect area, and updating the facet index and spatial range data of the sector boundaries.

[0077] Step 5.3: Within the final multi-directional feature analysis sector, the first set of internal sampling points is generated based on the preset radial step size and angular interval. Specifically, this includes: firstly, determining the sampling parameters based on the defect scale and detection accuracy requirements; the radial step size is set to 0.5 micrometers, which is based on the minimum fluctuation scale of 3-5 micrometer defects, ensuring at least two sampling points per 1-micrometer defect area to accurately capture minute height changes; the angular interval is set to 5 degrees, consistent with the sampling interval for surface curvature in Step 4, ensuring the continuity of the sampling data with the previous morphology analysis data. The aforementioned radial step size and angular interval... The spacing can be adjusted according to the defect type or imaging conditions; then, a polar coordinate sampling grid is constructed: with the topographic reference origin as the center, radial layers are uniformly divided along the radial direction of the sector, starting from the origin, with a step size of 0.5 micrometers. The number of radial layers = the final radial length of the sector ÷ the radial step size, ensuring that the outermost radial layer covers the end of the sector; along the angular direction of the sector, starting from the direction of the first feature radiation vector, angular rays are uniformly divided at 5-degree intervals. The number of angular rays = the sector angle ÷ the angular interval, ensuring that the rays uniformly cover the sector angle range. The intersection of the radial layer and the angular ray is the candidate location of the internal sampling point.

[0078] Then, the validity of the candidate points is verified: by using the patch index of the sector boundary, it is determined whether the candidate position falls on a valid triangular patch within the sector; for candidate points falling inside the patch, their precise three-dimensional coordinates are calculated by bilinear interpolation to avoid insufficient accuracy caused by sampling only the vertices of the patch; candidate points falling in the gaps between patches, topological break regions, or outside the sector are eliminated; for the core defect region, such as the center of a pinhole or the bottom of a scratch, the sampling density is increased by 1 time: based on the radial layer and angular interval of the core region, one more radial layer and two angular rays are subdivided to ensure that the key undulation features of the defect are fully captured; finally, a first set of internal sampling points is formed, which is evenly distributed, covers the entire core defect, and has high density in the key areas, and records the three-dimensional coordinates of each sampling point and its corresponding radial layer and angular position information.

[0079] Step 5.4: Within the adjacent reference region outside the final multi-directional feature analysis sector, a second set of external sampling points is generated along the normal extension direction of the sector boundary, based on preset offset distance and density rules. Specifically, this includes: first, accurately delineating the adjacent reference region; extracting the unit direction vectors of the two boundaries of the final sector; calculating the outer normal vector of each boundary, perpendicular to the boundary vector and pointing outwards from the sector; starting from each vertex of the outer edge of the sector, extending 3 micrometers along the outer normal to form the inner boundary (coinciding with the outer edge of the sector) and outer boundary of the adjacent reference region. This extension width is set according to the scale of the normal transition region around the defect, and can cover the complete transition section from the defect to the normal surface. Meanwhile, avoid including irrelevant areas that are too far away; then set sampling parameters that match the internal sampling points: the offset distance is set to 0.5 micrometers, consistent with the internal radial step size, to ensure that the data density of the two sets of sampling points is uniform. The density rule is angular synchronization and radial uniformity: that is, each angular ray corresponds to one external sampling point, and the radial distance between adjacent external sampling points is 0.5 micrometers. The above offset distance and density rule can be adjusted according to the defect type or imaging conditions; along the outer normal extension direction of the outer edge of the sector, determine the candidate positions of the external sampling points in sequence at an offset distance of 0.5 micrometers. Each candidate position must correspond to an angular ray to ensure that it corresponds one-to-one with the polar coordinate position of the internal sampling point.

[0080] Subsequently, the validity and benchmark verification of the candidate points were performed: First, it was determined whether the candidate position fell on a valid triangular facet; second, the height data of the candidate points was extracted and compared with the surface height threshold range (mean ± 3 times standard deviation) of normal semiconductor ferrule connectors in the same batch to ensure that the height was within the normal range; third, the candidate points were verified by a 3×3 neighborhood facet to confirm that the height difference within the neighborhood was ≤0.2 micrometers, ensuring that the area where the sampling point was located was a stable normal surface; candidate points that did not meet the verification conditions were eliminated, and the missing sampling points were supplemented by interpolation of valid neighborhood sampling points, finally forming a second set of external sampling points corresponding to the polar coordinate position of the internal sampling points. This set of sampling points covered the normal transition area around the defect, providing a stable benchmark for subsequent comparative analysis of defects and normal surfaces.

[0081] Step 5.5: Merge the first group of internal sampling points with the second group of external sampling points to form the first group of morphology sampling points arranged according to polar coordinates. Specifically, this includes: extracting the three-dimensional coordinate data of the two groups of sampling points, constructing a KD tree index to quickly retrieve neighboring points, and performing deduplication: setting a spatial coordinate error threshold of 0.1 micrometers to match micrometer-level detection accuracy, traversing all external sampling points, and retrieving their neighboring points among the internal sampling points; if the spatial distance between the two is <0.1 micrometers, it is determined to be an overlapping point, retaining the internal sampling points to prioritize the core defect data, and removing the external sampling points to avoid data redundancy; then performing ordered sorting according to polar coordinates: taking the morphology reference origin as the pole and the direction of the first feature radiation vector as the positive direction of the polar axis; for each sampling point, calculate its radial distance (three-dimensional spatial distance) from the reference origin and the angle with the polar axis, calculated by the dot product of the sampling point direction vector and the polar axis vector; first grouping by radial distance from near to far, with each group corresponding to a radial layer; within the same radial layer, sorting by angle from small to large to ensure that the sampling points of each radial layer are uniformly arranged along the angular direction.

[0082] After sorting, a sampling point index table is constructed to record the radial layer number, angular number, three-dimensional coordinates, and internal or external information of the sampling type for each sampling point. Finally, a sampling point set integrity check is performed: it is verified that each radial layer and each angular number has a corresponding sampling point. If there are any missing points, supplementary sampling points are generated by linear interpolation of adjacent valid sampling points to ensure that internal sampling points cover the complete defect core and external sampling points cover the complete normal reference area. The final morphological sampling point set retains the fine undulation features of the defect core and has reference data for the normal surface. It is also arranged in an orderly manner according to polar coordinates and can be directly used for subsequent defect type identification and precise analysis of dimensional parameters such as depth, width, and area.

[0083] In a preferred embodiment of the present invention, step 6 above may include:

[0084] Step 6.1: Based on the relative angle and distance between the topographic sampling points and the topographic reference origin, sort the topographic sampling points and connect them in segments to form a preliminary circular path. Specifically, this includes: first, extracting the three-dimensional coordinates of all generated topographic sampling points and the coordinate data of the topographic reference origin; for each sampling point, calculating two key parameters: one is the relative angle, calculated by taking the first feature radiation vector as the positive polar axis and obtaining it through the dot product of the line vector connecting the sampling point and the reference origin and the polar axis vector, retaining an accuracy of 1 degree; the other is the relative distance, i.e., the three-dimensional straight-line distance between the sampling point and the reference origin, retaining an accuracy of 0.1 micrometers; then, processing the sampling points according to the rule of first radial layering and then angular sorting: first, grouping all sampling points according to their relative distance, with each group corresponding to one... There are three radial layers, consistent with the radial layers in step 5. Within the same radial layer, the sampling points are sorted in ascending order of relative angle to ensure that they are arranged in an orderly manner along the ring direction. Then, segmented connections are performed: for the ordered sampling points of each radial layer, adjacent sampling points are connected with line segments in the sorting order to form a ring segment of that radial layer. If there are missing sampling points in a radial layer, such as points removed during edge verification, the missing virtual sampling points are first filled in by linear interpolation of adjacent valid sampling points, and then the connection is completed to avoid path breakage. The ring segments of all radial layers are combined to form a preliminary ring path around the origin of the topographic reference and conforming to the contour of the defect area. The above offset distance and density rules can be adjusted according to the defect type or imaging conditions.

[0085] Step 6.2 involves smoothing and closing the initial loop path to generate the final closed loop evaluation trajectory. This includes: first, smoothing optimization, traversing all segments on the initial loop path; for each sampling point (including completed virtual points), taking one adjacent point before and after it to form a three-point window; correcting the coordinates of this point using a moving average method, preserving the original polar angle trend of the sampling point during correction, eliminating only minor jitter caused by data noise, with a correction amplitude not exceeding 0.1 micrometers to avoid destroying the true defect contour; dynamically adjusting the smoothing intensity for different radial layers: the radial layer of the defect core region (the inner layer) has its smoothing amplitude halved to prevent loss of minor undulation features; the outer radial layer is smoothed normally to ensure overall path smoothness; then, closing the loop is performed. Processing: For each radial layer's annular segment, check the spatial distance between the first and last sampling points. If the distance is greater than 0.2 micrometers, it indicates a gap, and one or two transition points are generated by interpolation to fill the gap. If the first and last points overlap and the distance is less than 0.05 micrometers, one duplicate point is removed. After closure, the continuity of the entire annular trajectory is verified: calculate the angle between adjacent line segments on the trajectory. If the angle changes abruptly by more than 30 degrees, it indicates an inflection point anomaly, and the area is smoothed and corrected again. The final closed annular evaluation trajectory maintains the true contour features of the defect and has smooth and closed characteristics, which can serve as the precise boundary for subsequent region extraction and feature analysis. The smoothing amplitude and distance threshold in the closure process can be adjusted according to the defect type or imaging conditions.

[0086] Step 6.3: Determine the surface region enclosed by the final closed-loop evaluation trajectory, and extract all vertices from the surface 3D topology data within this surface region. Specifically, this includes: first confirming the definition of the enclosed region, using the final closed-loop evaluation trajectory as the boundary, and projecting the enclosed two-dimensional plane region onto... XY Plane, combined with sampling points on the trajectory Z The axis height range defines the envelope region within three-dimensional space. Z The axis range covers the highest and lowest points of the trajectory, ensuring complete undulations including defects; then, the vertices within the region are extracted: the surface 3D topology data generated in step 3 is called, and the vertices of all triangular facets are traversed, with the ray casting method used to determine the vertex position. XYThe system checks whether the projection falls within the two-dimensional region enclosed by the closed loop trajectory, and verifies whether the Z-axis height of the vertex is within the defined three-dimensional range. For vertices that satisfy both conditions, they are marked as vertices within the envelope region, and their three-dimensional coordinates, the index of the triangular facet to which they belong, and their connection relationship with adjacent vertices are recorded. If a triangular facet has two or more vertices falling within the envelope region, then the facet is completely included in the region, and all its vertices are extracted. Finally, all vertices that meet the conditions are summarized to form a vertex set of the envelope region. This set completely covers the three-dimensional topographic vertices of the defect core region, providing basic data for subsequent curvature and topology analysis.

[0087] Step 6.4: Calculate the local curvature characteristics of each vertex within the envelope region and statistically analyze their distribution to construct a micro-surface curvature distribution matrix. Specifically, this includes: calculating the local curvature of each vertex; for each vertex, selecting all adjacent vertices within its 3×3 neighborhood, covering 3 to 5 surrounding triangular facets; fitting a local quadratic surface based on the three-dimensional coordinates of these vertices; and extracting two core curvature parameters from the fitted surface: Gaussian curvature and average curvature. These two parameters accurately reflect the degree and type of surface curvature, such as convexity, concavity, and saddle shape. A 0.1 micrometer margin is retained during calculation. -1 The accuracy is adapted to the curvature variation scale of minute defects; then, a statistical distribution analysis of the curvature characteristics is performed: first, the value range of the curvature parameter is defined, with Gaussian curvature at 0.1 micrometers. -1 The intervals are divided, and the average curvature is set at 0.05 micrometers. -1 The interval division ensures that the curvature difference between normal surfaces and micro-defects can be distinguished; then, the number of vertices in each curvature interval is counted to form a curvature statistical histogram; finally, a micro-surface curvature distribution matrix is ​​constructed: based on the polar coordinate sampling grid in step 5, the envelope region is divided into several uniform sub-regions according to radial and angular intervals, each sub-region corresponding to a polar coordinate grid cell, and the average Gaussian curvature and average mean curvature of the vertices in each sub-region are calculated. These average values ​​are arranged in order according to the polar coordinate position of the sub-region to form a two-dimensional micro-surface curvature distribution matrix. The matrix can intuitively reflect the spatial distribution law of the curvature of the defect region. For example, the center of the matrix of pinhole defects will show a high negative Gaussian curvature, i.e., a concave feature, while scratch defects will show a continuous curvature anomaly band along the scratch direction. The above curvature calculation accuracy and interval division can be adjusted according to the defect type or imaging conditions.

[0088] Step 6.5 analyzes the regularity changes in the connection relationships of triangular facets within the envelope region and calculates the local topological variability. Specifically, this includes: first, determining the criteria for judging connection regularity; based on the surface topological data of normal semiconductor ferrule connectors from the same batch, the triangular facets in the normal region are mostly uniform equilateral triangles, with a stable number of adjacent facets at each vertex, typically 3 to 4; the difference in the length of the shared edge between adjacent facets is less than 0.2 micrometers, and the facet arrangement direction is consistent; then, traversing all triangular facets within the envelope region, extracting key topological features: the number of adjacent facets at each vertex, the length of each shared edge, and the normal vector direction of each facet; then calculating the local topological variability: for each sub-region, consistent with the sub-region of the curvature distribution matrix, first... The standard deviation of the number of adjacent patches is calculated. The larger the standard deviation, the more irregular the connection. Then, the coefficient of variation of the shared edge length is calculated. The coefficient of variation = standard deviation / mean, reflecting the uniformity of length. Finally, the variance of the angle between the normal vector directions is calculated, reflecting the consistency of the arrangement. These three indicators are weighted and summed in a weighted ratio of 3:3:4 to obtain the local topological variability of the sub-region. The higher the variability value, the more irregular the topological connection of the region. The variability of normal regions is low, close to 0. The variability of defective regions, such as the edges of pinholes and scratched sidewalls, will be significantly increased due to the distortion and stretching of patches. Finally, the variability value of each sub-region is output to form a topological variability distribution map corresponding to the curvature distribution matrix, which provides the core basis for subsequent accurate identification of defect types by combining curvature and topological features.

[0089] In a preferred embodiment of the present invention, step 7 above may include:

[0090] Step 7.1 involves performing eigenvalue decomposition on the microsurface curvature distribution matrix to extract the principal curvature distribution features characterizing the degree of curvature distribution concentration. Specifically, this includes: preprocessing the obtained microsurface curvature distribution matrix to remove outliers; calculating the mean and standard deviation of all elements in the matrix and replacing elements exceeding the mean ± 3 times the standard deviation with the mean to avoid individual noise points affecting the decomposition results; then performing eigenvalue decomposition, which yields a set of eigenvalues ​​and corresponding eigenvectors. The magnitude of the eigenvalue represents the energy intensity of the curvature distribution along the direction of the corresponding eigenvector; and selecting the two largest eigenvalues ​​from all eigenvalues. The eigenvector direction corresponding to the eigenvalue is the principal direction in which the curvature distribution is most concentrated. For example, the principal curvature direction of a pinhole defect is concentrated in the radial direction, while that of a scratch defect is concentrated in the direction perpendicular to the scratch. Then, the cumulative contribution rate of these two principal eigenvalues ​​is calculated as follows: Cumulative contribution rate = (first principal eigenvalue + second principal eigenvalue) ÷ sum of all eigenvalues. This is used to characterize the explanatory power of the principal direction for the overall curvature distribution. Finally, the first principal eigenvalue, the second principal eigenvalue, and the cumulative contribution rate are integrated to form the principal curvature distribution feature set. This feature set can accurately extract the core law of the curvature distribution in the defect area and distinguish between the uniform curvature of a normal surface and the concentrated curvature anomaly of a defect.

[0091] Step 7.2 involves normalizing the local topological variability to generate standardized topological variability coefficients. This includes: first, extracting the local topological variability values ​​for all sub-regions; and second, referencing the topological variability statistics of normal semiconductor ferrule connectors from the same batch to determine the normalization baseline range: using the maximum topological variability of normal samples as the upper threshold and the minimum topological variability of normal samples as the lower threshold, ensuring that the thresholds cover the topological fluctuation range of all normal surfaces. Then, normalization is performed on the variability values ​​for each sub-region, and the original variability is transformed using a linear transformation. The values ​​are mapped to the range of 0 to 1. Specifically, if the original variability of a sub-region is less than the lower threshold, the normalized result is 0, indicating that the topology of the region is completely normal. If it is greater than the upper threshold, the normalized result is 1, indicating that the topology of the region is severely variable. If it is within the threshold range, the standardized value is calculated by (original value - lower threshold) ÷ (upper threshold - lower threshold). After processing, the variability of all sub-regions is converted into a standardized topological variability coefficient of a uniform scale, eliminating the dimensional differences of the original variability of different batches and different regions, which facilitates subsequent fusion with curvature features.

[0092] Step 7.3 involves weighted fusion of the principal curvature distribution features and the standardized topological variability coefficients to calculate an initial measure of local topological distortion. This includes: standardizing the obtained principal curvature distribution features; mapping the first principal feature value, the second principal feature value, and the cumulative contribution rate to the interval between 0 and 1, using the same processing method as the topological variability normalization in Step 7.2 to ensure uniform scale; and then determining the weights for the weighted fusion: based on the verification results of a large number of defect samples, the principal curvature distribution features are more sensitive to defect topological distortion, so the weight is set to 0.6; standardization... The weight of the topological variation coefficient is set to 0.4. This weighting can more accurately capture curvature anomalies caused by defects, while also taking into account irregular changes in the topological structure. Then, the two standardized features are weighted and summed to obtain the initial local shape distortion metric for each sub-region. For example, if the standardized value of the principal curvature feature of a sub-region is 0.8 and the topological variation coefficient is 0.7, its initial metric is 0.8×0.6+0.7×0.4=0.76. The range of the initial metric is between 0 and 1. The larger the value, the more serious the deviation of the shape of the sub-region from the normal state.

[0093] Step 7.4: Map the initial local topography distortion metric to a preset quantization interval to generate the final topography distortion quantization factor. Specifically, this includes: first, preset a quantization interval adapted to the classification of minor defects, with the interval range set from 0 to 10, divided into five levels according to the degree of distortion: 0 to 2 represents no obvious defects, 2 to 4 represents slight distortion, 4 to 6 represents moderate defects, 6 to 8 represents relatively serious defects, and 8 to 10 represents severe defects. This interval division is determined based on the quality acceptance standards for semiconductor ferrule connectors; then, perform mapping processing, using a linear mapping rule to map the initial metric from 0 to 1 to the quantization interval from 0 to 10, i.e., the final quantization factor = initial degree. The value is multiplied by 10. After mapping, the rationality of the quantification factor is verified: select several groups of normal samples, minor pinhole samples, and severe scratch samples from the same batch, and check whether their quantification factors fall within the corresponding grade range. If there is a deviation, fine-tune the mapping coefficient to ensure accurate grade differentiation. The final generated morphology distortion quantification factor can convert the degree of morphology distortion of the defect into an intuitive value. For example, the quantification factor of 3 to 5 micrometer pinholes is usually between 4 and 6, and the quantification factor of 5 to 10 micrometer scratches is between 6 and 8. This provides an accurate quantitative basis for the subsequent automated judgment and grade classification of defects. The above quantification range can be adjusted according to the defect type or imaging conditions.

[0094] In a preferred embodiment of the present invention, step 8 above may include:

[0095] Step 8.1: Establish the spatial mapping relationship between the shape distortion quantization factor and the corresponding defect candidate region in the feature optimization image. Specifically, this includes: extracting all vertices of the three-dimensional sub-regions corresponding to the obtained shape distortion quantization factor, and reading the precise three-dimensional coordinates of each vertex. X , Y , Z Projecting these three-dimensional coordinates onto XY The system obtains the physical projection coordinates of each 3D sub-region onto the 2D plane, preserving the spatial positional correlation of vertices during projection to ensure that the relative positions of the 3D sub-regions are consistent with the planar projection. Simultaneously, it calls the generated feature optimization image, extracts the pixel coordinate range of the corresponding defect candidate region in the image, and retrieves the previously calibrated pixel-to-physical coordinate conversion ratio (e.g., 1 pixel corresponds to 0.1 micrometers) to clarify the correspondence between image pixel coordinates and actual physical coordinates, providing a scale benchmark for subsequent mapping. Then, three sets of key control points are selected to construct mapping bridges: the first set is the established morphological benchmark origin and its corresponding pixel in the feature optimization image; the second set is the 3D vertex at the end of the first feature radiation vector and its corresponding pixel in the image; and the third set is the 3D vertex at the end of the second feature radiation vector and its corresponding pixel in the image.

[0096] By manually verifying or edge matching, the one-to-one correspondence between the 3D physical coordinates and 2D pixel coordinates of each set of control points is accurately confirmed, ensuring that the control points cover the core defect area and are evenly distributed on the defect edge, thus guaranteeing the globality and accuracy of the mapping. Subsequently, based on the coordinate correspondence of the three sets of control points, a point-by-point mapping rule is constructed using linear interpolation: taking the control points as the reference, the planar physical projection coordinates of the 3D sub-region are divided into several uniform grids, and the corresponding pixel coordinates of each grid node in the feature optimization image are calculated. For each 3D sub-region with a shape distortion quantization factor, the corresponding image pixel coordinates are found by interpolation based on its planar projection coordinates. Finally, a one-to-one correspondence is established between each shape distortion quantization factor, the corresponding 3D sub-region, and the image pixel position, ensuring that subsequent correction operations can accurately locate the specific pixel position of the defect candidate region in the feature optimization image, achieving accurate association between 3D shape distortion information and 2D image region.

[0097] Step 8.2: Based on the spatial mapping relationship and the value of the shape distortion quantization factor, calculate the geometric correction parameters of the defect candidate region. Specifically, this includes: first, according to the spatial mapping relationship, associating the shape distortion quantization factor value corresponding to each image pixel position, and statistically analyzing the quantization factor distribution of each pixel position within the defect candidate region: the larger the quantization factor value, the more severe the shape distortion at the corresponding position, and the greater the required correction amplitude. Then, setting the calculation rules for the correction parameters: taking the surface flatness of normal semiconductor card sleeve connectors in the same batch as a benchmark, for pixel areas with a quantization factor of 0 that have no distortion, the correction parameter is set to 0 and no correction is needed; for pixel areas with a quantization factor greater than 0... For each pixel region, correction parameters are calculated proportionally to the quantization factor value, including translation, rotation angle, and scaling factor. For example, if a region with a quantization factor of 0.6 has a 0.3-micron concave distortion in the corresponding 3D space, the horizontal and vertical translation of that region in the image is calculated based on the conversion ratio between pixels and physical coordinates. If there is local distortion, the rotation angle and local scaling factor are calculated based on the distribution gradient of the quantization factor. Finally, the correction parameters for all pixel positions are summarized to form a correction parameter matrix that matches the size of the feature-optimized image. Each matrix element corresponds to the correction information of one pixel, ensuring that the correction parameters can accurately cover the entire defect candidate region.

[0098] Step 8.3 involves performing an affine transformation to correct the contour coordinates and size representation of the defect candidate region in the feature-optimized image using geometric correction parameters, generating a corrected defect geometric representation. Specifically, this includes: first, extracting the original contour pixel coordinates of the defect candidate region in the feature-optimized image; traversing each pixel on the contour and retrieving the corresponding translation, rotation angle, and scaling factor from the correction parameter matrix; performing an affine transformation correction on each contour pixel: first, adjusting the pixel position by the rotation angle to correct the contour offset caused by shape distortion; then, adjusting the local size by the scaling factor to correct the size stretching or compression deviation in the image caused by three-dimensional shape depressions or protrusions; finally... The coordinates are finely adjusted by the translation amount to ensure that the corrected contour is consistent with the true geometric shape of the defect in three-dimensional space. At the same time, the size representation of the defect is corrected: based on the corrected contour coordinates, the size parameters such as the width, length, and area of ​​the defect are recalculated. For example, the diameter of the pinhole defect is calculated based on the maximum inscribed circle diameter of the corrected contour, and the length of the scratch defect is calculated based on the longest axis length of the corrected contour. The corrected size parameters retain an accuracy of 0.1 micrometers. After the correction is completed, the continuity of the contour is verified: the distance between adjacent pixels on the corrected contour is calculated. If there is a sudden change of more than 1 pixel, the smoothing correction is re-performed on the area to ensure that the corrected defect geometric representation is smooth and complete.

[0099] Step 8.4 involves fusing and reconstructing the corrected defect geometric representation with the image background to generate and output preprocessed image data corrected for shape distortion. Specifically, this includes: extracting and optimizing background pixel information around the defect candidate region in the image, including the background's grayscale value and texture features, to ensure a consistent background style during fusion; then adjusting the grayscale value of the pixel region corresponding to the corrected defect geometric representation—referring to the height information of the defect in 3D space, the grayscale value of the concave defect region is proportionally darkened to simulate the concave shadow under real lighting, while the grayscale value of the convex defect region is proportionally brightened to make the corrected defect more visually consistent with its real shape in the image; then performing edge smoothing fusion: at the boundary between the corrected defect region and the background, a gradient fusion algorithm is used to gradually transition the grayscale value of the defect edge from the defect region to the background region, avoiding obvious splicing marks and ensuring overall visual coherence of the image; after fusion, the image quality is verified: checking whether the corrected defect outline is clear, whether the size is consistent with the actual size in the 3D topology data, and whether the background is free from distortion. Finally, the verified image data is stored and output in a preset format. This preprocessed image data eliminates the influence of shape distortion on defect characterization and can be directly used for subsequent accurate identification of defect types and quality level determination.

[0100] like Figure 2 As shown, embodiments of the present invention also provide a preprocessing data processing system for images of minute defects on the surface of semiconductor device components, comprising:

[0101] The optimization processing module is used to perform image quality normalization processing on the acquired high-magnification magnified surface original image to generate a quality-enhanced image; and to perform defect feature selective enhancement processing on the quality-enhanced image to generate a feature-optimized image.

[0102] The 3D reconstruction module is used to reconstruct the 3D topography of the feature-optimized image and generate surface 3D topography data containing height information.

[0103] The module for establishing and fitting is used to establish a topographic reference point at the highest point of the local topographic protrusion in the preliminary defect candidate region based on the surface three-dimensional topographic data. Starting from the topographic reference point, the first feature radiation vector and the second feature radiation vector are respectively fitted and generated along the main direction of the surface texture and the normal gradient direction.

[0104] The definition and deployment module is used to define a multi-directional feature analysis sector based on the first feature radiation vector and the second feature radiation vector, and to deploy a set of topographic sampling points arranged according to polar coordinates within the adjacent reference area inside and outside the multi-directional feature analysis sector.

[0105] The construction and calculation module is used to construct a closed-loop evaluation trajectory around the origin of the topography reference point based on the three-dimensional spatial coordinates of the topography sampling points, and to calculate the micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory.

[0106] The fusion generation module is used to fuse the micro-surface curvature distribution matrix and the local topological variability to generate a morphology distortion quantification factor that characterizes the degree of local morphology distortion.

[0107] The compensation and calibration module is used to compensate and calibrate the geometric representation of defects in the corresponding region of the feature-optimized image by using a shape distortion quantization factor, and outputs the final preprocessed image data after shape distortion correction. The above description is a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for preprocessing image data of minute defects on the surface of semiconductor device components, characterized in that, The method includes: The acquired high-magnification magnified original surface image is subjected to image quality normalization processing to generate a quality-enhanced image; Selective enhancement of defect features is performed on the quality-enhanced image to generate a feature-optimized image; Three-dimensional topography reconstruction is performed on the feature-optimized image to generate surface three-dimensional topography data containing height information; Based on the surface three-dimensional topological data, a topological reference origin is established at the highest point of the local topological protrusion in the preliminary defect candidate region. Starting from the topological reference origin, the first feature radiation vector and the second feature radiation vector are respectively fitted and generated along the main direction of the surface texture and the normal gradient direction. Based on the first and second feature radiation vectors, a multi-directional feature analysis sector is defined. Within the adjacent reference area inside and outside the multi-directional feature analysis sector, a set of topographic sampling points arranged according to polar coordinates are deployed. Based on the three-dimensional spatial coordinates of the topography sampling points, a closed-loop evaluation trajectory is constructed around the origin of the topography reference point. The micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory are calculated. By integrating the micro-surface curvature distribution matrix with the local topological variability, a morphology distortion quantification factor characterizing the degree of local morphology distortion is generated. By using a shape distortion quantization factor, the geometric representation of defects in the corresponding region of the feature-optimized image is compensated and calibrated, and the final preprocessed image data after shape distortion correction is output.

2. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 1, characterized in that, The acquired high-magnification original surface image is subjected to image quality normalization processing to generate a quality-enhanced image, including: A global illumination intensity analysis was performed on the acquired high-magnification magnified original surface image to generate a background illumination distribution model. Based on the background illumination distribution model, pixel-by-pixel illumination intensity compensation is performed on the original surface image to obtain an illumination-uniformed image; Based on the gray-level statistical characteristics of each local region in the uniformly illuminated image, an adaptive contrast stretching coefficient is dynamically calculated and applied to generate a contrast-normalized image. Guided smoothing filtering is applied to the contrast-normalized image to smooth the inherent texture background of the image while maintaining the integrity of structural edges in the image, resulting in an enhanced output image.

3. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 2, characterized in that, Selective enhancement of defect features is performed on the quality-enhanced image to generate a feature-optimized image, including: Multi-scale gradient feature extraction is performed on the quality-enhanced image to generate a gradient feature map containing edge information at different scales; Based on the gradient feature map, the edge structure of potential defects is located on the quality-enhanced image, and the corresponding edge structure mask is generated. Based on the edge structure mask, mask-guided local contrast enhancement is performed on the quality enhancement image to enhance the gray-level difference between defects and the background in the mask-covered area, generating a locally contrast-enhanced image; Adaptive edge sharpening is applied to the locally contrast-enhanced image to improve the clarity and continuity of the defect contours within the edge structure mask, and an optimized feature image is output.

4. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 3, characterized in that, The feature-optimized image is used to perform 3D topology reconstruction, generating surface 3D topology data containing height information, including: Structured light phase analysis is performed on the feature-optimized image to extract the phase gradient field that characterizes the micro-undulations of the surface; Based on the extracted phase gradient field, an initial 3D point cloud seed set for surface height is generated by integral reconstruction calculation; Based on the principle of photometric stereo vision, the model is generated by iteratively optimizing the image with features and the initial 3D point cloud seed set to correct the error in the height data and fill in the missing areas. The dense 3D point cloud model is triangulated and topological relationships are constructed to form the surface 3D topological data containing continuous height information and spatial connectivity.

5. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 4, characterized in that, Based on the surface 3D topological data, a topological reference origin is established at the highest point of the local topographic protrusion in the preliminary defect candidate region. Starting from the topological reference origin, a first feature radiation vector and a second feature radiation vector are generated by fitting along the principal direction of the surface texture and the normal gradient direction, respectively. These include: Based on the height field of the surface three-dimensional topology data, the preliminary topology boundary of the defect candidate region is calculated. Within the area enclosed by the preliminary topographic boundary, locate the vertex with the maximum height value and establish the vertex with the maximum height value as the topographic reference origin; Centered on the origin of the topography reference, the normal vector statistical analysis of the surface triangular patches in its neighborhood is performed to determine the main direction of the texture direction, and the first feature radiation vector is generated by fitting along this main direction. Calculate the surface curvature in each direction within the neighborhood of the topographic reference origin, determine the direction with the largest absolute value of curvature as the characteristic direction of normal change, and fit along this characteristic direction to generate a second characteristic radiation vector.

6. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 5, characterized in that, Based on the first and second characteristic radiation vectors, a multi-directional feature analysis sector is defined. Within the adjacent reference regions inside and outside the multi-directional feature analysis sector, a set of topographic sampling points arranged according to polar coordinates are deployed, including: An initial analysis sector is constructed with the morphological reference origin as the vertex and the first and second feature radiation vectors as the boundaries. Based on the continuity of the surface three-dimensional topology data at the edge of the initial analysis sector, the boundary of the initial analysis sector is adaptively expanded to form the final multi-directional feature analysis sector. Within the final multi-directional feature analysis sector, the first set of internal sampling points is generated based on the preset radial step size and angular interval; Within the adjacent reference region outside the final multi-directional feature analysis sector, a second set of external sampling points is generated along the normal extension direction of the sector boundary, based on the preset offset distance and density rules. The first group of internal sampling points and the second group of external sampling points are combined to form the first group of morphological sampling points arranged according to polar coordinates.

7. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 6, characterized in that, Based on the three-dimensional spatial coordinates of the topography sampling points, a closed-loop evaluation trajectory is constructed around the topography reference origin. The micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory are calculated, including: Based on the relative angle and distance between the topographic sampling points and the topographic reference origin, the topographic sampling points are sorted and connected in segments to form a preliminary circular path; The initial loop path is smoothed and closed at both ends to generate the final closed loop evaluation trajectory. Determine the surface region enclosed by the final closed-loop evaluation trajectory, and extract all vertices in the surface 3D topology data within this surface region; Calculate the local curvature characteristics of each vertex within the envelope region and statistically analyze their distribution to construct a micro-surface curvature distribution matrix; Analyze the regularity of the connection relationships of triangular facets within the envelope region and calculate the local topological variability.

8. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 7, characterized in that, By fusing the microsurface curvature distribution matrix with the local topological variability, a morphology distortion quantification factor characterizing the degree of local morphology distortion is generated, including: Eigenvalue decomposition is performed on the curvature distribution matrix of the micro-surface to extract the principal curvature distribution features that characterize the degree of curvature distribution concentration; The local topological variability is normalized to generate a standardized topological variability coefficient. The principal curvature distribution characteristics are weighted and fused with the standardized topological variation coefficient to calculate an initial local topographic distortion metric. The initial local shape distortion metric is mapped to a preset quantization range to generate the final shape distortion quantization factor.

9. The method for preprocessing image data of minute defects on the surface of semiconductor device components according to claim 8, characterized in that, By using a shape distortion quantization factor, the geometric representation of defects in the corresponding region of the feature-optimized image is compensated and calibrated, outputting the final preprocessed image data after shape distortion correction, including: Establish a spatial mapping relationship between the morphology distortion quantization factor and the corresponding defect candidate region in the feature-optimized image; Based on the spatial mapping relationship and the numerical value of the morphology distortion quantization factor, the geometric correction parameters of the defect candidate region are calculated; By using geometric correction parameters, the contour coordinates and size representation of the defect candidate region in the feature optimization image are corrected by affine transformation to generate the corrected defect geometric representation. The corrected defect geometric representation is fused and reconstructed with the image background to generate and output preprocessed image data that has been corrected for shape distortion.

10. A data processing system for image preprocessing of minute defects on the surface of semiconductor device components, the system implementing the method as described in any one of claims 1 to 9, characterized in that, include: The optimization processing module is used to perform image quality normalization processing on the acquired high-magnification magnified surface original image to generate a quality-enhanced image. Selective enhancement of defect features is performed on the quality-enhanced image to generate a feature-optimized image; The 3D reconstruction module is used to reconstruct the 3D topography of the feature-optimized image and generate surface 3D topography data containing height information. The module for establishing and fitting is used to establish a topographic reference point at the highest point of the local topographic protrusion in the preliminary defect candidate region based on the surface three-dimensional topographic data. Starting from the topographic reference point, the first feature radiation vector and the second feature radiation vector are respectively fitted and generated along the main direction of the surface texture and the normal gradient direction. The definition and deployment module is used to define a multi-directional feature analysis sector based on the first feature radiation vector and the second feature radiation vector, and to deploy a set of topographic sampling points arranged according to polar coordinates within the adjacent reference area inside and outside the multi-directional feature analysis sector. The construction and calculation module is used to construct a closed-loop evaluation trajectory around the origin of the topography reference point based on the three-dimensional spatial coordinates of the topography sampling points, and to calculate the micro-surface curvature distribution matrix and local topological variability of the region enclosed by the closed-loop evaluation trajectory. The fusion generation module is used to fuse the micro-surface curvature distribution matrix and the local topological variability to generate a morphology distortion quantification factor that characterizes the degree of local morphology distortion. The compensation and calibration module is used to compensate and calibrate the defect geometric representation of the corresponding region in the feature-optimized image by using the shape distortion quantization factor, and output the final preprocessed image data after shape distortion correction.

Images