Data processing method and system for pre-processing image of micro-defects on surface of semiconductor device component

Abstract

The application provides a semiconductor device component surface micro-defect image preprocessing data processing method and system, relates to the technical field of data processing, and comprises the following steps: performing image quality normalization processing on an obtained high-magnification surface original image to generate a quality enhanced image; performing selective defect feature enhancement processing on the quality enhanced image to generate a feature optimized image; performing three-dimensional topography reconstruction on the feature optimized image to generate surface three-dimensional topography topological data containing height information; and based on the surface three-dimensional topography topological data, establishing a topography reference origin at the highest point of local topography protrusion of a preliminary defect candidate region, and respectively fitting first feature radiation vectors and second feature radiation vectors along the main direction and the normal change gradient direction of the surface texture starting from the topography reference origin. The application can realize accurate characterization of micro-defects in a high-magnification scene, and guarantee the accuracy and real-time performance of defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a semiconductor equipment component surface micro-defect image preprocessing data processing method and system. BACKGROUND

[0002] In the field of semiconductor manufacturing equipment, the semiconductor ferrule connector is the core component of the gas connection, and the surface airtightness directly determines the stability of the equipment operation. In order to protect the performance of the component, the micron-level defects on the surface need to be detected under the condition of 150 times high magnification. The first step of this detection process is image preprocessing, and the accuracy of the preprocessing result directly determines the accuracy of the subsequent defect recognition.

[0003] The current mainstream micro-defect image preprocessing method has the following technical defects: only relying on two-dimensional gray scale information and edge features for defect characterization, which cannot eliminate the topography distortion problem caused by the component micro-topography fluctuation and the inherent distortion of the optical system in the high magnification imaging process. This limitation will cause the following consequences. For example, in the 150 times magnification detection scene of the semiconductor ferrule connector, the topography distortion will cause the deviation between the two-dimensional image profile of the micro-scratch pinhole and the actual three-dimensional topography. The component that meets the standard in size may be misjudged as a defective piece, and the micron-level defects that actually exist may be missed due to profile distortion, directly affecting the accuracy of component quality control. Moreover, the detection error caused by topography distortion will force the production line to increase the manual review link. Not only reduces the real-time performance of online detection, but also increases the detection cost, which cannot meet the efficiency demand of large-scale production.

[0004] In the prior art, some preprocessing schemes try to optimize image quality by improving the precision of filtering algorithm, and some schemes enhance the contrast to strengthen the defect edge features. But these schemes still cannot fundamentally solve the problem of accurate characterization of micro-defects in high magnification scenes. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a semiconductor equipment component surface micro-defect image preprocessing data processing method and system, to realize accurate characterization of micro-defects in high magnification scenes, to ensure the accuracy and real-time performance of defect detection, and to adapt to the large-scale production detection demand of precision components such as semiconductor ferrule connectors.

[0006] To solve the above technical problems, the technical scheme of the present application is as follows:

[0007] In the first aspect, a semiconductor equipment component surface micro-defect image preprocessing data processing method is provided, which comprises:

[0008] performing image quality normalization processing on the obtained high magnification surface original image to generate a quality enhanced image;

[0009] The quality-enhanced image is subjected to defect feature selective enhancement processing to generate a feature-optimized image.

[0010] The feature-optimized image is subjected to three-dimensional topography reconstruction to generate surface three-dimensional topography topology data containing height information.

[0011] Based on the surface three-dimensional topography topology data, a topographic reference origin is established at the local topographic protrusion highest point of the preliminary defect candidate region, and a first feature radiation vector and a second feature radiation vector are respectively fitted from the topographic reference origin along the main direction and the normal change gradient direction of the surface texture.

[0012] According to the first feature radiation vector and the second feature radiation vector, a multi-directional feature analysis sector is defined, and a set of topographic sampling points arranged according to polar coordinates are arranged in the inside and the adjacent reference region outside the multi-directional feature analysis sector.

[0013] Based on the three-dimensional space coordinates of the topographic sampling points, a closed ring-shaped evaluation trajectory is constructed around the topographic reference origin, and the micro-surface curvature distribution matrix and the local topological structure variation degree of the region enveloped by the closed ring-shaped evaluation trajectory are calculated.

[0014] The micro-surface curvature distribution matrix and the local topological structure variation degree are fused to generate a topographic distortion quantization factor representing the local topographic distortion degree.

[0015] Through the topographic distortion quantization factor, the defect geometric representation of the corresponding region in the feature-optimized image is compensated and calibrated, and the final preprocessed image data after topographic distortion correction is output.

[0016] In a second aspect, a semiconductor device component surface micro-defect image preprocessing data processing system includes:

[0017] The optimization processing module is configured to perform image quality normalization processing on the obtained high-magnification surface original image to generate a quality-enhanced image, and perform defect feature selective enhancement processing on the quality-enhanced image to generate a feature-optimized image.

[0018] The three-dimensional reconstruction module is configured to perform three-dimensional topography reconstruction on the feature-optimized image to generate surface three-dimensional topography topology data containing height information.

[0019] The establishment and fitting module is configured to establish a topographic reference origin at the local topographic protrusion highest point of the preliminary defect candidate region according to the surface three-dimensional topography topology data, and fit a first feature radiation vector and a second feature radiation vector from the topographic reference origin along the main direction and the normal change gradient direction of the surface texture.

[0020] A defining and arranging module is configured to define a multi-directional feature analysis sector according to the first characteristic radiation vector and the second characteristic radiation vector, and arrange a set of topographic sampling points arranged according to a polar coordinate rule in an adjacent reference area inside and outside the multi-directional feature analysis sector;

[0021] A constructing and calculating module is configured to construct a closed annular evaluation track surrounding the topographic reference origin based on the three-dimensional space coordinates of the topographic sampling points, and calculate a micro-surface curvature distribution matrix and a local topological structure variation degree of an area enveloped by the closed annular evaluation track.

[0022] A fusion generating module is configured to fuse the micro-surface curvature distribution matrix and the local topological structure variation degree to generate a topographic distortion quantization factor representing a local topographic distortion degree.

[0023] A compensation and calibration module is configured to compensate and calibrate defect geometric representation of a corresponding area in the feature optimization image through the topographic distortion quantization factor, and output a final preprocessed image data after topographic distortion correction.

[0024] The above scheme of the present application at least includes the following beneficial effects:

[0025] Because the preprocessing flow of the fused three-dimensional topography reconstruction is adopted, the optimized image is obtained through image quality normalization and defect feature selective enhancement, the three-dimensional topography topological data is generated by combining the structured light phase analysis and photometric stereo vision, and the topographic distortion quantization factor is generated through the reference origin establishment, the polar coordinate sampling point arrangement, the micro-surface curvature and the topological variation degree analysis, and finally the defect geometric representation calibration is realized, so that the technical problem that the traditional two-dimensional preprocessing cannot eliminate the topographic distortion of high magnification imaging is effectively overcome, and the micron-level micro-defects are accurately represented, the accuracy and online real-time of subsequent detection are ensured, the artificial re-inspection cost is reduced, and the large-scale production and detection needs of semiconductor card connectors and other precision components are adapted. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 FIG. 1 is a flowchart of a semiconductor device component surface micro-defect image preprocessing data processing method provided by an embodiment of the present application.

[0027] Figure 2 FIG. 5 is a schematic diagram of a semiconductor device component surface micro-defect image preprocessing data processing system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present disclosure will be described herein below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0029] As shown in Figure 1 An embodiment of the present application proposes a data processing method for pre-processing images of micro-defects on the surface of a semiconductor device component, the method comprising the following steps:

[0030] Step 1: performing image quality normalization processing on the obtained high-magnification surface original image to generate a quality-enhanced image;

[0031] Step 2: performing selective enhancement processing on the quality-enhanced image to generate a feature-optimized image;

[0032] Step 3: performing three-dimensional topography reconstruction on the feature-optimized image to generate surface three-dimensional topography topology data containing height information;

[0033] Step 4: based on the surface three-dimensional topography topology data, establishing a topographic reference origin at the highest point of the local topography protrusion in the preliminary defect candidate region, and fitting to generate a first feature radiation vector and a second feature radiation vector along the main direction and the normal variation gradient direction of the surface texture, respectively, starting from the topographic reference origin;

[0034] Step 5: according to the first feature radiation vector and the second feature radiation vector, defining a multi-directional feature analysis sector, and arranging a set of topographic sampling points arranged according to polar coordinates in the inside and adjacent reference area outside the multi-directional feature analysis sector;

[0035] Step 6: based on the three-dimensional space coordinates of the topographic sampling points, constructing a closed annular evaluation trajectory around the topographic reference origin, and calculating the micro-surface curvature distribution matrix and the local topological structure variation degree of the region enveloped by the closed annular evaluation trajectory;

[0036] Step 7: fusing the micro-surface curvature distribution matrix and the local topological structure variation degree to generate a topographic distortion quantification factor representing the distortion degree of the local topography;

[0037] Step 8: compensating and calibrating the defect geometric representation of the corresponding region in the feature-optimized image through the topographic distortion quantification factor, and outputting the final pre-processed image data after topographic distortion correction.

[0038] In the embodiment of the present application, the original image basic quality is improved by image quality normalization, and the distinction between the defect and the background is enhanced by selective enhancement of the defect features; the surface height information of the component is obtained by three-dimensional topography reconstruction, and the defect analysis area is accurately locked by establishing a reference origin, fitting a feature radiation vector, and arranging polar coordinate sampling points; the distortion degree of the topography is accurately quantified based on the micro-surface curvature distribution matrix calculated by the closed ring evaluation track and the local topological structure variation degree, and the targeted calibration of the defect geometric representation is realized through the distortion quantization factor; this method effectively solves the problem that the traditional two-dimensional preprocessing cannot eliminate the topography distortion of high magnification imaging, improves the accuracy of micro-defect representation, provides reliable image data support for defect detection, ensures the detection accuracy, and adapts to the online detection needs of semiconductor precision components, and helps to improve the production quality control efficiency.

[0039] In a preferred embodiment of the present application, step 1 can include:

[0040] Step 1.1, global illumination intensity analysis is performed on the obtained high magnification surface original image to generate a background illumination distribution model, specifically including: receiving a 150x high magnification surface original image of a semiconductor card connector, for the actual application scenarios such as illumination difference between the center and the edge of the optical lens during high magnification imaging of the semiconductor precision component, uneven reflection of the joint polished surface and the processed surface, and angle offset of different workstations, a background illumination distribution model based on an improved lightweight U-Net architecture is used to complete the global illumination intensity analysis; the model is improved from the traditional U-Net segmentation architecture, the traditional U-Net architecture is mainly used for medical image segmentation, and the encoding and decoding symmetric structure can efficiently extract multi-scale features, but has the problems of large parameter quantity, slow inference speed, and weak ability to capture local light mutation features, which cannot adapt to the real-time and accuracy requirements of semiconductor component online detection; the present application improves the above-mentioned defects, replaces the standard convolution at the encoding end with a depth separable convolution, reduces the parameter quantity by 70% while ensuring the feature extraction capability, and embeds a pixel-level attention module after each upsampling module at the decoding end to strengthen the feature extraction weight of the light mutation area and suppress the feature redundancy of the defect-free uniform illumination area.

[0041] The specific construction and training process of the model is as follows: in the model construction stage, first, 150 times magnification images of the semiconductor card sleeve joint defect-free area under different batches, different production stations and different illumination conditions are collected to construct an initial sample library, and the blurred samples caused by lens pollution and mechanical vibration are removed, and finally the effective sample amount is determined to be 8000; the sample images are divided into several non-overlapping local image blocks according to the principle of considering the stability of local illumination characteristics and capturing subtle illumination changes, and four kinds of illumination feature parameters such as the gray mean value, variance, gray distribution entropy and local gradient amplitude of each image block are extracted as the model input features; the output layer of the model is designed as a single-channel gray prediction matrix consistent with the size of the input image, and each element in the matrix corresponds to the illumination intensity prediction value of the original image pixel, so as to realize the pixel-level illumination distribution representation; at the same time, two cross-scale feature fusion branches are added between the encoding end and the decoding end, and the shallow detail illumination features and the deep global illumination features are fused to improve the perception ability of the model to the slight illumination difference.

[0042] In the model training stage, the 8000 samples are divided into training set, validation set and test set in the ratio of 7:2:1, the training set is used for model parameter iterative optimization, the validation set is used for monitoring the overfitting phenomenon in the training process, and the test set is used for final model performance evaluation; the noise-free illumination reference map obtained by averaging fusion of multiple defect-free images of the same joint and the same position is used as the label, which can accurately represent the real illumination distribution of the joint surface and avoid the interference of accidental noise of a single image; the weighted combination of L1 loss function and structural similarity loss function SSIM is used as the loss function, the L1 loss function is used to constrain the absolute error of pixel-level illumination prediction value and label value, and the structural similarity loss function is used to constrain the spatial structure consistency of illumination distribution, and the weighting coefficients are set to 0.7:0.3; the Adam optimizer is selected for parameter iteration, the initial learning rate is set to 0.001, the cosine annealing strategy is used to dynamically adjust the learning rate to avoid the model falling into local optimal solution; the training batch size is set to 16, the illumination prediction error and structural similarity index of the validation set are calculated after each round of training, when the illumination prediction error of the validation set is less than 0.8% for 10 consecutive rounds and the structural similarity index is higher than 0.98, the training is stopped, the optimal model parameters are saved, and the background illumination distribution model is constructed.

[0043] After the training is completed, the high-magnification original image of the semiconductor card sleeve joint to be processed is input into the model. The model extracts multi-scale illumination features through the encoding end, integrates deep and shallow features through the cross-scale feature fusion branch, focuses on the uneven illumination area through the pixel-level attention module of the decoding end, and outputs an illumination intensity prediction matrix consistent with the size of the original image. Then, the bilinear interpolation algorithm is used for post-processing. The specific process is as follows: first, locate the boundary area of adjacent image blocks in the prediction matrix output by the model, extract the illumination prediction values of 3 to 5 pixels on both sides of the boundary as the constraint reference, and then calculate the illumination completion values of the pixels in the boundary transition area by using the distance of the pixels from the boundary pixels of the adjacent blocks as the weight, so as to ensure that the difference between the completion values and the boundary pixel values on both sides is controlled within a preset threshold, and to avoid the occurrence of gray mutation faults between blocks. After the interpolation completion of the boundaries of all adjacent blocks is completed, the complete global background illumination distribution model is integrated. The model has both pixel-level prediction accuracy and global illumination transition smoothness.

[0044] The advantages of using the improved lightweight U-Net architecture background illumination distribution model are as follows: first, the lightweight design greatly improves the inference speed, and the model inference speed is improved by more than 45% compared with the traditional U-Net. The inference time of a single 2048x2048 pixel image is controlled within 20ms, which meets the frame rate requirement of more than 50fps for online detection of semiconductor components; second, the pixel-level attention module at the decoding end and the cross-scale feature fusion branch can accurately capture local illumination mutations under high-magnification imaging, and the representation accuracy of illumination differences caused by optical distortion and uneven material reflection is improved by more than 38% compared with traditional statistical illumination models; third, the model is trained based on a large number of industrial samples under different working conditions, has strong generalization ability, and does not need to retrain the model for different batches and different stations, thereby reducing the debugging cost of field application; fourth, the model trained in combination with the L1 and structural similarity loss functions outputs an illumination distribution that meets the pixel-level accuracy requirement and maintains spatial structure consistency, thereby providing a reliable reference for subsequent pixel-by-pixel illumination compensation.

[0045] Step 1.2, performing pixel-by-pixel illumination intensity compensation on the surface original image according to the background illumination distribution model to obtain an illumination homogenization image, specifically comprising: calling 150 times magnification images of the same batch of defect-free semiconductor card sleeve joints as standard samples, calculating the global gray mean average value of the standard samples, and setting it as a preset standard illumination intensity value; extracting the coordinate information of each pixel in the original image in accordance with the row-by-row scanning order, calling the illumination intensity prediction value of the corresponding coordinates from the generated background illumination distribution model, calculating the difference value between the illumination intensity prediction value and the preset standard illumination intensity value, taking the difference value as the illumination compensation amount of the current pixel, the compensation amount being negative when the illumination intensity prediction value is higher than the standard value, and the compensation amount being positive when the illumination intensity prediction value is lower than the standard value, threshold clipping the compensation amount to avoid the pixel gray value exceeding the normal display range after compensation, superimposing the clipped compensation amount on the original gray value of the corresponding pixel to complete the single-pixel illumination calibration, traversing all pixels of the original image to complete the calibration operation, and outputting the illumination homogenization image to eliminate the interference of local over-brightness or over-darkness in high magnification imaging on the observation of micro defects.

[0046] Step 1.3, based on the gray scale statistical characteristics of each local area in the light uniformization image, dynamically calculate and apply adaptive contrast stretching coefficient, generate contrast standardization image, specifically including: combining the size characteristics of the ≥3 μm pinhole, ≥5 μm scratch and other micro defects on the surface of the semiconductor sleeve joint, according to the principle of complete coverage of a single micro defect and not diluting the local contrast characteristics, the local area of the light uniformization image is divided, the area size needs to match the defect size, avoid too small area leading to too large statistical noise, or too large leading to the gray difference between defects and background being averaged; For each local area, perform global gray scale iteration, collect gray scale maximum, minimum, mean, variance, skewness and kurtosis and other core feature parameters, among which the variance is used to quantify the gray difference between the defects and the background, the skewness and kurtosis assist in judging the asymmetry and steepness of the gray distribution: if the skewness deviates from 0 value greatly, it means that there is a local gray mutation in the area, and there is a high probability of defects, and high kurtosis means that the gray is concentrated in a certain interval, background or defect; Based on the gray variance, set the difference judgment threshold, when the variance is less than the threshold, it is judged that the gray difference between the defects and the background is weak, at this time the stretching coefficient is dynamically calculated combined with the gray range of the area, the smaller the gray range, the larger the stretching coefficient, to ensure that the difference is effectively amplified; When the variance is greater than or equal to the threshold, it is judged that the difference is already significant enough, set a gentle stretching coefficient close to 1, to avoid excessive stretching leading to distortion of the defect edge details; Apply the corresponding stretching coefficient to the gray value in the area pixel by pixel, complete the adjustment, and verify the gray continuity of each area after adjustment, to ensure that there is no abnormal gray jump in the area, and finally integrate all areas to generate a contrast standardization image, so that the gray difference between the micro defects and the joint matrix is significantly improved, providing a clear feature basis for subsequent defect edge positioning. The above local area division size and gray scale statistical characteristics can be adjusted according to the defect type or imaging conditions.

[0047] Step 1.4, the contrast normalized image is subjected to a guided smoothing filter, which smoothes the imaging inherent texture background while keeping the integrity of the structural edges in the image, and outputs a quality enhanced image, specifically comprising: an edge-aware guided smoothing filter algorithm is used to process the contrast normalized image, and the core objective is to separate the imaging inherent texture noise such as machining lines and material graininess from the micro-defect edges; first, edge detection is performed on the image, and the gray level gradient values of each pixel in the horizontal and vertical directions are calculated respectively, and then the comprehensive gradient amplitude is obtained through the gradient amplitude formula, and a gray level gradient guide map consistent with the size of the original image is constructed: the greater the gradient amplitude, the more intense the gray level change at the position, and the higher the probability of defect edges; the filter kernel needs to balance the noise smoothing effect and the edge retention accuracy, and a small size rectangular kernel is preferred to avoid edge blurring caused by a large kernel; in the filtering process, the gray level gradient guide map is used as the weight distribution basis: for the area with a gradient amplitude less than a preset threshold, it is determined as an inherent texture background, the weight proportion of the filter kernel is increased, and the noise is weakened through multiple smoothing operations; for the area with a gradient amplitude greater than or equal to the preset threshold, it is determined as a potential defect edge, the weight of the filter kernel is reduced, and only light smoothing or the original gray value is retained to ensure that the edge structure is not eroded; after the filtering is completed, the edge integrity of the image is checked, the gradient amplitude change of the defect edge before and after the filtering is compared, and it is confirmed that the edge contour is not missing or broken, and finally a quality enhanced image with low noise and clear edges is output, which provides a high-quality two-dimensional image input basis for subsequent three-dimensional topography reconstruction.

[0048] In a preferred embodiment of the present application, step 2 can include:

[0049] Step 2.1, multi-scale gradient feature extraction is performed on the quality enhanced image to generate gradient feature maps containing edge information of different scales, specifically including: receiving the quality enhanced image, starting the multi-scale gradient feature extraction process according to the edge detection requirements of different sizes of micro defects on the surface of the semiconductor socket connector, such as pinholes of 3 to 5 microns and scratches of 5 to 10 microns; first, automatically match the gradient extraction scale range according to the defect size, small scale corresponds to the fine edge of 3 to 5 micron micro defects, medium scale corresponds to the edge of 5 to 10 micron medium scratches, and large scale corresponds to the extension area of the edge of defects above 10 microns; each scale is matched with an appropriate gradient extraction operator, small scale selects a small size operator to preserve fine edge details, and medium and large scales select slightly larger size operators to capture complete edge profiles; through pixel-by-pixel traversal of the image by each scale operator, the gradient response in the horizontal and vertical directions is calculated respectively to generate a gradient amplitude map containing edge strength information and a gradient direction map containing edge direction information; then multi-scale feature fusion is performed, and weights are assigned based on defect scale priority, small scale gradient is given 60% weight to highlight micro defect details, medium scale is given 30% weight to ensure medium defect edge integrity, and large scale is given 10% weight to suppress noise interference, and the fusion gradient feature map is generated by weighted superposition to ensure that the edge information of defects of all sizes is not missed.

[0050] Step 2.2, according to the gradient feature map, the edge structure of potential defects is located on the quality enhanced image, and the corresponding edge structure mask is generated, specifically including: loading the gradient feature map, locating the peak interval of the gradient amplitude by statistical frequency distribution curve of the gradient amplitude, taking one half of the upper limit of the peak interval as the initial edge distinguishing threshold, and adjusting the threshold through iterative verification to ensure that the threshold can accurately distinguish the real defect edge from the residual texture noise; based on the threshold, the gradient feature map is binarized, pixels with gradient amplitude higher than the threshold are marked as 1, i.e. potential edge pixels, and pixels with gradient amplitude lower than the threshold are marked as 0, i.e. background pixels; to solve the problem of edge breakage and isolated noise points after binarization, a morphological processing procedure is started in the order of expansion and then corrosion, 1 to 2 expansion operations are performed to fill the edge gap and connect the broken fragments with a small size structure element, and then 1 corrosion operation is performed to remove the edge redundancy and isolated micro noise introduced in the expansion process, obtaining a complete and clean binary edge image; based on the pixel coordinate mapping of the binary image, the contour range of the potential defect edge is accurately located on the quality enhanced image, and a single-channel edge structure mask consistent with the size of the quality enhanced image is generated, the 1 marked area in the mask strictly corresponds to the potential defect edge, providing accurate pixel-level area positioning basis for subsequent targeted enhancement.

[0051] Step 2.3, performing mask-guided local contrast enhancement on the quality-enhanced image based on the edge structure mask to strengthen the gray scale difference between the defects and the background in the mask-covered area, and generating a local contrast-enhanced image, specifically comprising: calling the generated edge structure mask, dividing the quality-enhanced image into a defect focus area (mask 1 mark area) and a background area (mask 0 mark area) according to the mask pixel mark; for the defect focus area, automatically dividing small sub-blocks based on edge connectivity, the sub-block size is set according to the standard of covering a single connected edge plus 5 to 8 background pixels around the edge, ensuring that each sub-block can completely contain local defects and a small amount of background, avoiding the problem of insufficient enhancement pertinence caused by too large sub-blocks; for each sub-block, first counting the gray mean value, variance and other characteristics, and then dynamically adjusting the contrast enhancement intensity based on the characteristics, the enhancement intensity of the sub-block with small gray scale difference is slightly higher, and the enhancement intensity of the sub-block with moderate gray scale difference is moderate; the original gray scale characteristics of the background area are not processed to avoid over-enhancement and texture noise amplification; after the enhancement is completed, the gray linear interpolation processing is performed on the 3 to 5 pixel wide transition zone between the defect focus area and the background area, so as to eliminate the gray jump marks after the area fusion, and finally generate a local contrast-enhanced image, ensuring that the transition between the defect area and the background is natural, and the defect features are more prominent.

[0052] Step 2.4, performing adaptive edge sharpening processing on the local contrast-enhanced image to improve the definition and continuity of the defect profile in the edge structure mask, and outputting a feature-optimized image, specifically comprising: receiving the local contrast-enhanced image, and performing adaptive edge sharpening only on the defect profile area marked by the mask 1 based on the edge structure mask; first constructing a 3*3 pixel local neighborhood in the defect profile area, calculating the gradient amplitude of each pixel in the neighborhood, and dividing the gradient amplitude into strong and weak intervals according to the gradient amplitude, the gradient amplitude higher than the preset threshold value being a clear edge area, and the gradient amplitude lower than the threshold value being a fuzzy edge area; dynamically adjusting the sharpening intensity for different intervals, using a low-intensity sharpening kernel for the clear edge area to slightly enhance the edge contrast, avoiding the generation of white edges and other sharpening artifacts; using a medium-intensity sharpening kernel for the fuzzy edge area to enhance the definition by improving the gray scale gradient difference, and not performing sharpening operation on the background area outside the mask to prevent amplifying texture noise; after the sharpening is completed, the continuity of the defect profile is verified through edge connectivity detection, and the integrity is verified by comparing with the original edge structure mask, and after confirming that there is no edge breakage, distortion or artifact, the feature-optimized image is output, providing an edge clear and feature complete high-quality two-dimensional input for subsequent three-dimensional topography reconstruction.

[0053] In a preferred embodiment of the present application, the above-mentioned step 3 can comprise:

[0054] Step 3.1, the structured light phase of the feature-optimized image is analyzed, and the phase gradient field representing the surface micro-undulation is extracted, specifically including: receiving the output feature-optimized image, synchronously calling the synchronization signals of the structured light projection module and the image acquisition module to ensure the accurate timing matching of the structured light projection and the image acquisition, and then obtaining the structured light modulation image of the semiconductor connector surface; for the detection requirements of the micro-undulation characteristics of the semiconductor connector surface, such as 3-5 micron pinholes and 5-10 micron scratches, the structured light phase analysis process suitable for micro-scale is started; the corner feature points of the feature-optimized image and the structured light modulation image are automatically extracted, then the registration alignment operation is performed on the two images, the displacement deviation is corrected to the sub-pixel level, and the displacement interference caused by mechanical vibration in the image acquisition process is completely eliminated; for the structured light modulation image with fine stripes, the modulation phase carrying the surface undulation information is separated by analyzing the grayscale change rule of the stripes; considering that micro-defects can cause phase mutation, first, the quality evaluation basis representing the reliability of the phase is constructed, the phase difference between adjacent pixels is taken as the constraint condition, the existing phase ambiguity is gradually corrected, the 2π periodic ambiguity problem of the modulation phase is eliminated, and the absolute phase distribution is obtained; the obtained absolute phase distribution is locally processed, the neighborhood window size is set according to the requirement of the micro-undulation, the high-frequency phase noise introduced by the uneven reflection of the surface metal material is eliminated, and finally the phase gradient field representing the micro-undulation characteristics of the connector surface is extracted, the change trend of the phase gradient field is linearly related to the surface height undulation, and accurate phase basic data is provided for subsequent three-dimensional reconstruction.

[0055] Step 3.2, according to the extracted phase gradient field, an initial three-dimensional point cloud seed set of surface height is generated by performing integral reconstruction calculation, specifically including: loading the extracted phase gradient field representing the micro relief of the semiconductor card sleeve joint surface, synchronously calling a preset high-precision calibration parameter package, the parameter package includes camera intrinsic parameters, structured light projection angle parameters and phase height mapping coefficients, wherein the camera intrinsic parameters are used to correct the influence of lens distortion in the imaging process, the structured light projection angle parameters are related to the correspondence between the stripe projection direction and the surface phase change, and the phase height mapping coefficients are obtained by fitting multiple calibration experiments of standard height templates (covering standard relief scales of 0 to 50 microns) in advance. All parameters are stored in batches to ensure the adaptation of semiconductor card sleeve joint detection requirements of different specifications; start the initial three-dimensional reconstruction process, first perform consistency check of horizontal and vertical directions on the phase gradient field, set the dual determination criteria of gradient direction change rate threshold and gradient amplitude mutation threshold; when the gradient direction of adjacent pixels changes beyond the direction threshold, or the gradient amplitude mutation exceeds the amplitude threshold, it is judged as an abnormal area, after automatically marking the abnormal area range, the weighted mean of the effective gradient in the 3x3 neighborhood of the abnormal area is used for completion, the closer to the abnormal point, the higher the gradient weight, ensuring that the completed gradient field has no abnormal mutation and conforms to the overall trend of the surrounding micro relief; select a snake-shaped integral path from the top left corner to the bottom right corner of the image, specifically, the odd rows are integrated from left to right, and the even rows are integrated from right to left. This path can offset the cumulative error generated by single-row single-direction integration through reverse integration; when performing integral operation on the phase gradient along the path, the camera intrinsic parameters are called synchronously to correct the pixel coordinates, ensuring the accuracy of the integral reference, and then the phase height mapping coefficients are used to complete the conversion from phase value to physical height value, obtaining the initial height value of each pixel corresponding to the micron-level surface.

[0056] The multi-stage effectiveness screening is performed on the initial height value, the first stage sets a dynamic height threshold range based on the surface height statistics of a large number of normal semiconductor card socket contacts in the same batch, to avoid false screening caused by normal height differences of different batches; the second stage performs local neighborhood verification on the height values after preliminary screening, to determine whether the deviation of each height value from the height values in its 3*3 neighborhood is within the allowed range, and to eliminate isolated extreme points caused by local phase noise; for the effective height data passing the two-stage screening, the corresponding sub-pixel level two-dimensional coordinates are associated, the sub-pixel accuracy after registration in step 3.1 is inherited, the key-value pair data of sub-pixel two-dimensional coordinates-micron level height are formed, and a reliability mark is added to each key-value pair based on the deviation size determined by neighborhood verification; all effective key-value pair data are sorted in the order of row priority and column second, the normal areas with gentle gradient changes are sparsely sampled at certain intervals, and the potential defect areas with sharp gradient changes are retained with all effective data, to finally generate an initial three-dimensional point cloud seed set containing key relief features of the contact surface; although the seed set is sparsely distributed as a whole, it maintains a high data density in the defect area, completely retains the core relief features of the defect area and the normal area, and the reliability mark attached can assist the weight distribution of subsequent iterative optimization.

[0057] In step 3.3, based on the principle of photometric stereo vision, iterative optimization is performed on the feature optimization image and the initial three-dimensional point cloud seed set to correct the errors of the height data and fill in the missing areas, to generate a dense three-dimensional point cloud model, which specifically includes: taking the principle of photometric stereo vision as the core technical support, the core logic of the principle is to obtain multiple gray scale images of the same object under different light source angles and fixed camera angles, use the reflection mapping relationship between the object surface gray value and the light source direction and the surface normal vector, and inversely deduce the normal vector distribution of the object surface, and then combine the normal vector integral to obtain the surface height information; in the semiconductor card socket detection scene, since the contact surface is made of metal, its gray value will change significantly with the light source angle, and small defects such as 3-5 micron pinholes and 5-10 micron scratches will destroy the continuity of the surface normal vector, resulting in obvious differences in the gray change rules between the defect area and the normal area, and the photometric stereo vision principle can accurately capture this difference and provide a core basis for the three-dimensional topography restoration of the small defects.

[0058] Based on this principle, the parameter information (including the direction vector of each light source, the light intensity parameter) of the preset multi-angle light source module and the semiconductor sleeve joint surface image data collected under the corresponding angle are called, and the dense three-dimensional point cloud generation model of the improved PointNet architecture is used to complete the densification optimization and error correction of the initial three-dimensional point cloud seed set. The model is improved from the traditional PointNet point cloud processing architecture. The traditional PointNet can only extract the spatial coordinate features of the point cloud, and has the defects of insufficient feature extraction capability of sparse point cloud and not fusing the two-dimensional gray change features generated by the photometric stereo vision principle, and cannot accurately adapt to the three-dimensional reconstruction demand of micro defects. Therefore, the gray and height correlation feature branch and the neighborhood feature fusion module are added, wherein the gray and height correlation feature branch specially extracts the gray change feature under the multi-angle light source and establishes the mapping with the point cloud height feature, and the neighborhood feature fusion module strengthens the local feature correlation of the defect area, and finally realizes the accurate reconstruction of the three-dimensional morphology of the micro defect on the surface of the semiconductor sleeve joint. The specific construction and training process of the model is as follows:

[0059] In the model construction stage, a training data set containing more than 10000 different working conditions of semiconductor sleeve joints is constructed, each group of data covering three core contents, one is the surface gray image under three or more different angle light sources, covering the common light source angle deviation working condition of the production line, two is the initial sparse point cloud seed set obtained by integral reconstruction, three is the high-precision real surface height data calibrated by the laser interferometer, which is used as the label data to ensure the height precision reference, and the data set comprehensively covers the typical defects such as 3 to 5 micron pinholes and 5 to 10 micron scratches and the material reflection difference of joints of different batches. The data set is divided into training set, validation set and test set in the ratio of 8:1:1, the training set is used for model parameter iterative optimization, the validation set is used for monitoring the overfitting phenomenon in the training process, and the test set is used for final model performance evaluation. On the basis of the traditional PointNet architecture, a double-input branch is added, the first branch inputs the spatial coordinate features of the point cloud seed set, and the global spatial distribution features of the point cloud are extracted through a one-dimensional convolution layer; the second branch inputs the gray gradient features and the gray difference features corresponding to the multi-angle light source, and the gray mutation features of the defect area are extracted through a two-dimensional convolution layer; a neighborhood feature fusion module is added at the output end of the double-branch features, which takes the point cloud pixel coordinates as the link, and pixel by pixel fusion is performed on the spatial features in each point cloud neighborhood and the gray change features of the corresponding area, the feature representation ability of the micro defect area is strengthened, and the defect reconstruction distortion caused by feature fragmentation is avoided. The output layer of the model is designed as a joint prediction layer, which synchronously outputs two kinds of core data, one is the height correction amount of each point in the sparse point cloud seed set, and the other is the grid height filling value of the missing area (the data area caused by shadow shielding and excessive reflection), realizing the integrated output of error correction and area filling, and improving the processing efficiency.

[0060] During model training, a weighted combined loss function was used. The height error loss weight was set to 0.6, constraining the model's height reconstruction accuracy by calculating the absolute deviation between the predicted height and the true height calibrated by the laser interferometer. The grayscale prediction error loss weight was set to 0.4, constraining the deviation between the theoretical grayscale value generated based on the predicted height and light source parameters and the actual acquired grayscale value, ensuring that the predicted height conforms to the reflection law of photometric stereo vision, balancing height accuracy and grayscale feature consistency. The Adam optimizer was used for parameter iteration, with an initial learning rate of 0.0005. A cosine annealing strategy was adopted, with the learning rate decaying to 50% of its current value every 50 iterations to effectively avoid the model getting trapped in local optima. The training batch size was set to 32. After each iteration, two core metrics of the validation set were calculated: the average height error and the edge matching degree of the defect region, i.e., the degree of overlap between the predicted defect edge and the actual defect edge. When the average height error of the validation set is less than 0.2 micrometers for 10 consecutive rounds and the edge matching degree is higher than 98%, training is stopped and the optimal model parameters are saved. The early stopping mechanism avoids the overfitting problem and ensures the generalization ability of the model in the new batch of joint detection.

[0061] After training, the initial 3D point cloud seed set and feature-optimized image generated in step 3.2 are input into the model. First, sub-pixel-level pixel coordinate mapping is performed. The bilinear interpolation algorithm is used to accurately match the image grayscale information and multi-angle grayscale difference features corresponding to each point cloud data. The model extracts the spatial features and grayscale change features of the point cloud simultaneously through a dual-branch feature extraction module. After neighborhood fusion, the height correction amount of each point cloud is output. The initial point cloud height data is adjusted according to the correction amount to correct the cumulative error introduced by integral reconstruction. For the missing regions in the initial point cloud seed set caused by shadow occlusion and excessive reflection, the model predicts the height value of the missing regions and completes dense filling based on the feature distribution of the effective point clouds in the neighborhood and the grayscale change law of photometric stereo vision. The above feature extraction, height correction and region filling process is repeated until the average height error of the global point cloud in two adjacent iterations is less than the preset 0.1 micrometer threshold, generating a dense 3D point cloud model with complete coverage and high accuracy.

[0062] The advantages of using the improved PointNet architecture to model the dense three-dimensional point cloud are as follows: first, the dual-branch feature fusion design combining the principle of photometric stereo vision fully utilizes the associated information of gray scale change and surface relief, and compared with traditional pure point cloud iterative optimization algorithms, the height reconstruction accuracy of small defect areas is improved by more than 40%, and the three-dimensional morphology of 3 to 5 microns pinhole can be accurately restored; second, the model is trained based on a large number of industrial samples under different working conditions, has strong generalization ability, and does not need to be retrained for different batches and different defect types of joints, which meets the needs of online detection of production lines; third, the joint prediction layer realizes synchronous height correction and area filling, and compared with the step-by-step processing algorithm, the iteration efficiency is improved by more than 35%, which meets the real-time requirements of online detection; fourth, the neighborhood feature fusion module strengthens the feature representation of the defect edge area, avoids the problem of blurred defect edges in the filling process, and ensures that the dense point cloud model completely retains the three-dimensional contour features of small defects.

[0063] Step 3.4, triangulate the dense three-dimensional point cloud model and construct the topological relationship to form the surface three-dimensional topological data containing continuous height information and spatial connection relationship, specifically including: loading the generated dense three-dimensional point cloud model, first performing point cloud deduplication processing, setting a very small spatial distance threshold, traversing all point clouds, and removing repeated point clouds with a spatial coordinate distance less than the threshold; then using a statistical filtering algorithm to perform noise filtering processing, for each point cloud, the distance distribution of other point clouds in its fixed neighborhood is counted, and noise points with a distance exceeding a certain number of standard deviations of the mean are removed, significantly improving the purity of the point cloud data; then based on the spatial coordinate distribution of the point cloud, the triangular mesh elements are constructed under the constraint condition of maximizing the minimum angle, the point clouds with similar spatial distances are preferentially selected to construct triangles, avoiding the appearance of narrow and long triangles, and ensuring that the mesh element size is uniform and closely fits the micro relief form of the joint surface; the topological relationship is constructed synchronously during the triangulation process, the triangular element index table is established, the three vertex coordinates of each triangular element, the index number of adjacent triangular elements are recorded, and the normal vector direction of each triangular element is calculated and stored.

[0064] After completing the meshing, the integrity of the topological relationship is checked, all triangular elements are traversed through connectivity analysis, whether there are isolated elements without adjacent elements, and whether the vertex association between elements is continuous are checked, if there are isolated elements, they are modified by supplementing adjacent elements, if there are topological breaks, the mesh of the broken area is reconstructed; finally, the surface three-dimensional topological data containing continuous height information and complete spatial connection relationship is formed, which can intuitively and accurately reflect the micro relief form of the semiconductor card sleeve joint surface, and the grid density and height accuracy meet the needs of subsequent defect identification and micron-level size measurement, providing a core spatial structure basis for the subsequent detection process.

[0065] In a preferred embodiment of the present application, step 4 above can include:

[0066] Step 4.1, according to the height field of the surface three-dimensional topography topological data, the preliminary topographic boundary of the defect candidate region is calculated and obtained, specifically including: the core is to accurately locate the defect candidate region and extract the preliminary topographic boundary from the height field of the surface three-dimensional topography topological data. The machine first loads the surface three-dimensional topography topological data output by step 3, and parses the complete height field information from it. This height field contains the three-dimensional coordinates of all triangle vertices, and the Z-axis data is the surface height value. The concave and convex of defects such as pinholes and scratches will form a significant deviation from the height of the normal area. Then, the machine calls the pre-stored statistical data of the surface height of the normal semiconductor card socket of the same batch, calculates the mean and standard deviation of the normal height, and sets a dynamic height threshold range based on this, for example, mean ± 3 times standard deviation. This threshold can accurately avoid the micro-texture fluctuations of the normal surface, and only target the height abnormal areas that exceed the normal range. The height of the pinhole is lower than the lower limit of the threshold, and the height of the scratch edge may be higher than the upper limit of the threshold. Subsequently, the machine traverses all the triangle facets of the entire height field, and compares the height value of each facet vertex with the dynamic threshold one by one, marks all the triangle facets whose vertex height exceeds the threshold range, and preliminarily determines the set of these facets as the defect candidate region. In order to eliminate noise interference in the candidate region, such as isolated abnormal facets, the machine will perform connectivity analysis on the marked facets, and only keep the continuous patch regions as valid candidate regions. Then start the boundary extraction process: based on the topological relationship, traverse each triangle facet of the valid candidate region, judge whether its adjacent facet is a normal height facet, if the adjacent facet is a normal region, mark the intersection line of the two facets as a boundary line segment. After collecting all the boundary line segments, the machine will perform smoothing processing on the line segments, remove the micro burrs with a length of less than 3 pixels corresponding to the three-dimensional line segments, connect the broken boundary segments through linear interpolation, and finally form a closed and continuous preliminary topographic boundary of the defect candidate region.

[0067] Step 4.2, locating the vertex with the maximum height value in the region enclosed by the preliminary topographic boundary, and establishing the vertex with the maximum height value as the topographic reference origin, specifically including: the core is to lock the vertex with the maximum height value in the region enclosed by the preliminary topographic boundary and establish it as the topographic reference origin. The machine first accurately circumscribes the region to be traversed according to the coordinate range of the preliminary topographic boundary, avoiding invalid calculation caused by exceeding the boundary range. Then, the machine traverses all the vertices of all the triangular facets in the enclosed region, including the three vertices of the facets and the internal interpolation vertices, reads the Z-axis height value of each vertex one by one, records the corresponding X, Y two-dimensional coordinates, and establishes a temporary coordinate-height data list. Then, the machine selects the vertex with the maximum height value from the list. Here, it should be noted that the position of the maximum height vertex is different for different defect types: the maximum height vertex of pinhole type recess defects is usually at the junction of the defect edge and the normal area; the maximum height vertex of scratch type defects is the edge protrusion on both sides of the scratch. In order to avoid misjudging isolated high points caused by noise as the reference, the machine will verify the reliability of the selected maximum height vertex: select 3x3 adjacent triangular facets to form a verification neighborhood with the vertex as the center, calculate the height mean of all vertices in the neighborhood, and judge whether the deviation of the maximum height vertex from the mean is within 0.2 microns, so as to adapt to the accuracy requirements of small defects. If the deviation exceeds the threshold, reselect the second maximum height vertex, until the vertex that meets the deviation requirement is found. After verification, the machine records the accurate three-dimensional coordinates of the vertex, retains the micron level decimal precision, and establishes it as the topographic reference origin, providing a unique and accurate reference core for subsequent feature vector fitting.

[0068] Step 4.3, with the topographic reference origin as the center, performing normal vector statistical analysis on the surface triangular facets in its neighborhood to determine the main direction of the texture direction, and fitting to generate the first feature radiation vector, specifically including: taking the topographic reference origin as the center, determining the main direction of the surface texture and fitting the first feature radiation vector by statistical analysis of the normal vectors of the triangular facets in the neighborhood, first set the neighborhood range according to the scale of the small defects: define a spherical neighborhood with a radius of 3 to 5 microns centered on the reference origin, ensure that the neighborhood covers the continuous triangular facets around the reference origin, and does not exceed the defect related area, avoiding the inclusion of irrelevant normal texture; this neighborhood usually contains 10 to 20 continuous triangular facets, which is enough to provide stable normal vector samples, then calculate the normal vector of each triangular facet in the neighborhood, get the normal vector based on the three-dimensional coordinates of each facet vertex by vector cross product, and unify the direction of the normal vector to ensure that all normal vectors point to the outside of the surface; then, perform directional distribution statistics on all normal vectors: map the normal vector direction of the three-dimensional space to a two-dimensional plane, based on the X, Y coordinates of the reference origin, and calculate the angle between the normal vector and the X axis, which is the normal vector direction angle, then calculate the angle histogram of the normal vector direction angle, and determine the main direction of the texture direction according to the peak value of the histogram, which is the first feature radiation vector. XYThe angle range is divided into intervals of 5 degrees, and the number of normal vectors in each angle interval is counted. Since the normal semiconductor card sleeve connector surface has fixed machining textures, such as turning textures and grinding textures, the corresponding normal vectors will form obvious clusters in specific angle intervals. The angle interval with the most clusters and the most concentrated normal vector distribution is the main direction of the surface texture.

[0069] To confirm the reliability of the main direction, the direction variance of the normal vectors in the cluster interval is calculated. If the variance is less than a preset threshold, it indicates that the direction consistency is good, and the direction is finally determined as the main direction. The preset threshold is determined based on the normal semiconductor card sleeve connector surface texture normal vector direction variance statistical value of the same batch, and the value range is usually [5 deg 2 , 10 deg 2 ], which can be expressed as 5 square degrees to 10 square degrees. If the variance is too large, the neighborhood range is expanded and re-counted, and the threshold can be adjusted according to the defect type or imaging conditions. After determining the main direction, the neighborhood of the top point in the same direction is selected along the main direction from the morphology reference origin, and a radial vector is generated by linear fitting, which is the first characteristic radial vector. This vector can accurately represent the machining texture direction of the normal surface around the defect.

[0070] Step 4.4, calculate the surface curvature of each direction in the neighborhood of the morphology reference origin, and determine the direction with the maximum curvature absolute value as the characteristic direction of the normal change, and generate the second characteristic radial vector along this characteristic direction, which includes: the core is to calculate the surface curvature of each direction in the neighborhood of the reference origin, find the characteristic direction with the most severe normal change and fit the second characteristic radial vector; use the spherical neighborhood range set in step 4.3 to ensure consistent analysis scene, first set 36 sampling directions on the plane projection of the reference origin, ensure comprehensive coverage of all possible directions; for each sampling direction, select the continuous top points in the neighborhood along the direction to form a straight line sampling path, calculate the ratio of the height change and the spatial distance of the adjacent top points on this path, and get the surface curvature value of this direction through the cumulative calculation of the local bending degree; the greater the absolute value of the curvature value, the more severe the bending of the surface in this direction, and the more likely it is the change direction of the defect profile; then, compare the absolute values of the curvatures of all sampling directions, and select the direction with the maximum absolute value as the characteristic direction of the normal change; for pinhole defects, this direction is usually the radial direction of the pinhole, pointing to the center of the pinhole; for scratch defects, this direction is usually perpendicular to the scratch direction, i.e. the depth change direction of the scratch. XY

[0071] ​In order to verify the effectiveness of the feature direction, it is necessary to check the continuity of the curvature value in the direction: if the absolute value of the curvature of the continuous three sampling points in the direction is greater than the peripheral direction, it is confirmed that the direction is the true feature direction; if there is a mutation, adjust the sampling interval and recalculate; after confirming the feature direction, take the top point of the defect edge along the feature direction, such as the concave top point of the pinhole edge and the top point of the scratch side wall, as the starting point, and generate another radial vector, that is, the second feature radial vector, through linear fitting; this vector can accurately represent the topographic change characteristics of the defect itself, such as the concave direction and depth change direction of the defect, which can cooperate with the first feature radial vector to completely outline the core topographic characteristics of the defect.

[0072] In a preferred embodiment of the present application, step 5 can include:

[0073] Step 5.1, taking the topographic reference origin as the vertex, and taking the first feature radial vector and the second feature radial vector as the boundary, an initial analysis sector is constructed, specifically including: taking the established topographic reference origin three-dimensional coordinate as the core reference, combining the direction parameters of the first and second feature radial vectors, and accurately constructing the initial analysis sector; first, extract the unit direction vectors of the two feature radial vectors, calculate the included angle between them through vector dot product operation, and keep the direction data with micron-level precision during the calculation process to ensure the accuracy of the included angle result; the included angle needs to be dynamically adapted according to the defect type: for 3-5 micron pinhole defects, the included angle is controlled at 60-90 degrees to ensure that the core concave area of the pinhole is covered; for 5-10 micron scratch defects, the included angle is adjusted according to the extension direction of the scratch, and is usually 45-70 degrees to ensure that the main fluctuation section and edge features of the scratch are enclosed.

[0074] Then, taking the topographic reference origin as the vertex, two rays are drawn along the unit direction vectors of the two feature radial vectors as the left and right boundaries of the sector, and the surface three-dimensional topological data needs to be associated during the generation of the rays to ensure that the boundary rays always fall on the effective triangular patches to avoid cross-patch gaps; at the same time, the initial radial length is set, which is determined based on the maximum statistical size of the semiconductor card socket joint defects in the same batch, and is set according to the principle of maximum possible defect size x 1.5: for example, for 5 micron pinhole defects, the initial radial length is set to 7.5 microns; for 10 micron scratch defects, the initial radial length is set to 15 microns, which not only ensures complete coverage of the core area of the defect, but also avoids premature inclusion of too much normal surface data; finally, through the two boundary rays, the initial radial length and the reference origin, a closed sector-shaped initial analysis sector is formed, and the triangular patch index corresponding to the sector boundary is recorded to provide data support for subsequent edge continuity checking.

[0075] Step 5.2, according to the surface three-dimensional topography topological data in the continuity of the initial analysis sector edge, the boundary of the initial analysis sector is adaptively expanded to form the final multi-directional feature analysis sector, specifically including: first locate the triangular facet set corresponding to the two boundary of the initial sector, traverse all the facets on each boundary in order from the reference origin to the radial end, extract the vertex height data of each facet and the topological connection relationship of adjacent facets, whether sharing edge; Set double continuity judgment index: one is the vertex height difference of adjacent facets ≤0.2 microns, match the height accuracy requirement of small defects, avoid misjudging normal texture fluctuation as discontinuous; Two is that adjacent facets exist sharing edge, no topological fracture, if all the facets on the two boundaries meet the double index, it means that the initial sector has completely covered the defect area, and there is no need to expand; If there is a height difference >0.2 microns or topological fracture on any boundary, it means that the sector boundary is just stuck in the defect edge, and the expansion process needs to be started.

[0076] The strategy of angular widening + radial synchronous extension is adopted during expansion: the included angle of the two boundaries is gradually expanded by 5 degrees step by step, after each widening, the radial length is synchronously extended by the proportion of (widening angle / initial included angle) x initial radial length, to ensure the coordination of sector shape; After each expansion, the triangular facets on the new boundary are traversed again to review the double continuity index; Optimize the expansion logic for different defect types: pinhole defects adopt symmetric widening to ensure uniform coverage of the pinhole circumferential area; Scratch defects preferentially widen the boundary close to the scratch extension direction to avoid excessive inclusion of irrelevant normal area; Repeat the checking and expanding process until all the facets on the two boundaries meet the conditions of height continuity and topological integrity, finally form a multi-directional feature analysis sector with smooth edges and complete coverage of the defect area, and update the facet index and spatial range data of the sector boundary.

[0077] Step 5.3, in the final multi-directional feature analysis sector, a first group of internal sampling points is generated according to a preset radial step and angular interval, specifically including: first, based on the defect scale and detection accuracy requirements, the sampling parameters are determined, the radial step is set to 0.5 microns, which is set according to the minimum fluctuation scale of 3 to 5 micron defects, ensuring that there are at least 2 sampling points per 1 micron defect area, which can accurately capture small height changes; the angular interval is set to 5 degrees, which is consistent with the sampling interval of the surface curvature in step 4, ensuring the continuity of the sampling data and the previous topography analysis data; the above-mentioned radial step and angular interval can be adjusted according to the defect type or imaging conditions; then the polar coordinate sampling grid is constructed: taking the topography reference origin as the center, starting from the origin along the radial direction of the sector, evenly dividing the radial layers by 0.5 microns, the number of radial layers = the final sector radial length ÷ the radial step, ensuring that the outermost radial layer covers the end of the sector; starting from the first feature radiation vector direction along the angular direction of the sector, evenly dividing the angular rays by 5 degrees, the number of angular rays = sector angle ÷ angular interval, ensuring that the rays evenly cover the sector angle range, the intersection of the radial layer and the angular ray is the candidate position of the internal sampling point.

[0078] Then the candidate point validity verification is performed: through the face index of the sector boundary, it is judged whether the candidate position falls on the valid triangular face within the sector; for the candidate points falling inside the face, the accurate three-dimensional coordinates of the candidate points are calculated through bilinear interpolation, avoiding the insufficient accuracy caused by only sampling the face vertices; the candidate points falling in the gap between the faces, the topological fracture area or outside the sector are removed; for the defect core area, such as the pinhole center and the scratch bottom, the sampling density is additionally increased by 1 times: based on the radial layer and angular interval of the core area, 1 radial layer and 2 angular rays are further subdivided, ensuring that the key fluctuation features of the defect are fully captured; finally, a first group of internal sampling points is formed, which are uniformly distributed, cover the complete defect core and the key area with increased density, and record the three-dimensional coordinates of each sampling point and the radial layer, angular position information.

[0079] Step 5.4, in the adjacent reference area outside the final multi-directional feature analysis sector, along the normal extension direction of the sector boundary, a second group of external sampling points is generated according to the preset offset distance and density rules, specifically including: first, accurately delineate the adjacent reference area, extract the unit directional vector of the two boundaries of the final sector, calculate the outside normal vector of each boundary, which is perpendicular to the boundary vector and points to the outside of the sector; take each vertex of the outer edge of the sector as the starting point, extend 3 microns along the outside normal to form the inner boundary (coinciding with the outer edge of the sector) and the outer boundary of the adjacent reference area, the extension width is set according to the size of the normal transition area of the defect periphery, which can cover the complete transition section from the defect to the normal surface, while avoiding the inclusion of too far irrelevant areas; then set the sampling parameters matching the internal sampling points: the offset distance is set to 0.5 microns, consistent with the internal radial step length, to ensure that the data density of the two groups of sampling points is uniform, and the density rule is angular synchronization and radial uniformity: that is, each angular ray corresponds to one external sampling point, and the radial distance between adjacent external sampling points is 0.5 microns, the above offset distance and density rules can be adjusted according to the defect type or imaging conditions; along the outside normal extension direction of the outer edge of the sector, the candidate positions of the external sampling points are determined in turn at an offset distance of 0.5 microns, each candidate position needs to correspond to an angular ray, to ensure one-to-one correspondence with the polar coordinate positions of the internal sampling points.

[0080] Subsequently, the effectiveness and reference of the candidate points are verified: first, it is judged whether the candidate position falls on the valid triangular patch; second, the height data of the candidate point is extracted and compared with the surface height threshold range (mean ± 3 times standard deviation) of the normal semiconductor card connector in the same batch, to ensure that the height is within the normal range; third, the 3x3 neighborhood patch verification is performed on the candidate point to confirm that the height difference in the neighborhood is ≤0.2 microns, ensuring that the area where the sampling point is located is a stable normal surface; the candidate points that do not meet the verification conditions are removed, and the missing sampling points are supplemented by interpolation of the neighborhood valid sampling points, finally forming a second group of external sampling points corresponding to the polar coordinate positions of the internal sampling points, which covers the normal transition area around the defect, providing a stable reference for subsequent comparison and analysis of the defect and the normal surface.

[0081] Step 5.5, the first group of internal sampling points and the second group of external sampling points are combined to form the group of topography sampling points arranged according to the polar coordinate rule, and specifically includes: extracting three-dimensional coordinate data of the two groups of sampling points, constructing a KD tree index to quickly retrieve adjacent points, and performing a de-duplication process: setting a spatial coordinate error threshold of 0.1 microns, matching micron-level detection accuracy, traversing all external sampling points, and retrieving adjacent points thereof in internal sampling points; if the spatial distance between the two is <0.1 microns, it is determined that they are coincident points, the internal sampling points are retained, the defect core data is preferentially ensured, the external sampling points are removed, and data redundancy is avoided; then, an ordered sorting is performed according to the polar coordinate rule: taking the topography reference origin as the polar point and the direction of the first characteristic radiation vector as the positive direction of the polar axis; for each sampling point, the radial distance (three-dimensional spatial distance) from the reference origin and the angle with the polar axis are calculated through the dot product of the sampling point direction vector and the polar axis vector; first, group them according to the radial distance from near to far, and each group corresponds to a radial layer; in the same radial layer, sort them according to the angle from small to large to ensure that the sampling points in each radial layer are uniformly arranged along the angular direction.

[0082] After sorting, a sampling point index table is constructed to record the radial layer number, angular number, three-dimensional coordinates, and sampling type (internal or external) information of each sampling point; finally, a sampling point set integrity check is performed: check whether each radial layer and each angular number has a corresponding sampling point, if there is a missing, generate a supplementary sampling point through linear interpolation of adjacent valid sampling points to ensure that the internal sampling points cover the complete defect core and the external sampling points cover the complete normal reference area; the finally formed topography sampling point set not only retains the fine fluctuation characteristics of the defect core, but also has reference data of the normal surface, and is ordered according to the polar coordinate rule, which can be directly used for subsequent defect type identification, size parameter measurement such as depth, width, and area accurate analysis.

[0083] In a preferred embodiment of the present application, the above-mentioned step 6 can include:

[0084] Step 6.1, according to the relative angle and distance of the topography sampling point and the topography reference origin, the topography sampling point is sorted and segmented to connect to form a preliminary ring path, specifically including: first extract the three-dimensional coordinates of all generated topography sampling points, and the coordinate data of the topography reference origin, for each sampling point, calculate two key parameters: one is the relative angle, with the first characteristic radiation vector as the polar axis positive direction, through the dot product operation of the connecting vector of the sampling point and the reference origin and the polar axis vector, the relative angle is obtained, and the accuracy is retained to 1 degree; the second is the relative distance, that is, the three-dimensional space straight line distance between the sampling point and the reference origin, the accuracy is retained to 0.1 microns; then the sampling points are processed according to the rule of first radial layering and then angular sorting: first group all the sampling points according to the relative distance, each group corresponds to a radial layer, which is consistent with the radial layer of step 5; in the same radial layer, the sampling points are sorted in order from small to large according to the relative angle, so as to ensure that the sampling points are arranged in order along the ring direction; then segment connection: for the ordered sampling points of each radial layer, the adjacent two sampling points are connected by line segment according to the sorting order to form the ring segment of the radial layer; if there are missing sampling points in a radial layer, such as the points removed in the edge verification, first complete the virtual sampling points at the missing position through the linear interpolation of the adjacent effective sampling points, and then complete the connection to avoid path breakage; the combination of all radial layer ring segments forms a preliminary ring path around the topography reference origin and adheres to the contour of the defect area. The above offset distance and density rules can be adjusted according to the defect type or imaging condition.

[0085] Step 6.2, smooth optimization and head-to-tail closing processing are performed on the preliminary ring-shaped path to generate a final closed ring-shaped evaluation trajectory, specifically including: first, performing smooth optimization, traversing all line segments on the preliminary ring-shaped path, for each sampling point, including the completed virtual point, taking the previous and next 1 adjacent point to form a 3-point window, and correcting the coordinates of the point by moving average method, preserving the original polar angle trend of the sampling point, only eliminating the slight jitter caused by data noise, and the correction amplitude is not more than 0.1 microns, avoiding destroying the real defect profile; the smoothness is dynamically adjusted for paths of different radial layers: the radial layer of the defect core region, that is, the inner layer, the smoothness is halved to prevent losing small fluctuation features; the outer radial layer is normally smoothed to ensure the overall smoothness of the path; then, head-to-tail closing processing is performed: for each radial layer ring segment, the spatial distance of the first and last sampling points is checked, if the distance is greater than 0.2 microns, it means that there is a gap, and 1-2 transition points are generated by interpolation to fill the gap; if the first and last points overlap, the distance is less than 0.05 microns, then one of the repeated points is removed; after closing, the continuity of the entire ring-shaped trajectory is also verified: the included angle of adjacent line segments on the trajectory is calculated, if the included angle suddenly changes by more than 30 degrees, it means that there is a corner abnormality, and the smoothness of the region is re-modified; the final closed ring-shaped evaluation trajectory not only maintains the real profile characteristics of the defect, but also has the characteristics of smoothness and closure, which can be used as the accurate boundary for subsequent region extraction and feature analysis. The smoothness and distance threshold in the above closing processing can be adjusted according to the defect type or imaging conditions.

[0086] Step 6.3, determining the surface region enveloped by the final closed ring-shaped evaluation trajectory, extracting all vertices in the surface three-dimensional topological data in the surface region, specifically including: first, confirming the definition of the envelope region, taking the final closed ring-shaped evaluation trajectory as the boundary, forming a two-dimensional planar region, projecting it to XY the plane, and combining the Z axial height range of the sampling points on the trajectory to demarcate the envelope region in the three-dimensional space, Z the axial range covers the highest and lowest points of the trajectory, ensuring that the complete fluctuation of the defect is included; then extracting the vertices in the region: calling the surface three-dimensional topological data generated in step 3, traversing all vertices of the triangular patches, judging the XYwhether the projection falls within the two-dimensional region enclosed by the closed loop trajectory, and whether the Z-axis height of the vertex is within the specified three-dimensional range; for vertices that satisfy both conditions, mark them as vertices within the envelope region, and record their three-dimensional coordinates, the index of the triangular facet to which they belong, and the connection relationship with adjacent vertices; if a triangular facet has 2 or more vertices falling within the envelope region, then the entire facet is included in the region, and all its vertices are extracted; finally, all vertices that meet the conditions are summarized to form a vertex set of the envelope region, which completely covers the three-dimensional topography of the defect core region, providing basic data for subsequent curvature and topological analysis.

[0087] Step 6.4, calculate the local curvature characteristics of each vertex within the envelope region, and count the distribution to construct a micro-surface curvature distribution matrix, which specifically includes: calculating the local curvature of each vertex, for each vertex, selecting all adjacent vertices within its 3x3 neighborhood, covering 3 to 5 triangular facets on the periphery, fitting a local quadratic surface based on the three-dimensional coordinates of these vertices, extracting two core curvature parameters, Gaussian curvature and mean curvature, from the fitted surface, which accurately reflect the bending degree and type of the surface, convex, concave, saddle; the calculation is performed with a precision of 0.1 microns -1 , which is suitable for the curvature variation scale of small defects; then, statistical distribution analysis is performed on the curvature characteristics: first, define the value interval of the curvature parameters, the Gaussian curvature is divided by 0.1 microns -1 , and the mean curvature is divided by 0.05 microns -1 , to ensure that the curvature difference between normal surfaces and small defects can be distinguished; then, count the number of vertices in each curvature interval to form a curvature statistical histogram; finally, construct a micro-surface curvature distribution matrix: based on the polar coordinate sampling grid in step 5, divide the envelope region into several uniform sub-regions according to the radial layer and angular interval, each sub-region corresponds to a polar coordinate grid cell, calculate the average Gaussian curvature and average mean curvature of the vertices in each sub-region, and arrange these average values in order according to the polar coordinate position of the sub-region, forming a two-dimensional micro-surface curvature distribution matrix, which can intuitively reflect the spatial distribution law of the curvature of the defect region, for example, the matrix center of a pinhole defect will present a high negative Gaussian curvature, i.e., a concave feature, and a scratch defect will present a continuous curvature anomaly band along the scratch direction. The above curvature calculation precision and interval division can be adjusted according to the defect type or imaging conditions.

[0088] Step 6.5, analyze the regularity change of the triangle facet connection relationship in the envelope region, calculate the local topological structure variation degree, specifically including: first, determine the judgment benchmark of connection regularity, based on the surface topological data of the same batch of normal semiconductor card socket connectors, the triangle facets in the normal region are mostly uniform equilateral triangles, the number of adjacent facets of each vertex is stable, usually 3 to 4, the length difference of adjacent facets is less than 0.2 microns, and the facet arrangement direction has consistency; then traverse all the triangle facets in the envelope region, extract the key topological features: the number of adjacent facets of each vertex, the length of each shared edge, and the normal vector direction of each facet; then calculate the local topological variation degree: for each sub-region, consistent with the sub-region of the curvature distribution matrix, first calculate the standard deviation of the number of adjacent facets, the larger the standard deviation, the more irregular the connection, then calculate the coefficient of variation of the length of the shared edge, the coefficient of variation = standard deviation / mean, reflecting the uniformity of the length, finally calculate the angle variance of the normal vector direction, reflecting the arrangement consistency; the three indexes are weighted and summed according to the weight 3:3:4, and the local topological structure variation degree of the sub-region is obtained; the higher the variation degree value, the more irregular the topological connection of the region, the variation degree of the normal region is low, close to 0, and the variation degree of the defect region such as pinhole edge and scratch side wall will be significantly increased due to the facet distortion and stretching; finally output the variation degree value of each sub-region, form the topological variation degree distribution map corresponding to the curvature distribution matrix, and provide the core basis for the subsequent precise identification of defect types combined with the curvature and topological features.

[0089] In a preferred embodiment of the present application, the above step 7 can include:

[0090] Step 7.1. Eigenvalue decomposition is performed on the micro-surface curvature distribution matrix to extract the principal curvature distribution features representing the concentration of the curvature distribution. Specifically, the obtained micro-surface curvature distribution matrix is preprocessed to remove outliers in the matrix; by calculating the mean and standard deviation of all elements in the matrix, elements exceeding the mean ± 3 times the standard deviation are replaced with the mean to avoid the influence of individual noise points on the decomposition result; then eigenvalue decomposition is performed, which will obtain a set of eigenvalues and corresponding eigenvectors, and the size of the eigenvalue represents the energy intensity of the curvature distribution in the direction of the corresponding eigenvector; the largest two eigenvalues are selected from all eigenvalues, and the directions of the eigenvectors corresponding to the two eigenvalues are the principal directions of the most concentrated curvature distribution; for example, the principal curvature direction of a pinhole defect will be concentrated in the radial direction, and the scratch defect will be concentrated in the direction perpendicular to the scratch, and then the cumulative contribution rate of the two principal eigenvalues is calculated, cumulative contribution rate = (first principal eigenvalue + second principal eigenvalue) ÷ sum of all eigenvalues, which represents the explanatory power of the principal direction to the overall curvature distribution; finally, the first principal eigenvalue, the second principal eigenvalue and the cumulative contribution rate are integrated to form the principal curvature distribution feature set, which can accurately extract the core law of the curvature distribution of the defect area and distinguish the uniform curvature of the normal surface from the concentrated curvature anomaly of the defect.

[0091] Step 7.2. The local topological structure variation is normalized to generate a standardized topological variation coefficient. Specifically, the local topological structure variation values of all sub-regions are extracted, and the topological variation statistics of the normal semiconductor card connectors in the same batch are called to determine the normalization reference range: the maximum topological variation of the normal sample is used as the upper threshold, and the minimum topological variation of the normal sample is used as the lower threshold, to ensure that the threshold covers the topological fluctuation range of all normal surfaces; then the variation value of each sub-region is normalized by linear transformation to map the original variation value to the interval of 0 to 1; specifically, if the original variation of a sub-region is less than the lower threshold, the normalized result is 0, representing that the topography of the region is completely normal; if it is greater than the upper threshold, the normalized result is 1, representing that the topography of the region is severely abnormal; if it is within the threshold range, the standardized value is calculated as (original value - lower threshold) ÷ (upper threshold - lower threshold); after processing, the variation of all sub-regions is converted into a unified scale of standardized topological variation coefficient, eliminating the dimensional differences of the original variation of different batches and different regions, facilitating subsequent fusion with the curvature feature.

[0092] Step 7.3, the principal curvature distribution characteristics and the standardized topological variation coefficient are weighted and fused to obtain an initial local topography distortion measurement value, specifically including: the obtained principal curvature distribution characteristics are standardized; the first principal eigenvalue, the second principal eigenvalue and the cumulative contribution rate are respectively mapped to the interval of 0 to 1, the processing mode is consistent with the normalization of the topological variation coefficient in step 7.2, and the scales are unified, and then the weight of weighted fusion is determined: based on the verification result of a large number of defect samples, the sensitivity of the principal curvature distribution characteristics to the defect topography distortion is higher, and the weight is set to 0.6; the weight of the standardized topological variation coefficient is set to 0.4, and such weight distribution can more accurately capture the curvature abnormality caused by defects, while considering the irregular change of the topological structure; then the two standardized characteristics are weighted and summed according to the weight, to obtain an initial local topography distortion measurement value of each sub-region, for example, the initial measurement value of a sub-region is 0.8 for the principal curvature characteristic standardized value and 0.7 for the topological variation coefficient, that is, 0.8*0.6+0.7*0.4=0.76; the range of the initial measurement value is between 0 and 1, and the greater the value, the more serious the topography deviation of the sub-region from the normal state.

[0093] Step 7.4, the initial local topography distortion measurement value is mapped to a preset quantization interval to generate a final topography distortion quantization factor, specifically including: first, a quantization interval suitable for grading micro defects is preset, the interval range is set to 0 to 10, and according to the distortion degree, it is divided into five levels: 0 to 2 represents no obvious defects, 2 to 4 represents slight distortion, 4 to 6 represents moderate defects, 6 to 8 represents more serious defects, and 8 to 10 represents serious defects, and the interval division is determined based on the quality acceptance standard of the semiconductor card connector; then the mapping processing is performed, and the initial measurement value of 0 to 1 is corresponded to the quantization interval of 0 to 10 by using a linear mapping rule, that is, the final quantization factor=initial measurement value*10; after the mapping is completed, the rationality of the quantization factor is verified: select several groups of normal samples, slight pinhole samples and serious scratch samples in the same batch, and check whether the quantization factor falls within the corresponding level interval; if there is deviation, the mapping coefficient is adjusted to ensure accurate level division; the finally generated topography distortion quantization factor can convert the topography distortion degree of the defect into an intuitive numerical value, for example, the quantization factor of a 3 to 5 micron pinhole is usually between 4 to 6, and the quantization factor of a 5 to 10 micron scratch is between 6 to 8, which provides accurate quantization basis for subsequent automatic defect judgment and level division, and the division of the above quantization interval can be adjusted according to the defect type or imaging condition.

[0094] In a preferred embodiment of the present application, the above step 8 can include:

[0095] Step 8.1: Establish the spatial mapping relationship between the shape distortion quantization factor and the corresponding defect candidate region in the feature optimization image. Specifically, this includes: extracting all vertices of the three-dimensional sub-regions corresponding to the obtained shape distortion quantization factor, and reading the precise three-dimensional coordinates of each vertex. X , Y , Z Projecting these three-dimensional coordinates onto XY The system obtains the physical projection coordinates of each 3D sub-region onto the 2D plane, preserving the spatial positional correlation of vertices during projection to ensure that the relative positions of the 3D sub-regions are consistent with the planar projection. Simultaneously, it calls the generated feature optimization image, extracts the pixel coordinate range of the corresponding defect candidate region in the image, and retrieves the previously calibrated pixel-to-physical coordinate conversion ratio (e.g., 1 pixel corresponds to 0.1 micrometers) to clarify the correspondence between image pixel coordinates and actual physical coordinates, providing a scale benchmark for subsequent mapping. Then, three sets of key control points are selected to construct mapping bridges: the first set is the established morphological benchmark origin and its corresponding pixel in the feature optimization image; the second set is the 3D vertex at the end of the first feature radiation vector and its corresponding pixel in the image; and the third set is the 3D vertex at the end of the second feature radiation vector and its corresponding pixel in the image.

[0096] By manually verifying or edge matching, the one-to-one correspondence between the 3D physical coordinates and 2D pixel coordinates of each set of control points is accurately confirmed, ensuring that the control points cover the core defect area and are evenly distributed on the defect edge, thus guaranteeing the globality and accuracy of the mapping. Subsequently, based on the coordinate correspondence of the three sets of control points, a point-by-point mapping rule is constructed using linear interpolation: taking the control points as the reference, the planar physical projection coordinates of the 3D sub-region are divided into several uniform grids, and the corresponding pixel coordinates of each grid node in the feature optimization image are calculated. For each 3D sub-region with a shape distortion quantization factor, the corresponding image pixel coordinates are found by interpolation based on its planar projection coordinates. Finally, a one-to-one correspondence is established between each shape distortion quantization factor, the corresponding 3D sub-region, and the image pixel position, ensuring that subsequent correction operations can accurately locate the specific pixel position of the defect candidate region in the feature optimization image, achieving accurate association between 3D shape distortion information and 2D image region.

[0097] Step 8.2, based on the spatial mapping relationship and the numerical value of the topographic distortion quantization factor, calculate the geometric correction parameters of the defect candidate area, specifically including: first, according to the spatial mapping relationship, the topographic distortion quantization factor value corresponding to each image pixel position is associated, and the quantization factor distribution of each pixel position in the defect candidate area is counted; the larger the quantization factor value, the more serious the topographic distortion of the corresponding position, the larger the correction amplitude required; then set the calculation rule of the correction parameter: taking the surface flatness of the normal semiconductor card connector in the same batch as the benchmark, for the pixel area with quantization factor of 0, no distortion, the correction parameter is set to 0, no correction is required; for the pixel area with quantization factor greater than 0, the correction parameter is calculated in proportion to the quantization factor value, including translation, rotation angle and scaling coefficient; for example, the area with quantization factor of 0.6, if there is 0.3 microns of concave distortion in the three-dimensional space, combined with the conversion ratio of pixel and physical coordinates, the horizontal and vertical translation of the area in the image is calculated; if there is local distortion, the rotation angle and local scaling coefficient are calculated according to the distribution gradient of the quantization factor; finally, all the correction parameters of the pixel positions are summarized to form a correction parameter matrix matching the size of the feature optimization image, each matrix element corresponds to the correction information of a pixel, ensuring that the correction parameter can accurately cover the entire defect candidate area.

[0098] Step 8.3, through the geometric correction parameters, affine transformation correction is performed on the contour coordinates and size representation of the defect candidate area in the feature optimization image to generate the corrected defect geometric representation, specifically including: first, extract the original contour pixel coordinates of the defect candidate area in the feature optimization image, traverse each pixel point on the contour, and retrieve the translation, rotation angle and scaling coefficient of the corresponding position from the correction parameter matrix; perform affine transformation correction on each contour pixel point: first, adjust the pixel position according to the rotation angle to correct the contour deviation caused by topographic distortion; then adjust the local size according to the scaling coefficient to correct the size stretching or compression deviation in the image caused by three-dimensional topographic concave or convex; finally, fine-tune the coordinates according to the translation to ensure that the corrected contour is consistent with the true geometric shape of the defect in the three-dimensional space; at the same time, correct the size representation of the defect: according to the corrected contour coordinates, recalculate the size parameters of the defect such as width, length and area, for example, the diameter of the pinhole defect is calculated according to the diameter of the largest inscribed circle of the corrected contour, and the length of the scratch defect is calculated according to the length of the longest axis of the corrected contour, and the corrected size parameter retains an accuracy of 0.1 microns; after correction, verify the continuity of the contour: calculate the distance between adjacent pixels on the corrected contour, if there is a mutation of more than 1 pixel, re-execute smoothing correction for the area to ensure that the corrected defect geometric representation is smooth and complete.

[0099] Step 8.4, the corrected defect geometry characterization is fused with the image background to reconstruct and output the preprocessed image data after topographic distortion correction, specifically including: extracting the background pixel information around the defect candidate region in the feature-optimized image, including the gray value and texture features of the background, to ensure that the background style is uniform during fusion; then adjusting the gray value of the pixel region corresponding to the corrected defect geometry characterization - referring to the height information of the defect in the three-dimensional space, the gray value of the concave defect region is proportionally darkened to simulate the concave shadow under real light, and the gray value of the convex defect region is proportionally brightened, so that the corrected defect is more consistent with the visual presentation of the real topography in the image; then performing edge smoothing fusion: at the junction of the corrected defect region and the background, a gradient fusion algorithm is used to gradually transition the gray value of the defect edge from the defect region to the background region, avoiding obvious splicing marks and ensuring the overall visual coherence of the image; after fusion, the image quality is verified: checking whether the corrected defect contour is clear, whether the size is consistent with the actual size in the three-dimensional topological data, and whether the background is distorted. Finally, the image data that passes the verification is stored and output in a preset format. The preprocessed image data eliminates the influence of topographic distortion on defect characterization, and can be directly used for subsequent precise identification of defect type and quality level determination.

[0100] As shown in Figure 2 Embodiments of the present application also provide a semiconductor device component surface micro-defect image preprocessing data processing system, comprising:

[0101] An optimization processing module is configured to perform image quality normalization processing on the obtained high-magnification surface original image to generate a quality-enhanced image, and perform defect feature selective enhancement processing on the quality-enhanced image to generate a feature-optimized image.

[0102] A three-dimensional reconstruction module is configured to perform three-dimensional topography reconstruction on the feature-optimized image to generate surface three-dimensional topography topological data containing height information.

[0103] A determination and fitting module is configured to determine a topographic reference origin at a local topographic convex highest point of a preliminary defect candidate region according to the surface three-dimensional topography topological data, and generate a first feature radiation vector and a second feature radiation vector by fitting from the topographic reference origin along the main direction and the normal change gradient direction of the surface texture, respectively.

[0104] A delimitation and layout module is configured to delimit a multi-directional feature analysis sector according to the first feature radiation vector and the second feature radiation vector, and to layout a set of topographic sampling points arranged according to polar coordinates within the multi-directional feature analysis sector and in the adjacent reference region outside the multi-directional feature analysis sector.

[0105] A construction and calculation module is configured to construct a closed loop evaluation track surrounding a topography reference origin based on three-dimensional space coordinates of topography sampling points, and to calculate a micro-surface curvature distribution matrix and a local topological structure variation degree of an area enveloped by the closed loop evaluation track;

[0106] A fusion generation module is configured to fuse the micro-surface curvature distribution matrix and the local topological structure variation degree to generate a topography distortion quantification factor representing a local topography distortion degree;

[0107] A compensation and calibration module is configured to compensate and calibrate defect geometric representations of corresponding regions in the feature optimization image by the topography distortion quantification factor, and to output a final pre-processing image data after topography distortion correction. The above is the preferred embodiment of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered within the scope of protection of the present application.

Claims

1. A method of data processing for pre-processing images of surface microdefects of semiconductor device components, characterized in that, The method comprises: The acquired high magnification surface original image is subjected to image quality normalization processing to generate a quality enhanced image; The quality enhanced image is subjected to defect feature selective enhancement processing to generate a feature optimized image; The feature optimized image is subjected to three-dimensional topography reconstruction to generate surface three-dimensional topography topology data containing height information; Based on the surface three-dimensional topography topology data, a topographic reference origin is established at the local topographic protrusion highest point of the preliminary defect candidate region, and a first feature radiation vector and a second feature radiation vector are fitted and generated from the topographic reference origin along the main direction and the normal variation gradient direction of the surface texture respectively; According to the first feature radiation vector and the second feature radiation vector, a multi-directional feature analysis sector is defined, and a group of topographic sampling points arranged according to polar coordinate rules are arranged in the inside and the adjacent reference region outside the multi-directional feature analysis sector; Based on the three-dimensional space coordinates of the topographic sampling points, a closed ring-shaped evaluation track surrounding the topographic reference origin is constructed, and the micro-surface curvature distribution matrix and the local topological structure variation degree of the region enveloped by the closed ring-shaped evaluation track are calculated; The micro-surface curvature distribution matrix and the local topological structure variation degree are fused to generate a topographic distortion quantification factor representing the distortion degree of the local topography; The defect geometric representation of the corresponding region in the feature optimized image is compensated and calibrated through the topographic distortion quantification factor, and the final preprocessing image data subjected to topographic distortion correction is output.

2. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 1, characterized by, The acquired high magnification surface original image is subjected to image quality normalization processing to generate a quality enhanced image, comprising: Global illumination intensity analysis is performed on the acquired high magnification surface original image to generate a background illumination distribution model; According to the background illumination distribution model, pixel-by-pixel illumination intensity compensation is performed on the surface original image to obtain an illumination homogenization image; Based on the gray scale statistical features of each local region in the illumination homogenization image, an adaptive contrast stretching coefficient is dynamically calculated and applied to generate a contrast standardized image; The contrast standardized image is subjected to directional smoothing filtering to smooth the inherent texture background of the image while maintaining the integrity of the structural edges in the image, and a quality enhanced image is output.

3. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 2, characterized by, The quality enhanced image is subjected to defect feature selective enhancement processing to generate a feature optimized image, comprising: Multi-scale gradient feature extraction is performed on the quality enhanced image to generate a gradient feature map containing different scale edge information; According to the gradient feature map, the edge structure of potential defects is located on the quality enhanced image, and a corresponding edge structure mask is generated; Based on the edge structure mask, mask-guided local contrast enhancement is performed on the quality enhanced image to enhance the gray scale difference between defects and background in the mask-covered region, and a local contrast enhanced image is generated; The local contrast enhanced image is subjected to adaptive edge sharpening processing to improve the clarity and continuity of the defect outline in the edge structure mask, and a feature optimized image is output.

4. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 3, characterized by, The feature optimized image is subjected to three-dimensional topography reconstruction to generate surface three-dimensional topography topology data containing height information, comprising: Structure light phase analysis is performed on the feature optimized image to extract a phase gradient field representing the surface micro-relief; According to the extracted phase gradient field, an initial three-dimensional point cloud seed set of surface height is generated by performing integral reconstruction calculation; Based on the principle of photometric stereo vision, the initial three-dimensional point cloud seed set is iteratively optimized by using feature optimization images to correct the error of height data and fill in the missing area, and a dense three-dimensional point cloud model is generated; The dense three-dimensional point cloud model is triangulated and the topological relationship is constructed to form the surface three-dimensional topography topological data containing continuous height information and spatial connection relationship.

5. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 4, characterized by, Based on the surface three-dimensional topography topological data, a topographic reference origin is established at the highest point of the local topographic protrusion in the preliminary defect candidate area, and a first characteristic radiation vector and a second characteristic radiation vector are fitted along the main direction and the normal change gradient direction of the surface texture from the topographic reference origin, including: According to the height field of the surface three-dimensional topography topological data, the preliminary topographic boundary of the defect candidate area is calculated and obtained; In the area enclosed by the preliminary topographic boundary, the vertex with the maximum height value is located, and the vertex with the maximum height value is established as the topographic reference origin; With the topographic reference origin as the center, the normal vector statistical analysis is performed on the surface triangular facets in its neighborhood to determine the main direction of the texture trend, and the first characteristic radiation vector is fitted along the main direction; The surface curvature in each direction in the neighborhood of the topographic reference origin is calculated, the direction with the maximum absolute value of curvature is determined as the characteristic direction of the normal change, and the second characteristic radiation vector is fitted along the characteristic direction.

6. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 5, characterized by, According to the first characteristic radiation vector and the second characteristic radiation vector, a multi-directional feature analysis sector is defined, and a group of topographic sampling points arranged according to polar coordinates are arranged in the inside and the adjacent reference area outside the multi-directional feature analysis sector, including: Taking the topographic reference origin as the vertex and the first characteristic radiation vector and the second characteristic radiation vector as the boundary, an initial analysis sector is constructed; According to the continuity of the surface three-dimensional topography topological data at the edge of the initial analysis sector, the boundary of the initial analysis sector is adaptively expanded to form the final multi-directional feature analysis sector; In the final multi-directional feature analysis sector, a first group of internal sampling points is generated according to a preset radial step and angular interval; In the adjacent reference area outside the final multi-directional feature analysis sector, a second group of external sampling points is generated according to a preset offset distance and density rule along the normal extension direction of the sector boundary; The first group of internal sampling points and the second group of external sampling points are merged to form the group of topographic sampling points arranged according to polar coordinates.

7. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 6, characterized by, Based on the three-dimensional space coordinates of the topographic sampling points, a closed ring-shaped evaluation trajectory around the topographic reference origin is constructed, and the micro-surface curvature distribution matrix and the local topological structure variation degree of the region enveloped by the closed ring-shaped evaluation trajectory are calculated, including: According to the relative angle and distance between the topographic sampling points and the topographic reference origin, the topographic sampling points are sorted and connected in segments to form a preliminary ring-shaped path; The preliminary ring-shaped path is smoothed and optimized and the first and last ends are closed to generate the final closed ring-shaped evaluation trajectory; determining a surface region enclosed by the final closed loop evaluation trajectory, extracting all vertices in the surface three-dimensional topography topology data within the surface region; calculating the local curvature features of each vertex within the enclosed region, and counting the distribution to construct a micro-surface curvature distribution matrix; analyzing the regularity of the connection relationship of the triangular patches within the enclosed region, and calculating the local topological structure variation degree.

8. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 7, characterized by, fusing the micro-surface curvature distribution matrix and the local topological structure variation degree to generate a topography distortion quantification factor representing the distortion degree of the local topography, including: performing eigenvalue decomposition on the micro-surface curvature distribution matrix to extract the principal curvature distribution features representing the concentration degree of the curvature distribution; normalizing the local topological structure variation degree to generate a standardized topological variation coefficient; performing weighted fusion on the principal curvature distribution features and the standardized topological variation coefficient to calculate an initial local topography distortion metric value; mapping the initial local topography distortion metric value to a preset quantization interval to generate a final topography distortion quantification factor.

9. The semiconductor device component surface minute defect image pre-processing data processing method according to claim 8, characterized by, Through the topography distortion quantification factor, the defect geometric representation of the corresponding region in the feature optimization image is compensated and calibrated, and the final preprocessed image data after topography distortion correction is output, including: establishing a spatial mapping relationship between the topography distortion quantification factor and the corresponding defect candidate region in the feature optimization image; based on the spatial mapping relationship and the value of the topography distortion quantification factor, calculating the geometric correction parameters of the defect candidate region; through the geometric correction parameters, performing affine transformation correction on the contour coordinates and size representation of the defect candidate region in the feature optimization image to generate a corrected defect geometric representation; fuse the corrected defect geometric representation with the image background to reconstruct and output the preprocessed image data after topography distortion correction.

10. A system for data processing of surface micro-defect images of semiconductor device components, the system implementing the method according to any one of claims 1 to 9, characterized in that, including: an optimization processing module for performing image quality normalization processing on the obtained high magnification surface original image to generate a quality enhanced image; performing selective enhancement processing on the quality enhanced image to generate a feature optimization image; a three-dimensional reconstruction module for performing three-dimensional topography reconstruction on the feature optimization image to generate surface three-dimensional topography topology data containing height information; establishing and fitting module for establishing a topographic reference origin at the local topographic highest point of the preliminary defect candidate region according to the surface three-dimensional topography topology data, and fitting to generate a first feature radiation vector and a second feature radiation vector along the main direction and the normal change gradient direction of the surface texture respectively from the topographic reference origin; defining and laying out module for defining a multi-directional feature analysis sector according to the first feature radiation vector and the second feature radiation vector, and laying out a set of topographic sampling points arranged according to polar coordinates within and outside the adjacent reference region in the multi-directional feature analysis sector; a construction and calculation module for constructing a closed loop evaluation trajectory around the topographic reference origin based on the three-dimensional space coordinates of the topographic sampling points, and calculating the micro-surface curvature distribution matrix and the local topological structure variation degree of the enclosed region of the closed loop evaluation trajectory; The fusion generation module is configured to fuse the micro-surface curvature distribution matrix and the local topological structure variation to generate a topography distortion quantification factor representing a distortion degree of the local topography; The compensation and calibration module is configured to compensate and calibrate the defect geometric representation of the corresponding region in the feature optimization image by using the topography distortion quantification factor, and output a preprocessed image data after topography distortion correction.

Images