A soldering pad needle mark detection method based on shape self-adaptation and multi-dimensional feature fusion

Abstract

This invention discloses a pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion. The method acquires pad image data and determines the pad type (e.g., rectangular, square, or circular) based on the aspect ratio and roundness factor. According to the pad type, edge partitioning and multi-scale adaptive determination are performed on the pads in the image data. A non-maximum suppression strategy and texture clustering based on spatial collinearity are applied to several candidate pin mark targets to eliminate redundant pin mark targets, obtaining several final pin mark targets. The pixel coordinates of these final pin mark targets are converted to normalized center coordinates through coordinate inverse mapping and format normalization. Marking boxes are drawn in the pad image data based on the normalized center coordinates of the final pin mark targets to mark their positions. The method adapts to the pad shape, suppresses texture interference, and separates edge noise, outputting pad image data with pin mark marking boxes.

Description

Technical Field

[0001] This invention relates to the technical field of pad pin mark detection, and in particular to a pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion. Background Technology

[0002] In the integrated circuit manufacturing process, wafer probing (CP) is a core step connecting wafer manufacturing and packaging testing. Probe cards use micro-probes to contact chip pads and apply electrical signals; the resulting pin marks contain crucial process information. The depth and shape of these pin marks directly reflect contact quality: too shallow a mark may lead to the rejection of good products, while too deep a mark may damage the passivation layer and introduce reliability risks. Therefore, high-precision automated inspection of pin marks is essential for monitoring probe card status and ensuring chip yield. Faced with advanced processes, traditional manual visual inspection is inefficient and highly subjective. Existing automated inspection technologies based on template matching or fixed thresholds struggle to meet industrial demands in terms of adaptability and accuracy, especially when dealing with complex textured backgrounds and irregularly shaped pads, facing two major bottlenecks: feature confusion and uneven light field.

[0003] Existing detection technologies face severe challenges in practical applications, mainly in four interconnected aspects: First, the generalization problem arising from the diversity of pad shapes makes it difficult for traditional single operators to take into account the vastly different light and shadow distributions of pads with different shapes, leading to missed detections or false alarms; second, long and narrow pads have significant surface brightness gradients due to their large span, and fixed threshold algorithms cannot take into account the global situation, urgently requiring a partitioned detection strategy that adapts to local illumination; on this basis, the strong interference from the surface texture of the pads further exacerbates the detection difficulty, as the grain microstructure formed during aluminum metal sputtering deposition and the surface morphology, as well as the residue from the passivation layer opening etching, all contribute to the challenge. Edge burrs and slight scratches accumulated on the pad surface by repeated probe contacts are inherent textures within the pads that are extremely similar to real pin marks in terms of grayscale and gradient features. They are often regularly distributed along a specific direction. Relying solely on single-point feature judgment can easily lead to false defect judgments. It is necessary to introduce multi-dimensional feature fusion and macroscopic spatial distribution analysis to eliminate such noise within the pads. Finally, in terms of edge effects and multi-scale detection, micron-level pin marks are often hidden in the high-brightness gradient bands at the edge of the pads. Furthermore, high-contrast deep holes and low-contrast shallow scratches coexist on the same wafer. Fixed-parameter algorithms cannot simultaneously achieve noise suppression and high sensitivity. Summary of the Invention

[0004] Purpose of the invention: In order to overcome the shortcomings of the existing technology, the present invention provides a pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion. It can adapt to the shape of the pad, suppress the interference of internal texture of the pad and accurately separate edge noise, accurately detect pin marks on the pad, and output pad image data with pin mark marking boxes.

[0005] Technical Solution: To achieve the above objectives, the present invention provides a method for detecting solder pad pin marks based on shape adaptation and multi-dimensional feature fusion, comprising the following steps:

[0006] S1. Obtain pad image data and construct an adaptive classifier of geometric moments. Determine whether the pad type is rectangular, square, or circular based on the aspect ratio and roundness factor of the pad in the pad image data.

[0007] S2. Based on the pad type, the pads in the pad image data are divided into edge partitions and multi-scale adaptive judgments. When the pad type is long and narrow, the overlapping sliding window partition detection strategy is used to equalize the local illumination of the long and narrow pad. The pin mark detection is performed by the constructed three-channel candidate extraction model based on local statistical characteristics to extract several initial candidate regions. A multi-dimensional feature fusion quality scoring system is constructed to calculate the confidence score of each initial candidate region. Based on the confidence score of each initial candidate region, edge distance partitioning and spatial clustering analysis, non-pin mark targets are filtered and removed to select several candidate pin mark targets.

[0008] When the pad type is square, a combination of local adaptive threshold and gradient direction verification is used to detect pin marks on square pads and select several candidate pin mark targets; when the pad type is circular, a detection strategy based on region growth and concentric partition statistical verification is used to detect pin marks on circular pads and select several candidate pin mark targets.

[0009] S3. Perform non-maximum suppression strategy and texture clustering operation based on spatial collinearity on several candidate needle mark targets to remove redundant needle mark targets and obtain several final needle mark targets.

[0010] S4. Convert the pixel coordinates of several final pin mark targets into normalized center coordinates through coordinate inverse mapping and format normalization. Draw a marker box in the pad image data according to the normalized center coordinates of several final pin mark targets to mark the position of several final pin mark targets.

[0011] Furthermore, in step S1, inputting the pad image data into an adaptive classifier of geometric moments to determine the pad type includes the following steps:

[0012] S1-1. Calculate the aspect ratio of the pads in the pad image data, and compare the aspect ratio of the pads with the set aspect ratio threshold. When the aspect ratio of the pads is greater than or equal to the aspect ratio threshold, the pad type is determined to be a long strip pad. When the aspect ratio of the pads is less than the aspect ratio threshold, dark corner pixel detection is performed in the four corner pixel detection.

[0013] S1-2. Compare the detected dark corner pixel value with the dark corner threshold; when the dark corner pixel value is less than the dark corner threshold, the pad type is determined to be a circular pad; when the dark corner pixel value is greater than or equal to the dark corner threshold, bright corner pixel detection is performed.

[0014] S1-3. Compare the detected bright corner pixel value with the bright corner threshold; when the bright corner pixel value is greater than or equal to the bright corner threshold, the pad type is determined to be a square pad; when the bright corner pixel value is less than the bright corner threshold, the outline roundness of the pad in the pad image data is calculated.

[0015] S1-4. Compare the calculated outline roundness with the set roundness threshold. If the outline roundness is greater than or equal to the roundness threshold, the pad type is determined to be a circular pad; if the outline roundness is less than the roundness threshold, the pad type is determined to be a square pad.

[0016] Furthermore, in step S2, when the pad type is elongated, an overlapping sliding window partitioning detection strategy is adopted to equalize the local illumination of the elongated pad, dividing the elongated pad into several sub-regions along the long side, with a certain proportion of overlap between adjacent sub-regions, and applying bilateral filtering to each of the several sub-regions; the several sub-regions after bilateral filtering are sequentially filtered by the parallel high contrast channel, dark pixel channel and mixed channel in the three-channel candidate extraction model based on local statistical characteristics to obtain several initial candidate regions.

[0017] Furthermore, in the multi-dimensional feature fusion quality scoring system, four physical dimensions—darkness score, roundness score, contrast score, and morphological score—are comprehensively considered to calculate the confidence score of each initial candidate region; the calculation formula is shown below:

[0018]

[0019] In the formula, S dark Represented as a darkness rating, S circ Represented as a roundness score, S cont Represented as a contrast rating, S morph The morphological score is represented by W1, W2, W3, and W4, which are the weighting coefficients for the octane score, roundness score, contrast score, and morphological score, respectively.

[0020] Furthermore, in the edge distance partitioning, the minimum Euclidean distance from the center of each initial candidate region to the boundary of the elongated pad is calculated. If the minimum Euclidean distance of any initial candidate region is less than a set distance threshold, the initial candidate region is marked as an edge region. Initial candidate regions marked as edge regions are judged using a high confidence threshold. If the confidence score of an initial candidate region marked as an edge region is greater than the high confidence threshold, the initial candidate region marked as an edge region is determined to be a candidate pin mark target. If the confidence score of an initial candidate region marked as an edge region is less than or equal to the high confidence threshold, the initial candidate region marked as an edge region is determined to be a non-pin mark target.

[0021] If the minimum Euclidean distance of any initial candidate region is greater than or equal to the set distance threshold, the initial candidate region is marked as an internal region. Initial candidate regions marked as internal regions are judged using a low confidence threshold. If the confidence score of an initial candidate region marked as an internal region is greater than the low confidence threshold, the initial candidate region marked as an internal region is judged as a candidate pin mark target. If the confidence score of an initial candidate region marked as an internal region is less than or equal to the low confidence threshold, the initial candidate region marked as an internal region is judged as a non-pin mark target.

[0022] Furthermore, when the pad type is square, a combination of local adaptive thresholding and gradient direction verification is used to perform pin mark detection on the square pad, filtering out several candidate pin mark targets; including the following steps:

[0023] S2-1. Calculate the pixel statistics of the internal region of the square pad to obtain the mean and standard deviation; calculate the dynamic binarization threshold using the mean and standard deviation; segment the Gaussian blurred pad image data in parallel based on the dynamic binarization threshold and Otsu's maximum inter-class variance method to obtain several dark candidate regions.

[0024] S2-2. Calculate the Sobel gradient field of dark candidate regions near the boundary of the square pad. When the gradient magnitude Mag of any dark candidate region exceeds the global edge intensity P... 60 When the percentile and the gradient direction θ shows a high degree of consistency, the dark candidate region is determined to be an edge gradient artifact and is removed to obtain the retained dark candidate region.

[0025] S2-3. Set multi-dimensional feature constraints, including photometric constraints, morphological constraints and positional constraints, for the retained dark candidate regions. Perform geometric and photometric joint screening on the retained dark candidate regions to select several candidate needle mark targets.

[0026] Furthermore, when the pad type is circular, a detection strategy based on region growth and concentric partitioning statistical verification is used to perform pin mark detection on the circular pads, and several candidate pin mark targets are screened out; including the following steps:

[0027] S3-1. The pure detection area is extracted using the edge reverse region growing method to obtain several non-connected needle mark areas;

[0028] S3-2, The distance transformation-based segmentation algorithm decomposes the connected regions of the adhesive needle marks into several independent needle mark regions;

[0029] S3-3. Based on the Euclidean distance from the pixel in the circular pad to the geometric center of the circular pad, divide the circular pad into three concentric regions: the core region, the middle region, and the edge region.

[0030] S3-4. Using the needle mark regions in the core area as reference samples, calculate the weighted standardized distance between the needle mark regions in the central and edge areas and the reference sample distribution in the core area. When the needle mark region is located in the central area, compare the weighted standardized distance with a lenient threshold. If the weighted standardized distance is less than or equal to the lenient threshold, the needle mark region is determined to be a candidate needle mark target. When the needle mark region is located in the edge area, compare the weighted standardized distance with a strict threshold. If the weighted standardized distance is less than or equal to the strict threshold, the needle mark region is determined to be a candidate needle mark target.

[0031] Furthermore, in step S3, the non-maximum suppression strategy sorts several candidate pin mark targets in descending order of their confidence scores, selects the candidate pin mark targets with the highest confidence scores as retained candidate pin mark targets, calculates the Euclidean distance between the center point of the remaining candidate pin mark targets with the highest confidence scores and the retained candidate pin mark targets, compares the center point Euclidean distance with a dynamic overlap threshold, and when the center point Euclidean distance is lower than the dynamic overlap threshold, the candidate pin mark target with the highest confidence scores among the remaining candidate pin mark targets is removed.

[0032] Furthermore, in step S3, the texture clustering operation based on spatial collinearity constructs a horizontal clusterer and a vertical clusterer. The horizontal clusterer groups candidate needle marks horizontally based on the center ordinate of the candidate needle mark targets, and the vertical clusterer groups candidate needle marks vertically based on the center abscissa of the candidate needle mark targets. For any non-empty group, the distance between adjacent needle mark targets is calculated after sorting according to the principal axis coordinates. When the maximum distance between adjacent needle mark targets is greater than the closeness condition threshold, all needle mark targets in the non-empty group are determined to be non-needle mark targets, and the non-empty group is removed. For any non-empty group, if the number of needle mark targets in the non-empty group is greater than or equal to 2, all needle mark targets in the non-empty group are determined to be non-needle mark targets, and the non-empty group is removed.

[0033] Beneficial Effects: This invention provides a pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion. The method categorizes pads into three types—strip, square, and circular—based on aspect ratio and roundness factor, laying the foundation for subsequent differentiated processing. A partitioned detection strategy using overlapping sliding windows ensures equal local illumination and prevents pin marks on the segmentation boundaries from being truncated. A quality scoring system based on multi-dimensional feature fusion and edge distance partitioning distinguishes pin marks from textures from a microscopic feature perspective. A combination of local adaptive thresholding and gradient direction verification is used to detect pin marks on square pads, solving the problems of sharpening artifacts and uneven illumination at right-angled edges. A detection strategy based on region growing and concentric partitioning statistical verification is used to detect pin marks on circular pads, maintaining high sensitivity to weak pin marks while suppressing edge noise. Attached Figure Description

[0034] Figure 1 The flowchart shows a pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion.

[0035] Figure 2 A flowchart of the workflow for an adaptive classifier for geometric moments;

[0036] Figure 3 A schematic diagram of the partition detection strategy for overlapping sliding windows;

[0037] Figure 4 A schematic diagram of a quality scoring system that integrates multi-dimensional features;

[0038] Figure 5 This is a schematic diagram of spatial clustering analysis filtering.

[0039] Figure 6 This is a schematic diagram of gradient suppression for square pads;

[0040] Figure 7 A schematic diagram showing the division of a circular pad into three concentric regions. Detailed Implementation

[0041] The invention will now be further described with reference to the accompanying drawings.

[0042] like Figure 1 As shown, a method for detecting solder pad pin marks based on shape adaptation and multi-dimensional feature fusion includes the following steps:

[0043] S1. Obtain pad image data and construct an adaptive classifier of geometric moments. Determine whether the pad type is rectangular, square, or circular based on the aspect ratio and roundness factor of the pad in the pad image data.

[0044] S2. Based on the pad type, the pads in the pad image data are divided into edge partitions and multi-scale adaptive judgments. When the pad type is long and narrow, the overlapping sliding window partition detection strategy is used to equalize the local illumination of the long and narrow pad. The pin mark detection is performed by the constructed three-channel candidate extraction model based on local statistical characteristics to extract several initial candidate regions. A multi-dimensional feature fusion quality scoring system is constructed to calculate the confidence score of each initial candidate region. Based on the confidence score of each initial candidate region, edge distance partitioning and spatial clustering analysis, non-pin mark targets are filtered and removed to select several candidate pin mark targets.

[0045] When the pad type is square, a combination of local adaptive threshold and gradient direction verification is used to detect pin marks on square pads and select several candidate pin mark targets; when the pad type is circular, a detection strategy based on region growth and concentric partition statistical verification is used to detect pin marks on circular pads and select several candidate pin mark targets.

[0046] S3. Perform non-maximum suppression strategy and texture clustering operation based on spatial collinearity on several candidate needle mark targets to remove redundant needle mark targets and obtain several final needle mark targets.

[0047] S4. Convert the pixel coordinates of several final pin mark targets into normalized center coordinates through coordinate inverse mapping and format normalization. Draw a marker box in the pad image data according to the normalized center coordinates of several final pin mark targets to mark the position of several final pin mark targets. Output the pad image data with pin mark marker boxes.

[0048] like Figure 2 As shown, in step S1, inputting the pad image data into an adaptive classifier of geometric moments to determine the pad type includes the following steps:

[0049] S1-1. Calculate the aspect ratio of the pads in the pad image data, and compare the aspect ratio of the pads with the set aspect ratio threshold. When the aspect ratio of the pads is greater than or equal to the aspect ratio threshold, the pad type is determined to be a long strip pad. When the aspect ratio of the pads is less than the aspect ratio threshold, dark corner pixel detection is performed in the four corner pixel detection.

[0050] The adaptive classifier for geometric moments first reads the coordinates of the pad bounding box in the pad image data, calculates the width W and height H of the pad, and then calculates the aspect ratio discriminant F of the pad based on the width and height. aspect The calculation process is as follows:

[0051]

[0052] In the formula, Represented as a tiny constant; when the aspect ratio F of the pad...aspect When the length and width thresholds are greater than or equal to the thresholds, the pad is identified as having significant linear features, is determined to be a long strip pad, and a texture suppression algorithm for the long strip pad target is automatically triggered; the length and width thresholds are set to 2.2.

[0053] S1-2. Compare the detected dark corner pixel value with the dark corner threshold; when the dark corner pixel value is less than the dark corner threshold, the pad type is determined to be a circular pad; when the dark corner pixel value is greater than or equal to the dark corner threshold, bright corner pixel detection is performed.

[0054] S1-3. Compare the detected bright corner pixel value with the bright corner threshold; when the bright corner pixel value is greater than or equal to the bright corner threshold, the pad type is determined to be a square pad; when the bright corner pixel value is less than the bright corner threshold, the outline roundness of the pad in the pad image data is calculated; both the dark corner threshold and the bright corner threshold are set to 3.

[0055] S1-4. Compare the calculated outline roundness with the set roundness threshold. If the outline roundness is greater than or equal to the roundness threshold, the pad type is determined to be a circular pad; if the outline roundness is less than the roundness threshold, the pad type is determined to be a square pad.

[0056] For non-elongated pads, further extract the brightness distribution and contour features of the four corner pixels from the pad image data, and calculate the roundness factor F. circ Through the roundness factor F circ The calculation process for determining the pad type is shown below:

[0057]

[0058] In the formula, S represents the area of ​​the pad outline, and L represents the perimeter of the pad outline; when the outline roundness F circ If the roundness threshold is greater than or equal to the circularity threshold, the pad type is determined to be a circular pad; otherwise, the pad type is determined to be a square pad. This ensures that subsequent algorithms can perform targeted processing for the physical characteristics of different pads, such as edge halo and surface texture. The roundness threshold is set to 0.7.

[0059] like Figure 3As shown, due to the large span of the long strip pad, the incident angle of the light source during imaging generates a significant brightness gradient on the pad surface, causing traditional global threshold segmentation methods to easily miss detections in bright areas and make false alarms in dark areas. A high recall rate for detecting long-span pads is achieved through a local illumination equalization and multi-channel parallel candidate extraction mechanism. In step S2, when the pad type is long strip, an overlapping sliding window partitioning detection strategy is used to equalize the local illumination of the long strip pad, dividing it into several sub-regions along its long side. Adjacent sub-regions maintain a certain proportion of overlap, and a bilateral filter operation is applied to each sub-region to remove high-frequency noise while preserving edge details, thus performing illumination-adaptive detection. The certain proportion of overlap between adjacent sub-regions means that there is a certain percentage of overlapping areas between two adjacent sub-regions; this certain percentage can be 25%. That is, 25% of the areas in two adjacent sub-regions are completely identical.

[0060] A three-channel candidate extraction model based on local statistical characteristics is used to sequentially filter several sub-regions after bilateral filtering, using parallel high-contrast channels, extremely dark pixel channels, and mixed channels, to obtain several initial candidate regions. First, the local mean background μ(x, y) of the pad image data is calculated, and an absolute contrast map M is generated. diff (x, y), the calculation process is as follows:

[0061]

[0062] In the formula, I(x, y) represents the pixel value in the pad image data.

[0063] Among the parallel high-contrast channel, dark pixel channel, and blending channel, in the high-contrast channel, for targets with a relatively bright background but clear needle mark edges, the absolute contrast M is selected. diff In the high-contrast channel, areas with pixel values ​​below a set pixel threshold are selected; in the extremely dark pixel channel, for deep hole-type needle marks with extremely dark colors and potentially insignificant contrast, pixel areas with gray values ​​below an extremely low threshold are directly extracted, where the extremely low threshold can be 92; in the mixed channel, for weak needle marks in shadow areas or mid-tones, a combination of medium darkness and strong contrast is used for screening; the union of several sub-regions selected from the high-contrast channel, the extremely dark pixel channel, and the mixed channel is used as several initial candidate regions, thereby minimizing the risk of missed detection.

[0064] The surface of the solder pads exhibits inherent texture interference from various sources. The microstructure of the grains formed during aluminum sputtering deposition results in an uneven granular or striped morphology. Edge burrs and micro-pits remain from the passivation layer etching process, while shallow scratches accumulate on the pad surface from probe contact during repeated wafer probing. These inherent textures within the pads are extremely similar to real pin marks in grayscale and gradient characteristics, and often exhibit a regular distribution along the metal grain growth direction or probe movement trajectory. Relying solely on single-point features for judgment can easily lead to a large number of false defect detections. Therefore, a quality scoring system based on multi-dimensional feature fusion, edge distance partitioning, and spatial clustering analysis is used to filter and remove non-pin mark targets, thereby suppressing texture interference and identifying several candidate pin mark targets.

[0065] like Figure 4 As shown, in order to remove noise from a large number of initial candidate regions, the quality scoring system based on multi-dimensional feature fusion comprehensively considers four physical dimensions: darkness score, roundness score, contrast score, and morphological score, and calculates the confidence score for each initial candidate region; the calculation formula is shown below:

[0066]

[0067] In the formula, S dark Represented as a darkness score, it is based on a negative correlation function of pixel grayscale values; the darker the pixel, the higher the score, simulating the human eye's sensitivity to black points; S circ Represented by a roundness score, rewards nearly circular spots and penalizes elongated lines. Needle marks are typically circular or oval indentations, while scratches or textures are linear; S cont Represented as a contrast score, it is based on the grayscale difference between the target and its local neighborhood. The higher the contrast, the more prominent the target is, and the higher the score. morph The morphological score is represented by a strong constraint of aspect ratio and solidity to address the pain point of severe texture interference in elongated pads. A penalty function is designed in the mathematical model, and the score drops sharply when the aspect ratio of the initial candidate region exceeds 2.0. W1, W2, W3 and W4 represent the weight coefficients of the darkness score, roundness score, contrast score and morphological score, respectively, W1+W2+W3+W4=1; W1=0.25, W2=0.20, W3=0.25, W4=0.30.

[0068] Considering the gradient bands caused by metallic reflections at the edges of solder pads, which can easily lead to false alarms, a spatial position awareness mechanism is introduced to partition the edges by distance. This edge distance partitioning calculates the minimum Euclidean distance from the center of each initial candidate region to the boundary of the elongated solder pad. If the minimum Euclidean distance of any initial candidate region is less than a set distance threshold, the initial candidate region is marked as an edge region. Initial candidate regions marked as edge regions are judged using a high confidence threshold. If the confidence score of an initial candidate region marked as an edge region is greater than the high confidence threshold, it is determined to be a candidate pin mark target. If the confidence score of an initial candidate region marked as an edge region is less than or equal to the high confidence threshold, it is determined to be a non-pin mark target. Simultaneously, morphological filtering conditions are used to filter non-pin mark targets in the initial candidate regions. If the aspect ratio of an initial candidate region is less than a threshold of 1.8, it is determined to be a non-pin mark target and removed to prevent edge jaggedness from being mistakenly identified as pin marks.

[0069] If the minimum Euclidean distance of any initial candidate region is greater than or equal to a set distance threshold, the initial candidate region is marked as an internal region. Initial candidate regions marked as internal regions are judged using a low confidence threshold to ensure that weak needle marks can be detected. If the confidence score of an initial candidate region marked as an internal region is greater than the low confidence threshold, the initial candidate region marked as an internal region is determined to be a candidate needle mark target. If the confidence score of an initial candidate region marked as an internal region is less than or equal to the low confidence threshold, the initial candidate region marked as an internal region is determined to be a non-needle mark target. The high confidence threshold is set to 0.55, the low confidence threshold is set to 0.40, and the distance threshold is set to 8 pixels.

[0070] like Figure 5 As shown, the spatial clustering analysis scans initial candidate regions along both the horizontal X-axis and vertical Y-axis. When any group of initial candidate regions is found to be continuously arranged on the same horizontal or vertical line, and the number of initial candidate regions in a group exceeds a threshold of 2, the spatial distribution regularity is used to determine that the group of initial candidate regions is metal grain texture or accumulated scratches on the pad surface, and the entire group of initial candidate regions is removed. If the Y-axis tolerance between several initial candidate regions is less than or equal to 5 pixels, then the several initial candidate regions are determined to be continuously arranged on the same horizontal line; if the X-axis tolerance between several initial candidate regions is less than or equal to 5 pixels, then the several initial candidate regions are determined to be continuously arranged on the same vertical line.

[0071] like Figure 6As shown, the difficulty in detecting square pads lies in the sharpening artifacts at right-angled edges and uneven illumination. When the pad type is square, a combination of local adaptive thresholding and gradient direction verification is used to detect pin marks on square pads, filtering out several candidate pin mark targets; including the following steps:

[0072] S2-1. Calculate the pixel statistics of the internal region of the square pad to obtain the mean and standard deviation. Calculate the dynamic binarization threshold using the mean and standard deviation. Based on the dynamic binarization threshold and Otsu's maximum inter-class variance method, segment the Gaussian-blurred pad image data in parallel to obtain several dark candidate regions. Specifically, perform parallel segmentation based on dynamic threshold and Otsu's maximum inter-class variance method on the Gaussian-blurred pad image. Perform a bitwise AND operation on the binarized mask regions obtained from the two parallel segmentations to find the intersection, and then use morphological opening to remove noise, thereby selecting several high-confidence dark candidate regions. The edges of the pad image data are cropped, and a certain percentage is cropped to obtain the internal region of the square pad, for example, 20%. Calculate the pixel statistics of the internal region of the square pad, including the mean μ. patch and standard deviation σ patch ; through the mean μ patch and standard deviation σ patch Calculate the dynamic binarization threshold T bin The calculation process for the dynamic binarization threshold is as follows:

[0073]

[0074] In the formula, k represents the sensitivity coefficient, and k is taken as 0.3.

[0075] S2-2. Calculate the Sobel gradient field of dark candidate regions near the boundary of the square pad. When the gradient magnitude Mag of any dark candidate region exceeds the global edge intensity P... 60 If the gradient direction θ shows a high degree of consistency, then the dark candidate region is determined to be an edge gradient artifact and is removed, resulting in several retained dark candidate regions. If the gradient magnitude Mag of any dark candidate region exceeds the global edge intensity P... 60 When the gradient direction θ shows a high degree of consistency (i.e., the variance is very small, indicating that it is part of the edge line rather than an independent needle mark), the dark candidate region is identified as an edge gradient artifact and is removed.

[0076] Calculate the Sobel gradient field of the dark candidate region near the boundary of the square pad. The calculated Sobel gradient field of the dark candidate region includes the gradient magnitude Mag of all pixels in the dark candidate region and the gradient direction of all pixels in the dark candidate region. Take the 60th percentile (P60) of the global edge intensity as the intensity index. If there is a gradient magnitude Mag of all pixels in the dark candidate region that exceeds the intensity index, it indicates that the region edge is extremely sharp. If the gradient directions of most pixels in the dark candidate region are clustered at a specific angle, that is, the gradient directions show high consistency, it indicates that the region is a linear structure rather than a divergent pinhole. When the condition "the gradient direction of all pixels in the dark candidate region exceeds the intensity index" is met, the gradient field is considered to be a region with a gradient magnitude Mag of 60th percentile (P60) ... 60 When the conditions of "percentile" and "high consistency of gradient direction" are met, the dark candidate region is determined to be an edge gradient artifact and is removed.

[0077] The straight edges of square pads often produce ringing effects or false edges due to image compression or sharpening algorithms. To eliminate such false alarms, gradient vector analysis is introduced to calculate the Sobel gradient field of the dark candidate region near the boundary of the square pad. The calculation process is shown below:

[0078]

[0079]

[0080] In the formula, I represents the grayscale value function, and x and y represent the spatial coordinates of the pixel.

[0081] S2-3. For several retained dark candidate regions, set multi-dimensional feature constraints including photometric, morphological, and positional constraints. Perform geometric and photometric joint screening on these regions to select several candidate pin mark targets. Then, sequentially perform joint screening on the retained dark candidate regions using the multi-dimensional feature constraints of photometric, morphological, and positional constraints. The photometric constraint requires that the average brightness of the dark candidate region must be lower than the background mean μ of the area inside the square pad. patch And below the dynamic binarization threshold T bin To ensure the target has significant dark features, dark candidate regions are eliminated or removed if they do not. The morphological constraint requires that the aspect ratio of dark candidate regions be less than 3.0 to prevent fine lines at the edges from being misjudged as needle marks. If the aspect ratio of dark candidate regions is greater than or equal to 3.0, the dark candidate regions are eliminated or removed. The positional constraint applies stricter area and roundness threshold filtering to dark candidate regions located at the extreme edges of the pad image data and with abnormal aspect ratios.

[0082] like Figure 7As shown, micron-sized pin marks are often hidden in the high-brightness gradient band at the edge of the pad, and high-contrast deep holes and low-contrast shallow scratches often coexist on the same wafer. Fixed-parameter algorithms struggle to maintain high sensitivity to weak pin marks while suppressing edge noise. Circular pads typically have regular geometric edges, but are easily affected by edge reflection gradients. Therefore, when the pad type is circular, a detection strategy based on region growth and concentric partitioning statistical verification is used to detect pin marks on circular pads, screening out several candidate pin mark targets. The steps include:

[0083] S3-1. The reverse edge region growing method is used to remove edge interference and extract a clean detection region, resulting in several non-connected pin mark regions. To eliminate the interference of bright halo around the pad edges on the detection of weak internal pin marks, the reverse edge region growing method is used to extract a clean detection region. Seed point sets are selected from the top, bottom, left, and right boundaries of the pad, respectively. For each seed point in the seed point set, it is determined whether any pixel I(x, y) in the neighborhood of the seed point satisfies the growth criterion. If it satisfies the growth criterion, the neighborhood of the seed point is marked as an edge gradient region M. edge The growth criteria are as follows:

[0084]

[0085] In the formula, I(x, y) represents the pixel grayscale value of any pixel within the neighborhood of the seed point. seed T represents the pixel grayscale value of the current seed point. grow This represents the growth threshold, which can be set to 15.

[0086] Set the binarized mask area M for the circular pad. pad This will effectively detect mask M valid Defined as the binarized mask region M of the circular pad pad With the edge gradient region M edge The difference set is calculated as follows:

[0087]

[0088] The above steps, through physical connectivity analysis, accurately eliminated edge halos with gradient characteristics while retaining non-connected pinhole regions.

[0089] S3-2. The distance transformation-based segmentation algorithm decomposes the connected regions of the adhesive pin marks into several independent pin mark regions. For the common phenomenon of pin mark overlap or adhesion in circular pads, a distance transformation-based segmentation algorithm is introduced to calculate the distance transformation map D(x,y) of the connected regions of the adhesive pin marks. The distance transformation map is represented as the Euclidean distance from the pixel to the nearest background point. The determined foreground center Sure Foreground is obtained through the thresholded distance transformation map D(x,y). The foreground center is used as the initial seed for watershed transformation or contour separation to decompose the connected regions of the adhesive pin marks into several independent pin mark regions.

[0090] S3-3. Based on the Euclidean distance from a pixel in the circular pad to the geometric center of the circular pad, divide the circular pad into three concentric regions: the core region, the middle region, and the edge region; calculate the distance from a pixel in the circular pad to the geometric center of the circular pad (C). x C y The Euclidean distance d between the pads is defined as follows: when d < R*(1-α), it is determined to be the Core Zone, where R is the radius of the circular pad and α is the core zone scale factor, which can be 0.25; when R*(1-α) ≤ d < R*(1-β), it is determined to be the Middle Zone, where R is the radius of the circular pad and β is the middle zone scale factor, which can be 0.75; when d ≥ R*(1-β), it is determined to be the Edge Zone.

[0091] S3-4. Using the pinhole regions in the core area as reference samples, calculate the weighted standardized distance between the pinhole regions in the central and edge areas and the core area reference sample distribution. When a pinhole region is located in the central area, compare the weighted standardized distance with a lenient threshold. If the weighted standardized distance is less than or equal to the lenient threshold, the pinhole region is determined to be a candidate pinhole target; if the weighted standardized distance is greater than the lenient threshold, the pinhole region is determined to be a non-pinhole target. When a pinhole region is located in the edge area, compare the weighted standardized distance with a strict threshold. If the weighted standardized distance is less than or equal to the strict threshold, the pinhole region is determined to be a candidate pinhole target; if the weighted standardized distance is greater than the strict threshold, the pinhole region is determined to be a non-pinhole target. The strict threshold can be 1.0, and the lenient threshold can be 1.5.

[0092] The core area has uniform illumination and minimal interference, resulting in high reliability of detected pinhole regions. Therefore, pinhole regions in the core area are used as reference samples to construct a statistical distribution model. Assuming N high-confidence pinhole regions are detected in the core area, multi-dimensional feature vectors are extracted from each reference sample to form a feature vector set Fcore = {f1, f2, ..., f...}. NThe feature distribution space of each reference sample includes 13 dimensions such as mean brightness, contrast, roundness, solidity, and aspect ratio. The mean vector μ and standard deviation vector σ of each reference sample are calculated to construct its feature distribution space. For any pinhole region x in the middle and edge regions, the feature vector of that region x is extracted. The weighted standardized distance D between the feature vector of any pinhole region x and the feature distribution space of each reference sample is calculated. stat The calculation process is as follows:

[0093]

[0094] In the formula, j represents the index of the feature dimension, N represents the total number of feature dimensions, and μ j Let σ be the feature mean vector of the j-th reference sample. j Let w be the feature standard deviation vector of the j-th reference sample. j Let x be the weight coefficient of the j-th reference sample, and ε be a minimal constant to prevent the denominator from being zero; j It is represented by the value of the needle mark region to be verified in the middle and edge regions on the j-th feature vector.

[0095] In step S3, a non-maximum suppression strategy is first implemented, followed by texture clustering based on spatial collinearity to filter out non-needle mark targets. To eliminate redundant needle mark targets caused by overlapping windows, in step S3, the non-maximum suppression strategy sorts several candidate needle mark targets in descending order of their confidence scores, selecting the candidate needle mark targets with the highest confidence scores as retained candidates. This ensures that the targets with the most significant features and the highest confidence in the overlapping region are preferentially retained. The calculation process is shown below:

[0096]

[0097] Traverse the sorted list of needle mark regions and calculate the Euclidean distance D between the center points of the remaining candidate needle mark targets and the candidate needle mark targets with the highest confidence score. center The Euclidean distance between the center point and a dynamic overlap threshold is compared to adapt to different sizes of needle marks. When the Euclidean distance between the center point and the dynamic overlap threshold is lower than the threshold, the candidate needle mark with the highest confidence score among the remaining candidates is eliminated. When the Euclidean distance between the center point and the dynamic overlap threshold is higher than or equal to the threshold, the candidate needle mark with the highest confidence score among the remaining candidates is retained as a retained candidate needle mark. Then, the Euclidean distance D between the center points of the candidate needle mark with the highest confidence score among the remaining candidates and the retained candidate needle mark is calculated.center The calculated Euclidean distance between the center points is compared with a dynamic overlap threshold until the last remaining candidate needle mark target is identified. The dynamic overlap threshold T... overlap The calculation process is as follows:

[0098]

[0099] In the formula, C i C represents the candidate needle mark target with the highest confidence score among the remaining candidate needle mark targets. kept Let λ represent the candidate pin mark targets to be retained, and let λ represent the overlap coefficient, which can be 0.7; Area() represents the area occupied by the candidate pin mark targets; the non-maximum suppression strategy effectively solves the problem of false deletion when pin marks of different sizes coexist.

[0100] Texture clustering based on spatial collinearity is a key step in eliminating persistent background noise. Chip pads often exhibit grain textures formed by aluminum sputtering or shallow scratches accumulated from repeated probe contacts. While these textures are locally very similar to probe marks, they exhibit significant collinearity in their macroscopic spatial distribution, displaying a regular horizontal or vertical arrangement along the grain growth direction or probe movement trajectory. In contrast, actual probe indentations typically exhibit a random or discrete distribution. In step S3, the texture clustering operation based on spatial collinearity constructs a horizontal clusterer and a vertical clusterer. The horizontal clusterer groups candidate probe marks horizontally based on their center ordinate, calculates the difference between the center ordinates of any two candidate probe marks, and then modifies this difference against a tolerance threshold T. tol Comparison, when the difference is less than or equal to the tolerance threshold T tol When, that is, |y i -y j |≤T tol Then, any two candidate needle mark targets are grouped into the same horizontal group. The vertical clusterer groups the candidate needle mark targets vertically based on the center x-coordinate of the candidate needle mark targets; it calculates the difference between the center x-coordinates of any two candidate needle mark targets and compares the difference with the tolerance threshold T. tol Comparison, when the difference is less than or equal to the tolerance threshold T tol When, that is, |x i -x j |≤T tol If this is true, then any two candidate needle mark targets will be grouped into the same vertical group. The tolerance threshold T... tol It can be 5 pixels; any group in any of the several horizontal groups and several vertical groups is a non-empty group as long as there is a candidate needle mark target.

[0101] The system sets tightness and quantity conditions to filter process texture noise. Under the tightness condition, for any non-empty group, the spacing S between adjacent needle mark targets is calculated after sorting according to the principal axis coordinates. gap When the maximum distance S between adjacent needle marks gap Greater than the tight condition threshold T space If the maximum distance S between adjacent needle mark targets is reached, then all needle mark targets in the non-empty group are determined to be non-needle mark targets, and the non-empty group is removed; gap Less than or equal to the tight condition threshold T space When the condition is met, all needle mark targets in the non-empty group are determined as the final needle mark targets; the tight condition threshold T space The value can be 30 pixels. In the quantity condition, for any non-empty group, if the number of needle mark targets in the non-empty group is greater than or equal to 2, then all needle mark targets in that non-empty group are determined to be non-needle mark targets, and the non-empty group is discarded; if the number of needle mark targets in a non-empty group is less than 2, then the needle mark targets in that non-empty group are determined to be the final needle mark targets. Texture clustering based on spatial collinearity successfully distinguishes textures resembling needle marks from real needle marks from the perspective of global spatial distribution, greatly reducing the false alarm rate.

[0102] In step S4, several final pin mark targets are all located in the local slice coordinate system. If the detection is performed in an overlapping slice, the local coordinates need to be calculated and restored to the original image coordinates by adding the slice offset. The calculated original image coordinates are the pixel coordinates of the final pin mark targets. The pixel coordinates of several final pin mark targets are converted into normalized center coordinates through coordinate inverse mapping and format normalization. According to the normalized center coordinates corresponding to several final pin mark targets, a marker box is drawn in the pad image data to mark the position of several final pin mark targets.

[0103]

[0104]

[0105] In the formula, (x norm y norm ) represents the normalized center coordinates, indicating the relative position of the final pin mark target center in the pad image data; w norm h represents the width of the marker box for the final needle mark target. norm W represents the height of the final marker box of the needle mark target. img H represents the width of the pad image data. imgThe height of the pad image data is represented by x1 and y1, respectively, and the lower right pixel coordinates of the final pin mark target bounding box are represented by x2 and y2, respectively. Based on the normalized center coordinates of the final pin mark target and the width and height of the final pin mark target's bounding box, a bounding box is drawn on the pad image data. The pad image data with the pin mark bounding box is output, and the detection results are written to an associated text file, completing the entire automated pin mark detection process.

[0106] The above description is merely a preferred embodiment of the present invention. Those skilled in the art can make several modifications and optimizations based on the above disclosure without departing from the basic principles described above. These modifications and optimizations should be considered within the scope of protection as understood by the present invention.

Claims

1. A method for detecting solder pad pin marks based on shape adaptation and multi-dimensional feature fusion, characterized in that: Includes the following steps: S1. Obtain pad image data and construct an adaptive classifier of geometric moments. Determine whether the pad type is rectangular, square, or circular based on the aspect ratio and roundness factor of the pad in the pad image data. S2. Based on the pad type, the pads in the pad image data are divided into edge partitions and multi-scale adaptive judgments. When the pad type is long strip, the partition detection strategy of overlapping sliding window is adopted to equalize the local illumination of the long strip pad. The pin mark detection is performed by the constructed three-channel candidate extraction model based on local statistical characteristics to extract several initial candidate regions. A quality scoring system based on multidimensional feature fusion is constructed to calculate the confidence score of each initial candidate region. Based on the confidence score of each initial candidate region, edge distance partitioning, and spatial clustering analysis, non-needle mark targets are filtered and removed, and several candidate needle mark targets are selected. When the pad type is square, a combination of local adaptive threshold and gradient direction verification is used to detect pin marks on square pads and select several candidate pin mark targets; when the pad type is circular, a detection strategy based on region growth and concentric partition statistical verification is used to detect pin marks on circular pads and select several candidate pin mark targets. S3. Perform non-maximum suppression strategy and texture clustering operation based on spatial collinearity on several candidate needle mark targets to remove redundant needle mark targets and obtain several final needle mark targets. S4. Convert the pixel coordinates of several final pin mark targets into normalized center coordinates through coordinate inverse mapping and format normalization. Draw a marker box in the pad image data according to the normalized center coordinates of several final pin mark targets to mark the position of several final pin mark targets.

2. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 1, characterized in that: In step S1, the pad image data is input into an adaptive classifier of geometric moments to determine the pad type, including the following steps: S1-1. Calculate the aspect ratio of the pads in the pad image data, and compare the aspect ratio of the pads with the set aspect ratio threshold. When the aspect ratio of the pads is greater than or equal to the aspect ratio threshold, the pad type is determined to be a long strip pad. When the aspect ratio of the pads is less than the aspect ratio threshold, dark corner pixel detection is performed in the four corner pixel detection. S1-2. Compare the detected dark corner pixel value with the dark corner threshold; when the dark corner pixel value is less than the dark corner threshold, the pad type is determined to be a circular pad; when the dark corner pixel value is greater than or equal to the dark corner threshold, bright corner pixel detection is performed. S1-3. Compare the detected bright corner pixel value with the bright corner threshold; when the bright corner pixel value is greater than or equal to the bright corner threshold, the pad type is determined to be a square pad; when the bright corner pixel value is less than the bright corner threshold, the outline roundness of the pad in the pad image data is calculated. S1-4. Compare the calculated outline roundness with the set roundness threshold. If the outline roundness is greater than or equal to the roundness threshold, the pad type is determined to be a circular pad; if the outline roundness is less than the roundness threshold, the pad type is determined to be a square pad.

3. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 1, characterized in that: In step S2, when the pad type is elongated, a partition detection strategy with overlapping sliding windows is adopted to equalize the local illumination of the elongated pad, dividing the elongated pad into several sub-regions along the long side, with a certain proportion of overlap between adjacent sub-regions, and applying bilateral filtering to each of the sub-regions; the high contrast channel, the darkest pixel channel and the mixed channel in the parallel three-channel candidate extraction model based on local statistical characteristics are used to sequentially filter the several sub-regions after bilateral filtering to obtain several initial candidate regions.

4. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 3, characterized in that: In the multi-dimensional feature fusion quality scoring system, four physical dimensions are comprehensively considered: darkness score, roundness score, contrast score, and morphological score. The confidence score of each initial candidate region is calculated; the calculation formula is shown below: In the formula, S dark Represented as a darkness rating, S circ Represented as a roundness score, S cont Represented as a contrast rating, S morph The morphological score is represented by W1, W2, W3, and W4, which are the weighting coefficients for the octane score, roundness score, contrast score, and morphological score, respectively.

5. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 4, characterized in that: The edge distance partitioning calculates the minimum Euclidean distance from the center of each initial candidate region to the boundary of the elongated pad. If the minimum Euclidean distance of any initial candidate region is less than a set distance threshold, the initial candidate region is marked as an edge region. Initial candidate regions marked as edge regions are judged using a high confidence threshold. If the confidence score of an initial candidate region marked as an edge region is greater than the high confidence threshold, the initial candidate region marked as an edge region is determined to be a candidate pin mark target. If the confidence score of an initial candidate region marked as an edge region is less than or equal to the high confidence threshold, the initial candidate region marked as an edge region is determined to be a non-pin mark target. If the minimum Euclidean distance of any initial candidate region is greater than or equal to the set distance threshold, the initial candidate region is marked as an internal region. Initial candidate regions marked as internal regions are judged using a low confidence threshold. If the confidence score of an initial candidate region marked as an internal region is greater than the low confidence threshold, the initial candidate region marked as an internal region is judged as a candidate pin mark target. If the confidence score of an initial candidate region marked as an internal region is less than or equal to the low confidence threshold, the initial candidate region marked as an internal region is judged as a non-pin mark target.

6. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 1, characterized in that: When the pad type is square, a combination of local adaptive thresholding and gradient direction verification is used to detect pin marks on the square pads, and several candidate pin mark targets are selected; including the following steps: S2-1. Calculate the pixel statistics of the internal region of the square pad to obtain the mean and standard deviation; calculate the dynamic binarization threshold using the mean and standard deviation; segment the Gaussian blurred pad image data in parallel based on the dynamic binarization threshold and Otsu's maximum inter-class variance method to obtain several dark candidate regions. S2-2. Calculate the Sobel gradient field of dark candidate regions near the boundary of the square pad. When the gradient magnitude Mag of any dark candidate region exceeds the global edge intensity P... 60 When the percentile and the gradient direction θ shows a high degree of consistency, the dark candidate region is determined to be an edge gradient artifact and is removed to obtain the retained dark candidate region. S2-3. Set multi-dimensional feature constraints, including photometric constraints, morphological constraints and positional constraints, for the retained dark candidate regions. Perform geometric and photometric joint screening on the retained dark candidate regions to select several candidate needle mark targets.

7. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 1, characterized in that: When the pad type is circular, a detection strategy based on region growth and concentric partitioning statistical verification is used to perform pin mark detection on the circular pads, and several candidate pin mark targets are screened out; including the following steps: S3-1. The pure detection area is extracted using the edge reverse region growing method to obtain several non-connected needle mark areas; S3-2, The distance transformation-based segmentation algorithm decomposes the connected regions of the adhesive needle marks into several independent needle mark regions; S3-3. Based on the Euclidean distance from the pixel in the circular pad to the geometric center of the circular pad, divide the circular pad into three concentric regions: the core region, the middle region, and the edge region. S3-4. Using the needle mark regions in the core area as reference samples, calculate the weighted standardized distance between the needle mark regions in the central and edge areas and the reference sample distribution in the core area. When the needle mark region is located in the central area, compare the weighted standardized distance with a lenient threshold. If the weighted standardized distance is less than or equal to the lenient threshold, the needle mark region is determined to be a candidate needle mark target. When the needle mark region is located in the edge area, compare the weighted standardized distance with a strict threshold. If the weighted standardized distance is less than or equal to the strict threshold, the needle mark region is determined to be a candidate needle mark target.

8. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 1, characterized in that: In step S3, the non-maximum suppression strategy sorts several candidate pinhole targets in descending order of their confidence scores, selects the pinhole targets with the highest confidence scores as retained candidate pinhole targets, calculates the Euclidean distance between the center point of the remaining candidate pinhole targets with the highest confidence scores and the retained candidate pinhole targets, compares the center point Euclidean distance with a dynamic overlap threshold, and removes the pinhole target with the highest confidence score from the remaining candidate pinhole targets if the center point Euclidean distance is lower than the dynamic overlap threshold.

9. The pad pin mark detection method based on shape adaptation and multi-dimensional feature fusion according to claim 1, characterized in that: In step S3, the texture clustering operation based on spatial collinearity constructs a horizontal clusterer and a vertical clusterer. The horizontal clusterer groups candidate needle marks horizontally based on the center ordinate of the candidate needle mark targets, and the vertical clusterer groups candidate needle marks vertically based on the center abscissa of the candidate needle mark targets. For any non-empty group, the distance between adjacent needle mark targets is calculated after sorting according to the principal axis coordinates. When the maximum distance between adjacent needle mark targets is greater than the closeness condition threshold, all needle mark targets in the non-empty group are determined to be non-needle mark targets, and the non-empty group is removed. For any non-empty group, if the number of needle mark targets in the non-empty group is greater than or equal to 2, all needle mark targets in the non-empty group are determined to be non-needle mark targets, and the non-empty group is removed.

Images