A chip defect detection method based on infrared thermal image and visible light image registration fusion

Abstract

The application discloses an infrared thermal image and visible light image registration and fusion method, relates to the technical field of semiconductor detection and failure analysis, and comprises the following steps: acquiring phase-locked infrared thermal image data of a chip in a working state based on a surface array phase-locked infrared thermal imager; and performing coordinate system alignment on the infrared thermal image data and visible light image data based on a feature point matching image registration algorithm to obtain spatially aligned infrared thermal distribution maps and visible light surface maps. The infrared thermal image and visible light image registration and fusion method can significantly improve the precision of chip defect positioning, the quantitative level of risk assessment and the reliability of detection results, reduces the probability of misjudgment and missed judgment, and has good engineering application value.

Description

Technical Field

[0001] This invention relates to the field of semiconductor testing and failure analysis technology, specifically to a chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light images. Background Technology

[0002] In the field of semiconductor manufacturing and failure analysis, with the continuous improvement of chip integration, increasingly complex packaging structures, and continuously increasing power density, even minute thermal anomalies, structural defects, and material abnormalities generated during device operation can lead to functional failures or even system-level malfunctions. Therefore, how to achieve high-precision defect location and risk assessment while the chip is in operation has become a core issue of concern in the industry.

[0003] In existing technologies, infrared thermography and optical microscopy are widely used in chip inspection. Infrared thermography can reflect the temperature distribution of a chip under power-on conditions, and is particularly suitable for detecting localized hot spots caused by leakage, short circuits, and electromigration. However, due to the existence of thermal diffusion effects and the relatively limited spatial resolution of infrared imaging, relying solely on thermograms often makes it difficult to accurately pinpoint the location of specific circuit structures or microscopic defects. Furthermore, some thermography systems only analyze temperature amplitude, failing to fully utilize the phase information of the thermal response, resulting in limitations in deep structure interpretation and the identification of subtle anomalies. On the other hand, visible light microscopy has high spatial resolution and can clearly present the surface morphology, crack patterns, foreign particles, and material damage of chips. However, this type of technology mainly reflects surface geometry and cannot directly determine whether it constitutes a functional thermal failure risk. Especially in high-density packaging, multi-layer metal wiring, and three-dimensional chip structures, it is difficult to identify potential electrical anomaly areas using only optical images. In actual inspection processes, even when infrared thermal images and visible light images are acquired simultaneously, the fundamental differences between the two types of images in terms of imaging mechanisms, spatial resolution, scale, and information representation often make accurate registration and effective fusion difficult. Insufficient registration accuracy can easily lead to inaccurate correspondence between thermal anomaly regions and structural locations, thus affecting the reliability of subsequent defect interpretation. Furthermore, existing technologies typically analyze thermal anomalies or structural anomalies independently, lacking a mechanism for comprehensive modeling and quantitative assessment of multi-dimensional data such as temperature gradient information, structural crack characteristics, and foreign object embedding. Especially in defect risk assessment, reliance on manual experience or simple threshold judgments is often insufficient to establish a unified comprehensive risk index and classification standard, easily leading to misjudgments or omissions. Summary of the Invention

[0004] The purpose of this invention is to provide a chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light images, thereby solving the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light images, comprising: S1, acquiring phase-locked infrared thermal image data of the chip in its working state using a phase-locked infrared thermal imager, and aligning the infrared thermal image data and visible light image data in coordinate systems using an image registration algorithm based on feature point matching to obtain a spatially aligned infrared thermal distribution map and a visible light surface map; S2, extracting temperature anomaly regions from the spatially aligned infrared thermal distribution map based on the phase map and amplitude map obtained by phase-locked imaging, identifying local hot spots through phase-locked signal phase difference analysis, and combining the color changes and micro-indicators in the visible light surface map. S3. Based on the structural morphology information, determine the spatial range of potential thermal anomaly sections; S4. According to the corresponding position of the potential thermal anomaly sections in the visible light surface image, use a micro-light microscopy system to perform high-resolution scanning imaging, extract surface detail features, and identify whether there are crack structure anomalies or foreign object particle size anomalies; S5. When crack structure anomalies or foreign object particle anomalies are detected, establish a multimodal risk assessment model, calculate the comprehensive risk index of the anomaly area, and obtain the preliminary defect classification results; S6. Based on the preliminary defect classification results, combined with foreign object distribution density, foreign object-substrate bonding characteristic parameters, and preset risk threshold standards, determine the specific spatial coordinates and risk level classification of high-risk defect areas.

[0006] Preferably, step S1 includes acquiring a first thermal image matrix and a first surface matrix; extracting corner features from the first thermal image matrix and the first surface matrix; if the corner feature response value is greater than a preset threshold, determining a matching point pair based on the corner features; calculating an affine transformation matrix based on the matching point pair; processing the first thermal image matrix using the affine transformation matrix to obtain an aligned thermal image matrix; extracting an edge pixel set from the aligned thermal image matrix and the first surface matrix; performing spatial correction based on the edge pixel set and texture integrity constraints to obtain a spatially aligned infrared thermal distribution map and a visible light surface map.

[0007] Preferably, step S2 includes performing threshold segmentation on the infrared thermal distribution map to obtain temperature anomaly regions; extracting phase-locked signals for the temperature anomaly regions and calculating the phase difference of the phase-locked signals to determine the location of local hot spots; extracting color change features and microstructure morphology features from the visible light surface map containing the location of local hot spots; if the gradient value of the color change features and the depth value of the microstructure morphology features are greater than a preset threshold, then performing boundary fusion in combination with the location of local hot spots to determine the spatial range of potential thermal anomaly segments.

[0008] Preferably, step S3 includes acquiring the corresponding position coordinates and scanning to obtain a high-resolution low-light image; separating the microcrack contour and foreign object particle contour based on the high-resolution low-light image; extracting microcrack width and crack hierarchy structure data for the microcrack contour, and calculating foreign object particle size data for the foreign object particle contour; if the microcrack width exceeds a preset width threshold or the crack hierarchy structure data is abnormal, it is determined that there is a crack structure abnormality; if the foreign object particle size data is greater than a preset size threshold, it is determined that there is a foreign object particle size abnormality, thereby realizing the identification of whether there is a crack structure abnormality or a foreign object particle size abnormality.

[0009] Preferably, step S4 includes extracting the number of crack branches and the foreign object embedding depth in the image to obtain a first feature set; locating abnormal regions based on the first feature set and extracting the phase gradient of the abnormal regions to generate a second feature set; fusing the stress distribution features extracted from the first feature set and the second feature set to construct a multimodal data matrix; establishing a model through the multimodal data matrix to calculate a comprehensive risk index; and if the comprehensive risk index is greater than a preset threshold, obtaining a preliminary defect classification result based on the comprehensive risk index.

[0010] Preferably, step S5 includes extracting the foreign object distribution density value from the preliminary defect classification data, processing the foreign object distribution density value to obtain a third feature set; extracting the foreign object adhesion degree value to the substrate based on the third feature set, processing the foreign object adhesion degree value to the substrate to obtain a fourth feature set; if the fourth feature set is greater than the preset risk threshold, then a high-risk defect area is determined; extracting the specific spatial coordinates of the high-risk defect area and classifying the risk level, thus determining the specific spatial coordinates and risk level classification of the high-risk defect area.

[0011] Preferably, the method further includes S6: for the identified high-risk defect area, local enhancement processing is performed on the infrared thermal distribution map and the visible light surface map, respectively. The infrared image is magnified for detail using phase-locked amplitude enhancement and phase contrast enhancement, and the visible light image is enhanced for feature enhancement using local contrast enhancement and edge enhancement algorithms to generate a fused high-resolution defect image. Specifically, this includes obtaining the initial infrared thermal distribution map and the initial visible light surface map of the high-risk defect area; obtaining a magnified infrared image for detail using phase-locked amplitude enhancement and phase contrast enhancement on the initial infrared thermal distribution map; if the contrast of the magnified infrared image for detail is lower than a preset threshold, then histogram equalization is used to obtain a secondary magnified infrared image for detail.

[0012] Preferably, step S6 further includes obtaining a feature-enhanced visible light image by applying local contrast and edge enhancement to the initial visible light surface image; if the edge sharpness of the feature-enhanced visible light image is lower than a preset threshold, then a second feature-enhanced visible light image is obtained by applying the Laplacian operator; and the second feature-enhanced visible light image is fused with the second detail magnified infrared image to generate a fused high-resolution defect image.

[0013] Preferably, the method further includes S7: extracting multimodal feature information from the fused high-resolution defect image, combining the lock-in thermal response time constant, crack morphology feature parameters, and optical reflection characteristic data related to the foreign material, performing defect pattern matching judgment, and outputting the final defect confirmation result and classification label. Specifically, this includes acquiring a high-resolution defect image, extracting multimodal feature information from the high-resolution defect image to obtain an initial multimodal feature set, and extracting the lock-in thermal response time constant and crack morphology feature values ​​based on the initial multimodal feature set to obtain a multidimensional physical feature vector.

[0014] Preferably, step S7 further includes extracting optical reflection characteristic data based on multidimensional physical feature vectors to determine a comprehensive defect feature matrix; using the comprehensive defect feature matrix for matching judgment; if the matching degree of the comprehensive defect feature matrix is ​​greater than a preset threshold, then outputting the final defect confirmation result and classification label based on the matching degree.

[0015] As can be seen from the above technical solution, the present invention has the following beneficial effects: This chip defect detection method, based on the registration and fusion of infrared thermal imaging and visible light images, simultaneously acquires lock-in infrared thermal image data and visible light or near-infrared surface image data while the chip is in operation. It then performs high-precision spatial registration and fusion analysis on the two types of images. This effectively solves the problems in existing technologies, such as the difficulty in accurately locating thermal anomalies to specific structural positions, the difficulty in judging electrical risks from optical structural anomalies, and the inability to uniformly assess multimodal information. By utilizing the amplitude and phase map information from lock-in infrared imaging, it improves the ability to identify weak thermal anomalies and deep structural anomalies, enhancing the sensitivity and reliability of anomaly detection. Furthermore, by correlating thermal response characteristics with structural features such as crack morphology and foreign object particle parameters, a multimodal risk assessment mechanism is constructed to calculate the comprehensive risk index and classify the risk level of anomaly areas, thereby reducing reliance on manual interpretation and improving the consistency and accuracy of defect judgment. Simultaneously, by locally enhancing and fusing high-risk areas with high-resolution images, the method further improves the presentation of defect details, facilitating subsequent analysis and confirmation. In summary, this invention can significantly improve the accuracy of chip defect localization, the quantification level of risk assessment, and the reliability of detection results, while reducing the probability of false positives and false negatives, and has good engineering application value. Attached Figure Description

[0016] Figure 1 This is a flowchart of the chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light images of the present invention; Figure 2 This is a structural block diagram of the local terminal of an exemplary electronic device of the present invention; Figure 3 This is a structural block diagram of the network terminal of an exemplary electronic device of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, the present invention provides a technical solution: a chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light image, comprising: S1. Based on the phase-locked infrared thermal imager, acquire phase-locked infrared thermal image data in the chip's working state. Use an image registration algorithm based on feature point matching to align the infrared thermal image data with the visible light image data in the coordinate system, and obtain the spatially aligned infrared thermal distribution map and visible light surface map. S2. Based on the phase map and amplitude map obtained by phase-locked array imaging, temperature anomaly regions are extracted from the spatially aligned infrared thermal distribution map. Local hot spot locations are identified by phase difference analysis of the phase-locked signal. Combined with color changes and microstructure morphology information in the visible light surface map, the spatial range of potential thermal anomaly sections is determined. S3. Based on the corresponding position of the potential thermal anomaly section in the visible light surface image, use a micro-light microscopy system to perform high-resolution scanning imaging, extract surface detail features, and identify whether there are abnormal crack structures or abnormal foreign particle sizes. S4. When abnormal crack structure or foreign particle abnormality is detected, establish a multimodal risk assessment model, calculate the comprehensive risk index of the abnormal area, and obtain the preliminary defect classification results. S5. Based on the preliminary defect classification results, combined with the foreign matter distribution density, the characteristic parameters of the foreign matter and the substrate bonding degree, and the preset risk threshold standard, determine the specific spatial coordinates and risk level classification of the high-risk defect area. S6. For the identified high-risk defect areas, local enhancement processing is performed on the infrared thermal distribution map and the visible light surface map, respectively. The infrared image is magnified by phase-locked amplitude enhancement and phase contrast enhancement, and the visible light image is enhanced by local contrast enhancement and edge enhancement algorithms to generate a fused high-resolution defect image. S7. Extract multimodal feature information from the fused high-resolution defect image, combine it with the phase-locked thermal response time constant, crack morphology feature parameters and optical reflection characteristic data related to foreign material, perform defect pattern matching judgment, and output the final defect confirmation result and classification label.

[0019] In this embodiment, the chip under test is in a powered-on operating state, a periodically excited state, or a thermally loaded state, causing internal and surface defects to generate thermal responses for detection. A phase-locked infrared (PLI) thermal imager performs PLI imagery on the chip, obtaining infrared thermal image data of the chip in its operating state. This infrared thermal image data includes a PLI amplitude map and a PLI phase map. The PLI amplitude map reflects the thermal response intensity of different regions of the chip, while the PLI phase map reflects the phase lag of the thermal response relative to the excitation signal. Defective regions alter the local heat conduction path, heat diffusion rate, and thermal resistance distribution, causing local amplitude increases or decreases in the PLI amplitude map and phase abrupt changes, phase lag anomalies, or phase gradient anomalies in the PLI phase map.

[0020] Furthermore, the system aligns the infrared thermal image data with the visible light image data in the same coordinate system, placing them under the same spatial coordinate reference. Specifically, the system extracts chip edges, pad outlines, metal trace intersections, positioning marks, surface texture corners, and stable structural feature points from the infrared thermal distribution map and the visible light surface map, and obtains the corresponding point relationship between the two types of images through feature point matching. Based on the corresponding point relationship, the system calculates the coordinate transformation matrix and performs rotation, translation, scaling, perspective correction, and local deformation correction on the infrared thermal image data, mapping the thermal anomaly positions in the infrared thermal distribution map to the actual surface structure positions in the visible light surface map, thus obtaining the spatially aligned infrared thermal distribution map and visible light surface map.

[0021] Specifically, the system extracts temperature anomaly regions based on spatially aligned infrared thermal distribution maps, phase-locked loop (PLL) phase maps, and PLL amplitude maps. For regions in the amplitude map where the thermal response intensity is higher than the surrounding background area, the system identifies them as candidate regions for thermal anomalies. For regions in the phase map exhibiting abrupt phase differences, abnormal phase delays, or local phase gradient changes, the system further determines whether they are local hot spots or points of abnormal thermal diffusion. The system performs joint analysis of amplitude and phase anomalies, eliminating false detections caused by relying solely on temperature intensity judgments and improving the accuracy of identifying hidden defects, microcracks, and defects with abnormal local thermal resistance.

[0022] The system maps identified localized hotspot locations onto a visible light surface image and combines this information with color variations, reflection differences, microstructural irregularities, edge discontinuities, and texture anomalies from the visible light surface image to determine the spatial extent of potential thermal anomaly zones. When a hotspot location corresponds to a localized discoloration, darkening, brightening, contamination, surface protrusion, depression, or texture breakage in the visible light surface image, the system identifies that location as a potential thermal anomaly zone. This processing method correlates infrared thermal anomaly information with visible surface structure information, providing the thermal anomaly localization results with clear spatial boundaries and surface structure evidence.

[0023] The system utilizes a low-light microscopy system to perform high-resolution scanning imaging based on the corresponding location of potential thermal anomaly sections in the visible light surface image. The low-light microscopy system performs point-by-point scanning, line scanning, grid scanning, or local area scanning on the potential thermal anomaly sections to obtain high-resolution surface detail images of the region. The system then performs image segmentation, grayscale gradient analysis, edge detection, contour extraction, and size measurement on the scanned surface detail images to identify crack structure anomalies and foreign object particle size anomalies. Crack structure anomalies manifest as elongated dark streaks, continuous linear edges, discontinuous linear edges, surface texture breaks, pointed extension structures, or bifurcated structures. Foreign object particle size anomalies manifest as granular targets that differ from the substrate in color, reflectivity, contour height, or boundary morphology, and whose equivalent diameter, area, or height exceeds a preset allowable range.

[0024] Furthermore, when abnormal crack structures or foreign object particles are detected, the system establishes a multimodal risk assessment model. This model uses infrared thermal response characteristics, visible light surface characteristics, and microscopic details as input parameters. Infrared thermal response characteristics include local temperature rise, peak value of phase-locked loop amplitude, phase difference, thermal response time constant, and thermal anomaly area; crack morphology characteristics include crack length, crack width, crack orientation, number of crack branches, crack tip curvature, and distance between the crack and the chip functional structure; foreign object particle characteristics include foreign object particle size, number, distribution density, boundary clarity, adhesion to the substrate, and differences in optical reflection properties. After normalizing the above characteristics, the system calculates a comprehensive risk index for the abnormal area through weighted calculation, rule-based discrimination, or classification models, and obtains preliminary defect classification results based on this comprehensive risk index.

[0025] In the above embodiments, the system further combines foreign matter distribution density, foreign matter-substrate bonding characteristic parameters, and preset risk threshold standards to classify the risk level of the preliminary defect classification results. When the comprehensive risk index of an abnormal area exceeds the high-risk threshold, or when the foreign matter distribution density, crack morphology severity, or phase anomaly degree reaches the preset risk conditions, the system determines the area as a high-risk defect area and outputs its specific spatial coordinates. These specific spatial coordinates are represented using an infrared thermal distribution map coordinate system, a visible light surface map coordinate system, or a chip physical coordinate system, and are used for subsequent defect re-inspection, failure analysis, and process traceability.

[0026] In addition, for identified high-risk defect areas, the system performs local enhancement processing on both the infrared thermal distribution map and the visible light surface map. For the infrared thermal distribution map, the system uses phase-locked amplitude enhancement to highlight differences in thermal response intensity and phase contrast enhancement to highlight local phase abrupt change boundaries, thus magnifying the details of thermal anomalies. For the visible light surface map, the system uses local contrast enhancement to enhance the brightness and color differences between the defect area and the background substrate, and employs an edge enhancement algorithm to highlight crack boundaries, foreign particle outlines, and microstructure unevenness boundaries. After enhancement processing, the system fuses the infrared thermal response information with the visible light surface structure information to generate a fused high-resolution defect image.

[0027] As a preferred embodiment, the system extracts multimodal feature information from the fused high-resolution defect image and combines it with the lock-in thermal response time constant, crack morphology feature parameters, and optical reflection characteristic data related to the foreign material to perform defect pattern matching and judgment. Specifically, the system compares the extracted feature information with feature templates in a preset defect pattern library, or identifies the defect type through a trained classification model. When a region simultaneously exhibits lock-in thermal anomalies, crack morphology features, and phase difference anomalies, the system classifies it as a crack-induced thermal anomaly; when a region exhibits granular surface targets, optical reflection characteristic anomalies, and local thermal diffusion anomalies, the system classifies it as a foreign particle-induced thermal anomaly; when the same region simultaneously contains crack structures and foreign particles, accompanied by local overheating, the system classifies it as a composite defect. Finally, the system outputs the defect confirmation result and classification label, thereby completing the registration and fusion detection of infrared thermal images and visible light images.

[0028] S1 includes acquiring a first thermal image matrix and a first surface matrix; extracting corner features from the first thermal image matrix and the first surface matrix; if the corner feature response value is greater than a preset threshold, determining a matching point pair based on the corner features; calculating an affine transformation matrix based on the matching point pair; processing the first thermal image matrix using the affine transformation matrix to obtain an aligned thermal image matrix; extracting an edge pixel set from the aligned thermal image matrix and the first surface matrix; performing spatial correction based on the edge pixel set and texture integrity constraints to obtain a spatially aligned infrared thermal distribution map and a visible light surface map.

[0029] In this embodiment, the system acquires a first thermal image matrix and a first surface matrix. The first thermal image matrix is ​​formed from infrared thermal image data acquired by a phase-locked infrared thermal imager during chip operation. The system first performs phase-locked demodulation on multiple frames of infrared images continuously acquired by the infrared thermal imager. Specifically, for the same pixel location, the thermal response values ​​of that pixel are read sequentially according to the period of the excitation signal at multiple sampling times. These thermal response values ​​are then synchronously compared with the reference phase signal to obtain the phase-locked amplitude response at that pixel location. Subsequently, the system traverses all pixel locations to obtain the phase-locked amplitude data corresponding to the entire image. The system then normalizes the phase-locked amplitude data. During normalization, the maximum and minimum amplitude values ​​among all pixels are first identified, and then the minimum amplitude value is subtracted from the amplitude value of each pixel. The resulting difference is divided by the difference between the maximum and minimum amplitude values. After this processing, the thermal response value of each pixel is converted to a uniform numerical range, thereby forming the first thermal image matrix.

[0030] In this embodiment, the first surface matrix is ​​formed from a chip surface image acquired by a visible light imaging device. If the visible light image is a color image, the system first converts the red, green, and blue channel values ​​of each pixel into grayscale values. During the conversion, the red channel value is multiplied by 0.299, the green channel value by 0.587, and the blue channel value by 0.114, and then the three products are added together to obtain the grayscale value of that pixel location. The system traverses all pixels in the visible light image in this manner to obtain a grayscale surface image. Subsequently, the system performs the same normalization processing on the grayscale surface image as on the first thermal image matrix, that is, subtracting the minimum grayscale value in the entire image from the grayscale value of each pixel, and then dividing by the difference between the maximum and minimum grayscale values ​​to form the first surface matrix. After this processing, both the first thermal image matrix and the first surface matrix enter the subsequent registration process in the form of pixel matrices.

[0031] Furthermore, when extracting corner features from the first thermal image matrix and the first surface matrix, the system first calculates the grayscale changes around each pixel. For any pixel to be judged, the system reads the values ​​of its left, right, top, and bottom adjacent pixels. It subtracts the left pixel value from the right pixel value to obtain the horizontal change, and subtracts the top pixel value from the bottom pixel value to obtain the vertical change. The system then selects a fixed-size neighborhood window centered on the pixel to be judged, such as a 3x3 pixel window or a 5x5 pixel window, and within this neighborhood window, it statistically analyzes the horizontal change, the vertical change, and the combination of both.

[0032] The corner feature response value is calculated based on the intensity of directional changes within the neighborhood window. The system accumulates the horizontal directional change intensity of all pixels within the neighborhood window to obtain the first directional statistic; it accumulates the vertical directional change intensity of all pixels to obtain the second directional statistic; and then accumulates the product of the horizontal and vertical directional changes to obtain the directional correlation statistic. If both the first and second directional statistics have significant values, and the directional correlation statistic shows changes in more than two directions within the pixel's neighborhood, then that location is considered a corner candidate. The system obtains the corner feature response value based on these three types of statistics. A higher response value indicates a more pronounced structural transition within the pixel's neighborhood, resulting in higher localization stability.

[0033] Specifically, the system compares the corner feature response values ​​with preset thresholds. If the corner feature response value of a pixel in the first thermal image matrix is ​​greater than the preset threshold, the system retains that pixel as a candidate thermal image corner; if the corner feature response value of a pixel in the first surface matrix is ​​greater than the preset threshold, the system retains that pixel as a candidate surface corner. Pixels with response values ​​not greater than the preset threshold are excluded. To avoid repeatedly retaining multiple adjacent corners in the same area, the system performs local filtering of candidate corners: within each fixed neighborhood window, only the candidate corner with the largest corner feature response value is retained, and the remaining candidate corners are deleted. After this step, the system obtains a set of thermal image corners and a set of surface corners.

[0034] As a preferred embodiment, when determining matching point pairs, the system establishes a local feature description for each thermal image corner. Specifically, the system extracts a fixed-size neighborhood window centered on the thermal image corner, and statistically analyzes the grayscale distribution order, gradient direction distribution, gradient intensity distribution, and local brightness variation relationships of each pixel within this neighborhood window to form the feature description information for that thermal image corner. The system then establishes feature description information for each surface corner in the same manner. Because the imaging mechanisms of infrared thermal images and visible light images differ, the system does not directly rely on the brightness of a single pixel during description; instead, it uses the direction of change, intensity of change, and structural arrangement relationships within the neighborhood to reduce interference caused by brightness differences between different imaging modalities.

[0035] In addition, after completing the corner description, the system compares the feature description information of each thermal image corner with the feature description information of all surface corners one by one. During the comparison, the system first compares whether the orientation of the neighborhood structure is consistent, then compares whether the distribution positions of the strongly changing regions within the neighborhood are consistent, and then compares whether the order of local grayscale changes is consistent. For each pair of thermal image corners and surface corners, the system provides a similarity result. The higher the similarity, the higher the probability that the thermal image corner and the surface corner correspond to the same chip structure position. For each thermal image corner, the system selects the surface corner with the highest similarity as the initial matching object.

[0036] To avoid erroneous matches, the system performs bidirectional verification on the initial matching objects. The bidirectional verification process is as follows: if a thermal imaging corner point selects a surface corner point as the best matching object, and simultaneously, when that surface corner point searches for a thermal imaging corner point in reverse, it also selects that thermal imaging corner point as the best matching object, then these two corner points form an initial matching point pair. If the reverse search results are inconsistent, the matching relationship is deleted. After completing the bidirectional verification, the system obtains the preliminarily filtered matching point pairs.

[0037] Further, geometric consistency screening of the matching point pairs continues. The specific process is as follows: The system selects no fewer than three point pairs from the initially screened matching point pairs. It first estimates the initial translation, rotation angle, scaling factor, and shearing amount from the first thermal image matrix to the first surface matrix. Then, it projects all thermal image corner points onto the coordinate positions of the first surface matrix according to this initial relationship. The system compares the projected positions with the corresponding surface corner point positions and calculates the pixel distance between them. If the pixel distance is less than a preset error threshold, the matching point pair is retained; if the pixel distance reaches or exceeds the preset error threshold, the matching point pair is deleted. After geometric consistency screening, the system obtains valid matching point pairs. When the number of valid matching point pairs is no less than three, the system proceeds to the affine transformation matrix calculation step.

[0038] Furthermore, when calculating the affine transformation matrix based on the matching point pairs, the system uses the corner coordinates in the first thermal image matrix as the coordinates before transformation and the corresponding corner coordinates in the first surface matrix as the coordinates after transformation. Affine transformation is used to simultaneously describe horizontal translation, vertical translation, rotation, scaling, and shearing. During calculation, the system establishes two coordinate correspondences for each valid matching point pair: the first describes how the lateral and longitudinal coordinates of the thermal image corner points are transformed to obtain the lateral coordinates of the surface corner points; the second describes how the lateral and longitudinal coordinates of the thermal image corner points are transformed to obtain the longitudinal coordinates of the surface corner points. Since each matching point pair provides two constraints, three matching point pairs provide six constraints, satisfying the six transformation parameters required to solve the affine transformation.

[0039] When there are more than three valid matching point pairs, the system uses the principle of minimizing error to solve for the affine transformation parameters. Specifically, the system calculates a set of translation, rotation, scaling, and shearing parameters, and uses these parameters to transform all thermal image corner points to the first surface matrix coordinate system. The system calculates the pixel distance between each transformed thermal image corner point and its corresponding surface corner point, and accumulates the squares of all pixel distances. The system continuously adjusts the transformation parameters to minimize the accumulated result. The translation, rotation, scaling, and shearing parameters corresponding to the minimum accumulated result are used as the parameters in the affine transformation matrix. The resulting affine transformation matrix minimizes the overall error of the valid matching point pairs.

[0040] Furthermore, when processing the first thermal image matrix using the affine transformation matrix, the system employs a reverse mapping method to generate an aligned thermal image matrix. The specific process is as follows: the system uses the pixel coordinates of the first surface matrix as the target coordinates, reads the target coordinate positions one by one, and then calculates the original sampling position corresponding to the target coordinates in the first thermal image matrix according to the reverse relationship of the affine transformation matrix. If the original sampling position falls exactly at the center of a pixel in the first thermal image matrix, the system directly reads the thermal response value of that pixel; if the original sampling position falls between four adjacent pixels, the system assigns weights according to the distance from that position to the centers of the four adjacent pixels, with pixels closer to each other having larger weights and pixels farther away having smaller weights. The thermal response values ​​of the four pixels are then added together according to their weights to obtain the thermal response value at the target coordinates. The system repeats the above process for all target pixels in the coordinate system of the first surface matrix to obtain the aligned thermal image matrix. This aligned thermal image matrix has the same coordinate reference as the first surface matrix.

[0041] In this embodiment, after obtaining the aligned thermal image matrix, the system extracts an edge pixel set based on the aligned thermal image matrix and the first surface matrix. For the aligned thermal image matrix, the system calculates the thermal response difference between each pixel and its surrounding neighboring pixels. If the thermal response difference between a pixel and its left, right, top, or bottom neighboring pixels reaches a preset thermal image edge threshold, then the pixel is marked as a thermal image edge pixel. The system aggregates all thermal image edge pixels to form a thermal image edge pixel set. For the first surface matrix, the system calculates the grayscale difference between each pixel and its neighboring pixels in the same way. If the grayscale difference reaches a preset surface edge threshold, then the pixel is marked as a surface edge pixel. The system aggregates all surface edge pixels to form a surface edge pixel set.

[0042] After the edge pixel set is formed, the system performs connected component filtering on the edge pixels. Specifically, starting from any one edge pixel, the system searches for other edge pixels that are adjacent vertically, horizontally, or diagonally, and groups these interconnected edge pixels into the same connected component. The system calculates the number of pixels contained in each connected component. If the number of pixels in a connected component is less than a preset area threshold, the connected component is identified as isolated noise and deleted; if the number of pixels in a connected component reaches the preset area threshold, the connected component is retained. After processing, the thermal image edge pixel set retains hot spot boundaries, thermal anomaly contours, and chip thermal response boundaries; the surface edge pixel set retains chip contours, pad boundaries, metal trace boundaries, microstructure boundaries, and surface texture boundaries.

[0043] Specifically, spatial correction is performed jointly based on the edge pixel set and texture integrity constraints. The system first selects a thermal image edge point from the thermal image edge pixel set, and then searches for the nearest surface edge point in the surface edge pixel set. During the search, the system simultaneously compares the edge directions of the two edge points. The edge direction is determined by the direction of the largest change in pixel values ​​near the edge point. If the pixel distance between two edge points is less than a preset distance threshold, and the difference in edge directions between the two edge points is less than a preset direction threshold, then the system identifies these two edge points as a corresponding edge pair. The system repeats this step for all valid edge points in the thermal image edge pixel set, obtaining multiple sets of corresponding edge pairs.

[0044] Furthermore, a local correction is generated for each pair of corresponding edge points. The local correction is calculated as follows: subtract the lateral coordinates of the thermal image edge points from the lateral coordinates of the surface edge points to obtain the lateral correction; subtract the longitudinal coordinates of the thermal image edge points from the longitudinal coordinates of the surface edge points to obtain the longitudinal correction. These lateral and longitudinal corrections represent the local offset that still exists at the edge position after affine alignment. The system aggregates the local corrections for all corresponding edge point pairs to generate the spatial correction for the entire aligned thermal image matrix.

[0045] To extend discrete local corrections to the entire image, the system establishes a regular control grid on the image plane. The regular control grid consists of multiple control points, each recording one lateral correction and one vertical correction. The system determines the correction value for a control point based on corresponding edge pairs near that control point. If multiple pairs of corresponding edge pairs exist near a control point, the system calculates the average of the lateral and vertical correction values ​​for these pairs and uses this average as the correction value for that control point. If no corresponding edge pairs exist near a control point, the system interpolates and supplements the correction value based on the correction values ​​of adjacent control points, ensuring that each control point in the control grid has a defined correction value.

[0046] Furthermore, texture integrity constraints are used to prevent spatial correction from disrupting the continuous structure in the visible light surface map. The system first extracts continuous texture segments from the first surface matrix. These continuous texture segments include chip edges, pad contours, metal trace boundaries, and microstructure texture lines. During extraction, the system searches for continuously connected edge pixels along the set of surface edge pixels, grouping edge pixels that are continuous in direction, spacing, and grayscale variation into the same texture segment. For each texture segment, the system records the pixel arrangement order, the distance between adjacent pixels, and the connection direction between adjacent pixels.

[0047] The system incorporates texture integrity constraints when calculating the correction amount for the control mesh. Specifically, if a correction amount causes breaks, misalignments, intersections, or abrupt changes in direction in originally continuous texture segments, the system reduces the weight of that correction amount and replaces it with a smooth correction amount from surrounding control points. If a correction amount causes the thermal image edge to move closer to the corresponding surface texture segment while maintaining the pixel arrangement order, connection direction, and adjacent distance variation within a preset range for that texture segment, the system retains that correction amount. Through this processing, spatial correction not only aligns the thermal image edge with the surface edge but also maintains the continuity and structural integrity of the visible light surface texture.

[0048] Finally, the system applies the correction values ​​from the control grid to the aligned thermal image matrix. For each target pixel in the aligned thermal image matrix, the system first determines the specific grid cell in the control grid where the pixel is located, and then reads the lateral and longitudinal correction values ​​of the four corner control points of that grid cell. The system assigns weights based on the distance from the target pixel to the four control points, with closer control points having higher weights and farther control points having lower weights. The lateral correction values ​​of the four control points are then weighted and summed to obtain the lateral correction value of the target pixel; the longitudinal correction values ​​of the four control points are also weighted and summed to obtain the longitudinal correction value of the target pixel. The system then redetermines the sampling position of the target pixel in the aligned thermal image matrix based on these lateral and longitudinal correction values, and obtains the corrected thermal response value by weighting the values ​​of neighboring pixels.

[0049] After the above processing, the aligned thermal image matrix is ​​corrected into a spatially calibrated infrared thermal distribution map, and the first surface matrix is ​​output as a spatially calibrated visible light surface map as a spatial reference. At this time, the hot spot boundaries, thermal anomaly regions, and thermal response gradient change positions in the infrared thermal distribution map are coordinately correlated with the chip outline, pad boundaries, metal traces, microstructure textures, and surface morphology boundaries in the visible light surface map, thus obtaining the spatially aligned infrared thermal distribution map and visible light surface map.

[0050] S2 includes threshold segmentation of the infrared thermal distribution map to obtain temperature anomaly regions; extraction of phase-locked signals for the temperature anomaly regions, and phase difference calculation of the phase-locked signals to determine the location of local hot spots; extraction of color change features and microstructure morphology features from the visible light surface map containing the location of local hot spots; if the gradient value of the color change features and the depth value of the microstructure morphology features are greater than a preset threshold, boundary fusion is performed in combination with the location of local hot spots to determine the spatial range of potential thermal anomaly segments.

[0051] In this embodiment, the system performs threshold segmentation on the infrared thermal distribution map to obtain temperature anomaly regions. The infrared thermal distribution map is formed from spatially aligned phase-locked infrared data, and each pixel in the map corresponds to one thermal response value. Before threshold segmentation, the system first performs noise smoothing on the infrared thermal distribution map. Specifically, the system reads the pixel thermal response values ​​within a fixed window surrounding each pixel, excludes abnormally isolated high and low values ​​within the window, and then uses the thermal response values ​​retained within the window to correct the central pixel. After this processing, single-point noise in the infrared thermal distribution map is weakened, and the continuity of the thermal anomaly region is preserved.

[0052] Specifically, during threshold segmentation, the system reads a preset temperature anomaly threshold. This preset temperature anomaly threshold is written into the detection system before detection based on the chip type, operating current, excitation frequency, and allowable temperature rise range. The system scans the pixels in the infrared thermal distribution map line by line, comparing the thermal response value of each pixel with the preset temperature anomaly threshold. When the thermal response value of a pixel is greater than the preset temperature anomaly threshold, the pixel is marked as an anomaly pixel; when the thermal response value of a pixel is not greater than the preset temperature anomaly threshold, the pixel is marked as a background pixel. After completing the comparison of all pixels, the system generates a binary segmentation result consisting of anomaly pixels and background pixels.

[0053] Furthermore, the binary segmentation result cannot be directly output as the temperature anomaly region; the system continues to perform connected component sorting. Starting from any one anomalous pixel, the system checks if the pixels above, below, left, right, and at the four diagonal positions are also anomalous pixels. Connected anomalous pixels are grouped into the same connected component. The system counts the number of pixels, width, height, center position, and outer contour boundary of each connected component. When the number of pixels in a connected component is less than a preset area threshold, the system deletes the component; when the number of pixels in a connected component reaches the preset area threshold, the system retains it. For the retained connected components, the system fills in unmarked small holes and smooths the jagged edges on the outer contour. After the above processing, the system obtains the temperature anomaly region. This temperature anomaly region has continuous boundaries, a defined area, and a corresponding image coordinate range.

[0054] Furthermore, when extracting phase-locked signals from areas of temperature anomalies, the system does not merely read a single frame of infrared thermal distribution image; instead, it traces back the original time sequence of each pixel within that temperature anomaly area during the phase-locked acquisition process. Specifically, based on the pixel coordinates of the temperature anomaly area, the system reads the thermal response values ​​at the same coordinate location at different sampling times from a continuous infrared frame sequence acquired by the area array phase-locked infrared thermal imager. For each pixel within the temperature anomaly area, the system arranges its thermal response values ​​in chronological order of sampling time, obtaining the phase-locked thermal response sequence corresponding to that pixel. This phase-locked thermal response sequence reflects the temperature fluctuation process at that pixel location under periodic excitation.

[0055] The phase-locked thermal response sequence is then synchronized with the reference excitation signal. The reference excitation signal is provided by a modulation excitation source during chip testing, and the system records the start time, peak time, and end time of each cycle of the reference excitation signal. For each pixel within the temperature anomaly region, the system segments the pixel's phase-locked thermal response sequence according to the cycle of the reference excitation signal, ensuring that each segment of thermal response data corresponds to a complete excitation cycle. The system then averages the thermal response values ​​at the same phase position across multiple cycles to obtain the stable periodic thermal response curve for that pixel. This processing reduces the impact of random thermal noise on phase determination.

[0056] The phase difference calculation uses a reference excitation signal as a baseline. The system first determines the reference excitation signal's baseline position within one cycle, then determines the peak position of the thermal response in the stable cycle thermal response curve of that pixel. Subsequently, the system calculates the time lag between the peak position and the reference baseline position. When calculating the time lag, the system subtracts the time of the reference baseline sampling point from the time of the sampling point where the thermal response peak is located; a positive time difference indicates that the thermal response lags behind the reference excitation signal. The system then compares this time lag with the length of a complete excitation cycle and converts it into a phase difference angle according to the proportion of the time lag in the complete cycle. The system performs the above calculation for each pixel within the temperature anomaly region to obtain the phase difference distribution of that temperature anomaly region.

[0057] In this embodiment, to improve the stability of phase difference calculation, the system also performs a consistency check on the phase differences of adjacent pixels within the same connected region. Specifically, the system reads multiple phase difference values ​​from the neighborhood surrounding the target pixel and calculates the concentration of these phase difference values. If the phase difference of the target pixel changes in the same direction as the phase differences of most surrounding pixels, the phase difference value is retained. If the phase difference of the target pixel changes in the opposite direction to the phase differences of surrounding pixels, and the target pixel is not located within the thermal response peak region, the system determines the phase difference value as phase noise and replaces it with the average result of the surrounding effective phase differences. After completing this step, the phase difference distribution within the temperature anomaly region exhibits continuity.

[0058] Specifically, when determining the location of local hot spots, the system uses both thermal response intensity and phase difference results. The system checks the thermal response value of each pixel within the temperature anomaly region, identifying local peak points where the thermal response value is higher than that of other pixels in its neighborhood. For each local peak point found, the system continues to read the phase difference data of the surrounding neighborhood. When the phase difference of a local peak point shows a concentrated shift relative to the surrounding background area, or when a significant phase difference gradient forms around the local peak point, the system marks that local peak point as a candidate hot spot. If a local peak point only has a thermal response intensity peak without any phase difference anomaly, the system does not output it as a local hot spot.

[0059] As a preferred embodiment, clustering and merging are performed on candidate hotspots. During processing, the system calculates the pixel distance between any two candidate hotspots. When the pixel distance between two candidate points is less than a preset merging distance, and the two candidate points are located within the same temperature anomaly connectivity region, the system groups these two candidate points into the same hotspot cluster. For each hotspot cluster, the system compares the thermal response value and phase difference anomaly degree of each candidate point within the cluster, and identifies the candidate point with the highest thermal response value and the greatest phase difference anomaly degree as the local hotspot location of that cluster. The local hotspot location is represented by both horizontal and vertical pixel coordinates, and a correspondence is established with the temperature anomaly region in the infrared thermal distribution map.

[0060] Specifically, when extracting color change features from a visible light surface image containing local hot spots, the system first maps the local hot spots onto the visible light surface image using the previous registration results. The system then extracts a local surface region centered on the mapped coordinates. This local surface region covers the corresponding projection ranges of the local hot spots, temperature anomaly areas, and the normal surface reference area outside these projection ranges. The system first reads the average color and average brightness of the normal surface reference area as the reference color for this local region. Subsequently, the system reads the red channel value, green channel value, blue channel value, and brightness value of each pixel within the local surface region and compares them with the reference color. The comparison results are used to determine whether the pixel exhibits darkening, brightening, discoloration, or reflection anomalies.

[0061] Specifically, the gradient value of the color change feature is calculated through the color difference between adjacent pixels. The system scans the local surface area row by row and column by column, reading the color channel value and brightness value of two adjacent pixels each time. The system calculates the difference in the red channel, green channel, blue channel, and brightness value respectively, and then combines these differences according to preset weights to form the color change intensity of the adjacent pixel pair. For each pixel, the system reads the color change intensity of its horizontal and vertical adjacent pixel pairs and takes the largest value as the color gradient value of that pixel. The system summarizes the color gradient values ​​of all pixels in the local surface area to obtain a color change feature map. Regions with large color gradient values ​​correspond to color abrupt change boundaries, and regions with continuous distribution of color gradient values ​​correspond to color anomalous zones.

[0062] Furthermore, the extraction of microstructure morphological features is based on the brightness distribution, edge shadows, and texture deformation in the visible light surface image. The system first extracts the surface structure edges within a local surface region. During extraction, the system compares the brightness difference between each pixel and its neighboring pixels; when the brightness difference exceeds a preset edge threshold, the pixel is marked as a structural edge pixel. Subsequently, the system reads the brightness change direction along both sides of the structural edge. If bright pixels appear consecutively on one side of a structural edge and dark shadow pixels appear consecutively on the other side, the system identifies this location as a protruding convex / concave candidate region; if the center of a region is continuously lower than the surrounding brightness, and a ring-shaped or strip-shaped brightness transition boundary is formed on the outer side, the system identifies this location as a concave / concave candidate region.

[0063] The depth value of the microstructure's concavity / convexity morphological features is jointly determined by the brightness difference, shadow width, and edge transition width. The system first identifies the highest and lowest brightness positions within the concavity / convexity candidate region, then calculates the brightness difference between them. The system then measures the pixel width extending outwards from the structure's edge in the low-brightness shadow region, and the pixel width traversed as brightness transitions from the high-brightness to the low-brightness region. The greater the brightness difference, the wider the shadow extension, and the more concentrated the edge transition, the greater the depth value assigned by the system. To avoid misjudgments caused by uneven overall surface illumination, the system also compares the brightness difference of the concavity / convexity candidate region with the average brightness difference of its surrounding normal surface region. Only concavity / convexity candidate regions exceeding the normal surface fluctuation range are assigned a valid depth value. The resulting depth value characterizes the degree of concavity or protrusion of the local microstructure relative to the surrounding surface.

[0064] Specifically, the gradient value of the color change feature is compared with a preset color gradient threshold, and the depth value of the microstructure morphology feature is compared with a preset morphology depth threshold. When the color gradient value of a local surface region is greater than the preset color gradient threshold, and the morphology depth value of the same or adjacent regions is greater than the preset morphology depth threshold, the system confirms that the local surface region has a visible surface anomaly. Here, "same region" refers to overlapping boundary pixels of the two types of features; "adjacent region" refers to the nearest pixel distance between the boundaries of the two types of features being less than a preset association distance. If the color gradient value is not greater than the preset color gradient threshold, the system does not use the color change as a basis for boundary fusion; if the morphology depth value is not greater than the preset morphology depth threshold, the system does not use the morphology candidate region as a basis for boundary fusion. After this processing, the visible light features entering boundary fusion simultaneously satisfy the color change abruptness condition and the morphology anomaly condition.

[0065] Furthermore, boundary fusion is performed centered on the location of local hot spots. The system first reads the outer contour of the temperature anomaly region where the local hot spot is located, as the thermal anomaly boundary. The thermal anomaly boundary is composed of the outermost anomalous pixels of the temperature anomaly region. The system then extracts the color anomaly boundary from the color change feature map by tracing the connectivity of pixels with color gradient values ​​greater than a preset color gradient threshold, and forming closed or semi-closed contours by connecting consecutive color gradient pixels. Afterward, the system extracts the concavity and convexity anomaly boundary from the microstructure concavity and convexity morphology feature map by tracing the boundary pixels along the outer edge of the region with a depth value greater than a preset concavity and convexity depth threshold. All three types of boundaries are located in the visible light surface map coordinate system, and the system records the coordinates of each boundary point, the boundary direction, and the boundary connectivity.

[0066] During the fusion process, the system first confirms the spatial relationship between local hotspot locations and three types of boundaries. If a thermal anomaly boundary includes a local hotspot location, the system uses the thermal anomaly boundary as the initial fusion boundary. If a local hotspot location falls outside the thermal anomaly boundary but is less than a preset distance from it, the system includes the shortest path between the local hotspot location and the thermal anomaly boundary in the initial fusion range. Subsequently, the system merges color anomaly boundaries and convexity anomaly boundaries with the initial fusion boundary in sequence. The merging rules are as follows: when color anomaly boundaries or convexity anomaly boundaries have overlapping pixels with the initial fusion boundary, the overlapping parts are directly merged; when there is no overlap but the nearest boundary distance is less than the preset fusion distance and the boundary directions are continuous, the system connects the nearest endpoints of the two and merges them; when the distance between the two exceeds the preset fusion distance, or connecting them would cause the boundary to cross the normal surface area, the system does not merge them.

[0067] For regions that simultaneously meet both the color gradient threshold and the concavity / convexity depth threshold, the system increases the retention priority of these regions during the fusion process. Specifically, the system marks these regions as strongly correlated surface anomaly regions. When a strongly correlated surface anomaly region is connected to a local hot spot, the system fully incorporates it into the potential thermal anomaly segment. When a strongly correlated surface anomaly region is not directly connected to a local hot spot but is located within the same temperature anomaly region, the system checks whether a continuous thermal response transition zone exists between them. If a continuous thermal response transition zone exists, the system incorporates the strongly correlated surface anomaly region into the fusion range; if no continuous thermal response transition zone exists, the system retains this region as an independent candidate region and does not incorporate it into the current potential thermal anomaly segment.

[0068] After boundary fusion is completed, the system performs boundary closure and range determination. For breakpoints in the fusion boundary, the system calculates the pixel distance between adjacent breakpoints; when the breakpoint distance is less than a preset closure distance, the system connects the two breakpoints according to the shortest pixel path. For small holes inside the fusion boundary, the system determines whether the hole area is less than a preset hole area threshold; if the hole area is less than the preset hole area threshold, the system fills the hole; if the hole area reaches the preset hole area threshold, the system retains the hole as a non-abnormal internal region. Subsequently, the system deletes isolated color aberration fragments and isolated bump aberration fragments outside the boundary that are not connected to local overheating spots, retaining regions with thermal response continuity or boundary continuity with local overheating spots.

[0069] Finally, the system calculates the pixel range covered by the fused closed boundary and defines this pixel range as the spatial range of the potential thermal anomaly zone. This spatial range includes local hotspot locations, the main body of the temperature anomaly region, color change regions reaching a preset color gradient threshold, and microstructural anomaly regions reaching a preset concavity / convexity depth threshold. The system outputs the coordinates of the circumscribed rectangle, the coordinates of the closed contour, the center coordinates, and the coverage area of ​​this spatial range.

[0070] S3 includes acquiring the corresponding position coordinates and scanning to obtain a high-resolution low-light image; separating the microcrack contour and foreign object particle contour from the high-resolution low-light image; extracting microcrack width and crack hierarchy structure data for the microcrack contour, and calculating foreign object particle size data for the foreign object particle contour; if the microcrack width exceeds a preset width threshold or the crack hierarchy structure data is abnormal, it is determined that there is an abnormal crack structure; if the foreign object particle size data is greater than a preset size threshold, it is determined that there is an abnormal foreign object particle size, thereby realizing the identification of whether there is an abnormal crack structure or an abnormal foreign object particle size.

[0071] In this embodiment, when the system acquires the corresponding position coordinates, it first reads the spatial range of the potential thermal anomaly segment determined in the previous steps. This spatial range includes the coordinates of the closed contour, the coordinates of the circumscribed rectangle, the center coordinates, and the coverage area. Using the visible light surface map coordinate system as a reference, the system takes the center coordinates of the potential thermal anomaly segment as the scanning center of the micro-light microscopy system and the coordinates of the circumscribed rectangle as the scanning boundary. To prevent the boundary from truncating the ends of cracks or the edges of foreign particles, the system adds boundary margins in four directions of the circumscribed rectangle. The boundary margins are determined by the average error of the previous image registration, the actual length corresponding to a single pixel in the micro-light microscopy system, and the platform repetition error. Specifically, the average image registration error is first converted into pixel distance in the visible light surface map, then the platform repetition error of the micro-light microscopy system is converted into the same pixel scale, and finally, the sum of the two is added by two pixels as the scanning boundary margin. The expanded area is used as the actual scanning area of ​​the micro-light microscopy system.

[0072] Furthermore, when converting the corresponding position coordinates to the coordinates of the motion platform of the low-light microscopy system, the system invokes the calibration relationship established before detection. The calibration process is as follows: a calibration piece with standard scribe lines or standard positioning points is placed on the stage of the low-light microscopy system, and visible light surface images and low-light microscopic images are acquired respectively. The system identifies no fewer than three non-collinear positioning points on the calibration piece and records the pixel coordinates of these positioning points in the visible light surface image and their actual coordinates in the microscopic stage. The system establishes a mapping relationship from pixel coordinates to the actual displacement coordinates of the stage based on these positioning points. During detection, the system inputs the center coordinates and boundary coordinates of the potential thermal anomaly section into this mapping relationship to obtain the lateral movement distance, longitudinal movement distance, scan start point, scan end point, and scan path of the stage.

[0073] When the actual scanning area is less than or equal to the field of view of a single microscopy session, the system acquires a high-resolution low-light image at the corresponding coordinates. When the actual scanning area is greater than the field of view of a single microscopy session, the system divides the scanning area into multiple scanning units. The width and height of each scanning unit are determined by the effective field of view of the low-light microscopy system, and a 15% overlap is maintained between adjacent scanning units. The overlap is used to identify the same surface structure during subsequent image stitching, preventing cracks, fractures, or misalignment of particle boundaries caused by stitching seams. The scanning path is generated in a line-by-line or serpentine manner, and the system records the center coordinates and acquisition order of each scanning unit.

[0074] Furthermore, before scanning and imaging, the system determines the microscopic imaging magnification. The magnification is determined based on the minimum crack width and the minimum particle size to be identified. Before inspection, the system reads the minimum crack width and minimum foreign object particle size required to be identified in the product inspection specifications, and reads the actual length of a single pixel corresponding to different magnifications in the low-light microscopy system. The system selects a magnification that meets the following conditions: the minimum crack width corresponds to no less than 3 pixels in the image, and the minimum foreign object particle diameter corresponds to no less than 5 pixels in the image. If multiple magnifications meet the conditions, the system selects the magnification with the largest field of view to reduce the number of scans. After this magnification is determined, the system writes it into the parameters for this scan.

[0075] Autofocus is then performed. During autofocus, the system continuously acquires test images at different focal lengths at the center of the scanning unit and calculates the edge sharpness for each test image. The edge sharpness calculation process is as follows: the system counts the number of pixels in the image whose brightness changes between adjacent pixels reach the edge condition, and then counts the intensity of the brightness change of these edge pixels. The number of edges and the intensity of the change are used together as the sharpness evaluation value. The focal length position with the highest sharpness evaluation value is taken as the focal plane of the scanning unit. After the focal plane is determined, the system sets the exposure time and illumination intensity so that the brightness distribution of the effective area in the image is within the preset brightness range. The preset brightness range is determined by the effective dynamic range of the camera in the low-light microscope system. 20% to 80% of the camera's saturation value is taken as the effective brightness range. Areas below 20% will lose details of dark cracks, and areas above 80% will produce high reflectivity saturation.

[0076] During the scanning process, after the stage moves to the center coordinates of each scanning unit, the system automatically focuses and acquires local low-light images. After all scanning units are acquired, the system first performs initial stitching based on the stage coordinates of the scanning units, and then performs fine alignment using the same textures, edges, and particle positions in the overlapping areas of adjacent images. If there is a misalignment at the same structural position in the overlapping area between two adjacent local images, the system corrects the stitching position by the offset. After stitching all local images, the system obtains a high-resolution low-light image covering the potential thermal anomaly area. This image includes fine lines of cracks on the chip surface, outlines of foreign particles, surface texture, and the substrate background.

[0077] Before separating the microcrack contours from high-resolution low-light images, the system performs image preprocessing. The system performs brightness equalization on the entire image, specifically by reading the average brightness of local areas and adjusting areas with excessive or insufficient brightness to a uniform background level. Subsequently, the system removes isolated bright spots and dark spots. Isolated spots are determined based on their brightness difference from surrounding pixels and the number of connected pixels; when a bright spot or dark spot occupies only one pixel or a few adjacent pixels and is not connected to linear or granular structures, it is removed as noise. After preprocessing, the differences between crack boundaries, foreign particle boundaries, and the substrate background are preserved, while random noise is suppressed.

[0078] Specifically, the separation of microcrack contours employs a process of "linear structure recognition plus contour restoration." The system scans high-resolution low-light images row by row and column by column, searching for elongated regions where grayscale or brightness abruptly changes relative to the surrounding area. For each candidate region, the system calculates its length, width, aspect ratio, extension direction, and boundary continuity. A crack candidate region must simultaneously meet the following conditions: its length is greater than its width, its two sides extend continuously in the same direction, and its internal grayscale exhibits a stable difference from the background of the substrate on both sides. The length threshold is determined by the pixel calibration results of the low-light microscopy system and the minimum effective crack length in the product inspection specifications. The system converts the minimum effective crack length into pixel counts as the length threshold. The aspect ratio threshold is obtained statistically from normal texture lines in qualified samples. The system collects normal surface textures from no fewer than 10 qualified samples, calculates the aspect ratio distribution of normal texture lines, takes the 95th percentile as the upper limit for normal textures, and adds one safety unit as the threshold for determining linear crack structures. Linear regions meeting these threshold conditions proceed to the crack contour restoration step.

[0079] In addition, during crack contour recovery, the system first extracts the centerline of the crack candidate region. The centerline is obtained by progressively shrinking the crack candidate region, preserving the crack's extension direction and endpoint positions during the shrinkage process. After obtaining the centerline, the system searches for crack boundaries to the left and right along each sampling point of the centerline. During the search, the system compares the brightness changes of pixels on both sides of the centerline; when the brightness changes from the crack interior state to the substrate background state, this position is recorded as a crack boundary point. The system connects all boundary points sequentially along the centerline to form a microscopic crack contour. If a candidate region exhibits a closed block structure, a sudden increase in width, or a boundary that coincides with the particle contour after centerline extraction, the system removes it from the crack results to avoid misjudging the edges of foreign particles as cracks.

[0080] Furthermore, the separation of foreign object particle contours employs a process of "region segmentation plus morphological screening." The system searches for local regions in high-resolution low-light images that exhibit color, brightness, or reflectance differences compared to the substrate background. The segmentation threshold for candidate foreign object regions is determined by the normal substrate region in the currently scanned image. The system reads pixel brightness and color values ​​in the normal substrate region outside the potential thermal anomaly area, removing the highest 5% of highly reflective pixels and the lowest 5% of shadow pixels, and calculating the normal fluctuation range of the remaining pixels. Pixels exceeding this normal fluctuation range and forming continuous regions are extracted as candidate foreign object regions. Subsequently, the system examines the geometry of the candidate regions; only regions with closed boundaries, areas reaching the minimum particle area threshold, and internal textures discontinuous with the substrate texture are retained as foreign object particle contours.

[0081] Furthermore, the minimum particle area threshold is determined by the resolution of the microscopic system. Before inspection, the system reads the actual area corresponding to a single pixel and the minimum effective particle area specified in the product inspection specification. The system converts the minimum effective particle area into the number of pixels to obtain the minimum particle area threshold. If the product inspection specification only specifies the minimum particle diameter, the system first calculates the particle projected area based on this diameter, and then converts it into the number of pixels. When the number of pixels in a candidate region is less than the minimum particle area threshold, the system deletes it as surface noise or tiny reflective points; when the number of pixels in a candidate region reaches the threshold, the system retains its outer contour, center position, enclosing boundary, and internal pixel set.

[0082] When extracting the width of a microcrack from its contour, the system uses the crack centerline as the measurement reference. The system samples sequentially from the crack initiation point along the centerline to the termination point, establishing a measurement section perpendicular to the centerline at fixed pixel intervals. The fixed pixel interval is determined by the microscopic image resolution, taking half the number of pixels corresponding to the minimum crack width, and rounding up to an integer. Each measurement section extends from the centerline to both sides until it encounters the left and right boundaries of the crack, respectively. The system calculates the pixel distance between the left and right boundary points and then converts it into the actual width based on the pixel calibration results of the microscopic system. For the same crack, the system obtains multiple width measurements and outputs the maximum width, average width, minimum width, and width fluctuation range. The maximum width is prioritized when determining the crack width because it corresponds to the most severe crack opening location.

[0083] The determination of the microcrack width threshold is based on a combination of product process specifications, reliability test results, and microscopic system measurement errors. The system first reads the maximum allowable crack width from the product process specifications; this value serves as the process upper limit. Next, the system reads the minimum crack width from the reliability test database for the same chip model that leads to electrical performance drift, increased thermal resistance, or decreased package reliability, and multiplies this minimum crack width by 0.8 to obtain the reliability control upper limit. Subsequently, the system measures the width error of the microscopic system using a standard scribing tool, performing at least 10 consecutive measurements, and taking the maximum measurement error as the measurement safety margin. Finally, the system subtracts the measurement safety margin from the smaller of the process upper limit and the reliability control upper limit to obtain the preset width threshold. This predetermined width threshold simultaneously satisfies both manufacturing control and reliability control requirements while compensating for microscopic measurement errors.

[0084] For crack hierarchy data, the system first converts the crack profile into a crack centerline network. The system then identifies main cracks, branch cracks, and junctions within this network. The main crack is determined by the longest and most continuous centerline segment; cracks directly connected to the main crack and extending laterally are designated as layer 1 branches; cracks continuing to extend from layer 1 branches are designated as layer 2 branches; subsequent branches are numbered layer by layer according to the same connection relationship. The system records the number of main cracks, the number of layer 1 branches, the number of layer 2 branches, the highest branch level, the number of branch junctions, the total branch length, and the crack network coverage area. For each crack, the system also records whether its endpoints are close to pads, metal traces, or chip edges.

[0085] Furthermore, the threshold for crack hierarchy structure anomalies is jointly determined by statistical values ​​from qualified samples and product failure rules. The system selects no fewer than 10 chip samples of the same model, process, and electrical performance that have passed testing. Following the same low-light microscopy imaging process, it extracts normal surface textures and permissible microcrack features, and statistically analyzes the number of main cracks, branches, intersections, the highest branch level, and the crack network coverage area. For quantity-related indicators, the system takes the maximum value from the qualified sample statistical results and adds 1 as the corresponding threshold. For area-related indicators, the system takes the 95th percentile of the qualified sample results and adds a measurement safety value as the corresponding threshold. If the product failure rules stipulate that cracks must not cross metal traces, connect to pad edges, or form multi-level branches, this rule is directly written into the system as the criterion for hierarchical structure anomalies. During detection, if any one of the following exceeds the corresponding threshold: number of main cracks, number of branches, number of intersections, highest branch level, or network coverage area; or if a crack connects to a prohibited area, the system determines that the crack hierarchy structure data is abnormal.

[0086] When calculating the size data of foreign object particles based on their outlines, the system processes each particle individually. First, the system counts the number of pixels within the particle's outline and converts the particle's projected area based on the actual area of ​​each pixel. Then, the system finds the two furthest boundary points on the particle's outer outline and uses the actual distance between these two boundary points as the particle's maximum size. Next, the system measures the particle's maximum width along a direction perpendicular to the maximum size direction, using this as the particle's minimum size. For particles with outlines close to circles or ellipses, the system converts the projected area into an equivalent diameter; for irregularly shaped particles, the system retains the maximum size, minimum size, projected area, equivalent diameter, and aspect ratio. If multiple particles exist within the scanned area, the system generates independent size data for each particle and calculates the total number of particles, the maximum particle size, and the particle distribution density.

[0087] In this embodiment, the foreign particle size threshold is jointly determined by the product contamination control specifications, the minimum structural spacing on the chip surface, and the measurement error of the microscopic system. The system first reads the maximum allowed foreign particle size in the product contamination control specifications and uses it as the upper limit of the process tolerance. The system then reads the minimum conductive structural spacing in the current chip layout or package structure and takes 10% of this minimum conductive structural spacing as the structural safety upper limit. Subsequently, the system uses a standard particle sample to calibrate the microscopic system for size measurement, performing at least 10 consecutive measurements, and taking the maximum measurement error as the particle measurement safety amount. Finally, the system subtracts the particle measurement safety amount from the smaller of the process tolerance upper limit and the structural safety upper limit to obtain the preset size threshold. Once this threshold is determined, the system writes it into the current detection parameter table for subsequent foreign particle size anomaly judgment.

[0088] Furthermore, the crack structure anomaly detection is divided into two paths: width detection and hierarchy detection. The system first reads the maximum width of each crack and compares it with a preset width threshold. If the maximum width of any crack exceeds the preset width threshold, the system immediately determines that a crack structure anomaly exists. If the width of all cracks does not exceed the preset width threshold, the system continues to read the crack hierarchy structure data and compares it item by item with the corresponding hierarchy structure threshold. If any one of the following criteria—the number of main cracks, the number of branches, the highest branch hierarchy, the number of intersection points, the crack network coverage area, or the connection relationship of prohibited regions—meets the anomaly criteria, the system also determines that a crack structure anomaly exists. This detection method includes both width-exceeding cracks and multi-level propagation cracks in the crack structure anomaly results.

[0089] The determination of abnormal foreign object particle size is based on particle size data. The system reads the maximum size and equivalent diameter of each foreign object particle and compares them with preset size thresholds. For elongated particles, the system uses the maximum size as the primary criterion; for aggregated particles, the system uses the equivalent diameter and projected area as the primary criterion. If the maximum size or equivalent diameter of any single particle is greater than the preset size threshold, the system determines that there is an abnormal foreign object particle size. If the size of any single particle does not exceed the preset size threshold, but the total number of particles or the particle distribution density exceeds the quantity threshold in the product contamination control specifications, the system records this result as a foreign object aggregation risk and uses it as input for subsequent multimodal risk assessment models; the direct output of "abnormal foreign object particle size" in this step is still based on whether the size data of a single particle is greater than the preset size threshold.

[0090] After completing the above processing, the system outputs the identification results and corresponding data. When the width of the microcrack exceeds the preset width threshold, or when the crack hierarchy structure data is abnormal, the system outputs that there is a crack structure abnormality, and simultaneously outputs the crack outline coordinates, crack centerline, maximum width, average width, branch level, number of intersection points, and crack network coverage. When any foreign object particle size data is greater than the preset size threshold, the system outputs that there is a foreign object particle size abnormality, and simultaneously outputs the foreign object particle outline coordinates, maximum size, minimum size, projected area, equivalent diameter, and particle center coordinates. If both types of abnormalities are met simultaneously, the system outputs that both crack structure abnormalities and foreign object particle size abnormalities exist simultaneously. If neither type of abnormality is met, the system outputs that neither crack structure abnormality nor foreign object particle size abnormality was identified.

[0091] S4 includes extracting the number of crack branches and the foreign object embedding depth from the image to obtain a first feature set; locating abnormal regions based on the first feature set and extracting the phase gradient of the abnormal regions to generate a second feature set; fusing the stress distribution features extracted from the first feature set and the second feature set to construct a multimodal data matrix; establishing a model through the multimodal data matrix to calculate a comprehensive risk index; if the comprehensive risk index is greater than a preset threshold, obtaining a preliminary defect classification result based on the comprehensive risk index.

[0092] In this embodiment, before performing multimodal risk assessment, the system first reads the high-resolution low-light image, microcrack profile, foreign object particle profile, infrared phase-locked image, and infrared thermal distribution map obtained in the previous steps. All of these data have been spatially registered, therefore the same defect location has corresponding coordinate relationships in the low-light image, visible light surface map, infrared thermal distribution map, and phase-locked image. The system uses this coordinate relationship as the basis for subsequent feature extraction and risk calculation, avoiding positional mismatches between different modal data.

[0093] Furthermore, when extracting the number of crack branches in the image, the system first reads the microscopic crack outline and refines it into the crack centerline. During the refinement process, the system retains the crack endpoints, intersections, and extension directions, converting crack outlines with inconsistent widths into a continuous linear skeleton. The system then traces the crack centerline point by point, identifying crack endpoints, intersections, and bifurcation points. The longest and continuous crack path is identified as the main crack; crack segments extending laterally from and intersecting the main crack are identified as the first-level branches; crack segments extending outward from the first-level branches are identified as the second-level branches; subsequent branches are identified according to the same connection relationships. The system counts the number of main cracks, the number of first-level branches, the number of second-level branches, the highest branch level, the number of branch intersections, and the total number of branches as characteristics of the number of crack branches.

[0094] Whether crack branches are included in the statistics requires effective branch screening. The effective branch length threshold is determined jointly by the pixel calibration results of the low-light microscopy system and the minimum effective crack length in the product inspection specifications. The system reads the minimum identifiable effective crack length in the product inspection specifications, and then converts the minimum effective crack length into the number of pixels based on the actual length corresponding to a single pixel in the low-light microscopy system. This number of pixels serves as the effective branch length threshold. Linear structures with a length less than this threshold are not included in the crack branch count. The branch angle threshold is obtained from the statistics of normal textures on the surface of qualified chips. The system collects no less than 10 qualified samples, extracts the angle distribution between normal texture lines, takes the upper limit of the normal texture angle distribution, and adds an angle compensation corresponding to a pixel-level direction error, as the branch angle threshold. Only linear structures with an angle to the main crack or the previous branch reaching this threshold and a length reaching the effective branch length threshold are included in the crack branch count. After this processing, normal processing textures, shallow scratches, and noise lines will not be mistakenly counted as crack branches.

[0095] When extracting the foreign object embedding depth, the system first reads the outline of the foreign object particle and the surrounding substrate area. The system selects a ring of substrate area outside the foreign object particle that is not covered by the particle, not penetrated by cracks, and has stable brightness; this area is used as the substrate reference plane. The system reads the width of the light-dark transition at the boundary of the foreign object particle, the shadow width, and the indentation range of the particle edge from the high-resolution low-light image. For images obtained through autofocus scanning, the system also reads the focal length position when the foreign object particle area reaches its highest edge sharpness, and the focal length position when the surrounding substrate area reaches its highest edge sharpness. The focal length difference between the foreign object particle area and the substrate area is used to represent the height change of the particle relative to the substrate surface; the boundary shadow width and indentation range are used to represent the degree to which the particle is pressed into the substrate. The system converts the focal length difference, shadow width, and indentation range into embedding depth characterization values ​​and records the maximum embedding depth, average embedding depth, and embedding area.

[0096] As a preferred embodiment, the effective threshold for determining the foreign object embedding depth is jointly determined by the axial measurement error of the microscopy system and the surface undulations of qualified samples. The system uses a standard step plate to calibrate the low-light microscopy system axially, performing at least 10 consecutive measurements, and taking the maximum repeatability measurement error as the axial measurement error. Subsequently, the system collects normal surface areas from at least 10 qualified chips, calculates the normal surface undulation depth, and removes the highest 5% of reflective anomalies and the lowest 5% of shadow anomalies, taking the 95th percentile as the upper limit of normal undulation. The system adds the upper limit of normal undulation to the axial measurement error to obtain the effective threshold for determining the foreign object embedding depth. Only when the foreign object embedding depth is greater than this threshold is the foreign object recorded as a target with an effective embedding depth. After the above processing, the system writes the number of crack branches, the highest branch level, the number of intersection points, the maximum embedding depth of the foreign object, the average embedding depth of the foreign object, and the number of embedded foreign objects into the first feature set.

[0097] Specifically, when locating abnormal regions based on the first feature set, the system processes crack features and foreign object features separately. For crack features, the system reads the coordinates of the main crack, branch cracks, intersection points, and endpoints to generate a crack circumscribed region covering the entire crack structure. For foreign object features, the system reads the outer contour of the foreign object particle, the coordinates of the particle center, the maximum embedding depth, and the indentation boundary to generate a foreign object circumscribed region covering the foreign object and its embedding influence range. After the crack circumscribed region and the foreign object circumscribed region are formed, the system compares the spatial relationship between the two types of regions. If the two types of regions overlap, the system merges them into the same abnormal region; if the two types of regions do not overlap, but their nearest distance is less than the region merging threshold, the system still merges them; if their nearest distance reaches or exceeds the region merging threshold, the system treats them as two independent abnormal regions.

[0098] Specifically, the region merging threshold is jointly determined by the image registration error, the repeatability error of the low-light microscopy system platform, and the range of infrared thermal diffusion influence. The system first reads the average reprojection error of the matching points in the preceding registration step, then reads the repeatability error of the low-light microscopy system platform, and converts both to the pixel scale of the visible light surface image. The system then reads the minimum thermal diffusion radius corresponding to the current chip material and packaging structure, and converts this thermal diffusion radius into a pixel distance. The sum of these three pixel distances is used as the region merging threshold. Using this threshold, regions originating from the same physical defect but exhibiting slight offsets in different imaging modalities can be merged, while unrelated defect regions will not be forcibly merged.

[0099] Specifically, after generating the anomalous region, the system extracts the phase gradient of the anomalous region to form a second feature set. The system first uses the prior registration relationship to map the anomalous region from the visible light surface map coordinate system to the phase-locked phase map coordinate system. After mapping, the system reads the phase-locked phase value of each pixel within the anomalous region and reads the normal reference band outside the anomalous region. The normal reference band is formed by extending outward from the outer boundary of the anomalous region, and its width is determined by the infrared image registration error and the minimum thermal diffusion radius. The reference band must not cross other identified anomalous regions. If the reference band overlaps with other anomalous regions, the system deletes the overlapping portion, retaining only the phase data corresponding to the defect-free surface.

[0100] Specifically, the phase gradient calculation process is as follows: The system compares the phase value differences between adjacent pixels within the abnormal region point by point. For each pixel, the system reads the phase values ​​of its left, right, top, and bottom neighboring pixels, and calculates the phase change between that pixel and its neighboring pixels. The system then statistically analyzes the phase change intensity along the transverse, longitudinal, and main crack propagation directions to obtain the phase gradient characterization result at each location. If the phase changes of adjacent pixels within a certain region are concentrated, and the direction of this change is continuous with the crack propagation direction, the foreign object boundary direction, or the thermal diffusion direction, then that region is identified as a concentrated phase gradient region.

[0101] Furthermore, the phase gradient threshold is obtained through calibration using normal samples. The system acquires phase-locked phase maps of qualified chips of the same model under the same operating current, the same phase-locked excitation frequency, the same imaging distance, and the same ambient temperature. Regions free of cracks, foreign objects, and localized overheating are selected as normal reference regions. The system calculates the phase change intensity between adjacent pixels within the normal reference region, removes the highest 5% and lowest 5% discrete values, and then takes the 95th percentile as the upper limit of the normal phase gradient. Subsequently, the system calculates the phase compensation amount based on the phase position offset caused by registration errors and adds this compensation amount to the upper limit of the normal phase gradient to obtain the phase gradient threshold. During detection, if the phase gradient in an abnormal region exceeds this threshold, the system records it as a phase gradient anomaly. The second feature set includes the maximum phase gradient value, average phase gradient, abnormal phase gradient area, phase gradient direction, number of phase gradient concentration points, and the overlap area between the abnormal phase gradient region and the crack or foreign object contour.

[0102] When fusing the first and second feature sets, the system maps data according to the anomaly region number. Each anomaly region corresponds to one set of crack branch data, one set of foreign object embedding data, and one set of phase gradient data. If an anomaly region contains only cracks and no foreign objects, the foreign object embedding related data is recorded as 0; if an anomaly region contains only foreign objects and no cracks, the crack branch related data is recorded as 0; if an anomaly region contains both cracks and foreign objects, both types of surface structure features are retained. The system merges the first and second feature sets within the same anomaly region and then continues to extract stress distribution features.

[0103] Furthermore, the stress distribution characteristics are determined through the spatial relationship between structurally weak locations and phase gradient concentration locations. The system first marks structurally weak locations within the anomalous region. These locations include crack endpoints, crack junctions, crack bifurcation points, sharp corners of foreign object boundaries, points of maximum foreign object embedding depth, and edges of foreign object indentations. Subsequently, the system checks for phase gradient concentration regions around these structurally weak locations. If a phase gradient concentration exists around a crack endpoint, the system records this location as a crack tip stress concentration point; if a phase gradient concentration exists around a crack junction, the system records this location as a bifurcation stress concentration point; if a ring-shaped or semi-ring-shaped phase gradient change exists around the deepest foreign object embedding location, the system records this location as a foreign object indentation stress concentration point. The system then counts the number of stress concentration points, the area of ​​the stress concentration region, the distance between the stress concentration point and the edge of the metal trace or pad, the stress concentration direction, and the continuous length of the stress concentration.

[0104] In this embodiment, the stress concentration threshold is jointly determined by qualified samples and failed samples. The system extracts the number and area of ​​phase gradient concentration points near the edges of normal structures from qualified samples to obtain the normal stress response range; then, it extracts the number and area of ​​phase gradient concentration points from known crack propagation, foreign object indentation, or thermal failure samples to obtain the failure stress response range. The system selects a critical value that can separate qualified and failed samples as the stress concentration point number threshold and stress concentration area threshold. If the two types of samples overlap, the system prioritizes avoiding missed detections and sets the threshold between the upper limit of qualified samples and the lower limit of failed samples, close to the upper limit of qualified samples. During detection, if the number of stress concentration points is greater than the stress concentration point number threshold, or the area of ​​the stress concentration region is greater than the stress concentration area threshold, the abnormal region is recorded as having a stress concentration risk.

[0105] Specifically, when constructing the multimodal data matrix, the system treats each abnormal region as a data record and uses various features as corresponding fields. Each data record sequentially contains the abnormal region number, region center coordinates, region area, total number of crack branches, highest branch level, number of crack intersections, maximum foreign object embedding depth, average foreign object embedding depth, number of embedded foreign objects, maximum phase gradient, average phase gradient, phase gradient anomaly area, number of phase gradient concentration points, stress concentration area, and distance from the abnormal region to the nearest functional structure. Functional structures include pads, metal traces, chip edges, and heat-sensitive areas. If a field does not have a corresponding defect source, the system writes 0; if the feature corresponding to a field cannot be effectively extracted from the current image, the system writes an invalid flag and does not include that field in the risk index calculation during model computation.

[0106] Specifically, the multimodal data matrix needs to undergo feature standardization before being input into the model. The system establishes a standardization benchmark before detection. This benchmark is derived from qualified, suspicious, and failed samples of the same chip model. For features such as the number of crack branches, foreign object embedding depth, phase gradient, and stress concentration area, the system records the upper limit of normal risk in qualified samples and the lower limit of risk risk in failed samples, respectively. During detection, the system compares the feature values ​​of the current abnormal region with the corresponding standardization benchmark. When the feature value is below the upper limit of normal risk, the feature receives a low-risk score; when the feature value is between the upper limit and the lower limit of risk risk, the feature receives an intermediate-risk score; when the feature value reaches or exceeds the lower limit of risk risk, the feature receives a high-risk score. After this processing, features of different dimensions are converted into a unified risk score.

[0107] When calculating the comprehensive risk index using a multimodal data matrix model, the system weights and summarizes the risk scores for each item. The weights are determined through training on historical test samples, not through arbitrary manual assignment. The system reads the correspondence between various features in historical samples and actual failure results, statistically analyzing the contribution of crack branching number, foreign object embedding depth, phase gradient anomaly area, and stress concentration area to the failure results. Features that cause electrical performance drift, increased thermal resistance, encapsulation cracking, or localized ablation are assigned high weights; features that only cause slight surface changes and are not associated with thermal or phase gradient anomalies are assigned low weights. During model calculation, the system multiplies each standardized risk score by its corresponding weight, and then sums the weighted results of all valid features to obtain the comprehensive risk index. The higher the comprehensive risk index, the higher the risk that the anomaly area will develop into a substantial defect.

[0108] The preset threshold for the comprehensive risk index is determined during the model building phase. The system categorizes historical samples into qualified samples, questionable samples, and confirmed defect samples. Each historical sample generates a multimodal data matrix and calculates its comprehensive risk index according to the same process. The system sorts the comprehensive risk indices of qualified samples from smallest to largest, taking the 95th percentile as the upper limit of qualified risk; simultaneously, it sorts the comprehensive risk indices of confirmed defect samples from smallest to largest, taking the 5th percentile as the lower limit of defect risk. When the upper limit of qualified risk is less than the lower limit of defect risk, the system takes the midpoint between the two as the preset threshold. When there is overlap, the system adjusts the threshold according to product testing requirements: to avoid missed detections, the threshold is set below the 5th percentile of confirmed defect samples; to reduce false alarms, the threshold is set above the 95th percentile of qualified samples. The final threshold is entered into the testing system after verification by at least one batch of validation samples.

[0109] If the comprehensive risk index exceeds a preset threshold, the system obtains a preliminary defect classification result based on the comprehensive risk index. During classification, the system first checks if the comprehensive risk index exceeds the preset threshold; abnormal regions exceeding the preset threshold enter the preliminary defect classification process. The system then reads the feature sources that contribute the most to the comprehensive risk index. If the number of crack branches, the highest branch level, the number of crack intersections, and the stress concentration points at the crack tip are dominant, the system initially classifies the region as a crack propagation type defect. If the maximum embedding depth of foreign matter, the number of embedded foreign matter, the stress concentration points of foreign matter indentation, and the phase gradient contribution of particle boundaries are dominant, the system initially classifies the region as a foreign matter embedding type defect. If the maximum value of the phase gradient, the abnormal area of ​​the phase gradient, and the abnormal contribution of thermal response are dominant, while crack and foreign matter features do not meet the dominant conditions, the system initially classifies the region as a thermal response abnormal type defect. If crack branch features, foreign matter embedding features, and phase gradient features all exceed their respective risk lower limits, the system initially classifies the region as a composite defect.

[0110] When the comprehensive risk index is not greater than a preset threshold, the system does not output the preliminary defect classification result, but instead records the area as a low-risk abnormal area or an area to be reviewed. For low-risk abnormal areas, the system retains its first feature set, second feature set, multimodal data matrix, comprehensive risk index, and image evidence for subsequent batch review or model parameter updates. Finally, the system outputs the coordinates of the abnormal area, the comprehensive risk index, the preliminary defect classification result, the main risk contribution features, the corresponding threshold comparison result, and the associated image data.

[0111] S5 includes extracting the foreign object distribution density value from the preliminary defect classification data, processing the foreign object distribution density value to obtain a third feature set; extracting the foreign object adhesion degree value to the substrate based on the third feature set, processing the foreign object adhesion degree value to the substrate to obtain a fourth feature set; if the fourth feature set is greater than the preset risk threshold, then a high-risk defect area is determined; extracting the specific spatial coordinates of the high-risk defect area and classifying the risk level, thus determining the specific spatial coordinates and risk level classification of the high-risk defect area.

[0112] In this embodiment, the system first reads preliminary defect classification data. This preliminary defect classification data is output by the preceding multimodal risk assessment model and includes at least the anomaly region number, anomaly region contour coordinates, preliminary defect classification label, foreign object particle contour, foreign object particle center coordinates, foreign object particle size, foreign object particle quantity, foreign object embedding depth, overlap relationship between the foreign object and crack or thermal anomaly region, and comprehensive risk index. The system first filters regions related to foreign objects from all anomaly regions. The filtering rules are as follows: regions whose preliminary defect classification label belongs to foreign object embedding type defects or composite defects directly enter this step; regions whose preliminary defect classification label is not foreign object type, but have segmented foreign object particle contours, also enter this step. Regions without foreign object particle contours, without foreign object size data, and without foreign object embedding data are not included in the foreign object distribution density calculation.

[0113] When extracting the density values ​​of foreign objects, the system first determines the density statistical range. For each abnormal region entering this step, the system reads its closed contour coordinates and uses the pixel range inside the closed contour as the main statistical area. If there is a gap in the closed contour, the system first closes the gap according to the segmentation results of the previous image; if multiple unconnected sub-regions still exist after closure, the system calculates the density for each sub-region separately, and then retains the density of the largest sub-region and the overall region density. The system also generates the bounding rectangle of the abnormal region for subsequent coordinate output, but the density statistics are based on the actual area inside the closed contour to avoid including the normal background area in the bounding rectangle in the density area.

[0114] The system then counts the number of foreign particles within the main statistical area. Each foreign particle undergoes validity screening before counting. During screening, the system reads the particle's maximum size, projected area, outline closure degree, and boundary sharpness. Targets with a maximum size smaller than the minimum effective particle size threshold are not included in the count; targets with unclosed outlines and boundary sharpness below the effective boundary threshold are not included; targets that coincide with bright reflective points but lack a stable outline are not included. After screening, the system obtains the number of valid foreign particles.

[0115] The minimum effective particle size threshold is jointly determined by the product contamination control specifications and the resolution of the low-light microscopy system. The system first reads the minimum particle size of interest specified in the product contamination control specifications, and then reads the actual length of a single pixel in the low-light microscopy system at the current magnification. The system converts the minimum particle size of interest into pixel size, requiring that the particle occupy at least three consecutive pixels in the image. If the converted minimum particle size of interest in the product contamination control specifications is less than three pixels, the system uses the actual length corresponding to three pixels as the minimum effective particle size threshold; if the converted size is three pixels or more, the size corresponding to the product contamination control specifications is used as the minimum effective particle size threshold. This threshold simultaneously satisfies both the detection specification requirements and the microscopic imaging resolution.

[0116] The effective boundary threshold is determined by qualified samples and manually calibrated samples. The system collects no fewer than 10 normal reflective points, normal texture points, and non-particle edges from the surface of qualified chips, and then collects no fewer than 10 calibration samples containing real foreign matter particles. The system separately analyzes the brightness abruptness, contour closure ratio, and boundary continuity length of the target edges in the two types of samples. The system takes the 5% position with the lowest boundary sharpness in the real foreign matter particle samples as the lower limit of the effective particle boundary, while confirming that this lower limit is higher than the upper limit of the boundary sharpness of normal reflective points. If the two types of samples overlap, the system sets the effective boundary threshold at the 5th percentile of the boundary sharpness of the real foreign matter particles to avoid missed detections.

[0117] After obtaining the effective number of foreign matter particles, the system calculates the foreign matter distribution density. During calculation, the system first counts the number of pixels within the main statistical region, and then converts the pixel count into an actual area based on the pixel calibration results of the low-light microscopy system. The effective number of foreign matter particles is divided by this actual area to obtain the overall foreign matter distribution density of the anomalous region. For large anomalous regions, the system continues to perform local density calculations. Specifically, the system divides the anomalous region into multiple grid cells, and counts the effective number of foreign matter particles and the actual area of ​​each grid cell to obtain the local foreign matter distribution density. When a grid cell crosses the boundary of the anomalous region, only the pixel area falling within the closed contour is calculated.

[0118] The grid cell size is determined jointly based on the product contamination assessment area and the microscopic scanning field of view. If the product contamination control specification stipulates a particle limit per unit area, the system directly uses that unit area as the grid area. If the specification does not stipulate a particle limit per unit area, the system reads the minimum functional structure spacing of the chip and the single effective field of view area of ​​the low-light microscopy system, and selects the area that can cover at least one minimum functional structure spacing and no more than one microscopic field of view area as the grid cell. In this way, the local density calculation can reflect the particle aggregation situation without merging particles that are far apart into the same risk assessment.

[0119] When processing foreign object distribution density values ​​to obtain the third feature set, the system expands a single density value into a set of density description data. The third feature set includes: the overall foreign object distribution density of the abnormal region, the highest grid density, the average grid density, the number of grids with densities exceeding the normal density threshold, the total number of effective foreign object particles, the grid density of the largest particle, the coordinates of the foreign object aggregation center, the direction of the foreign object density gradient, and the percentage of the area covered by foreign objects. The foreign object aggregation center is determined by the grid with the highest density; if multiple consecutive high-density grids exist, the system takes the geometric center of the region composed of these high-density grids as the foreign object aggregation center. The direction of the foreign object density gradient points from low-density areas to high-density areas, representing the trend of foreign objects accumulating in local areas.

[0120] The normal density threshold is determined before testing. The system prioritizes reading the maximum allowable particle count and corresponding statistical area specified in the product's contamination control specifications and converts it into a unit area density threshold. If the product's contamination control specifications do not provide explicit density requirements, the system collects normal surface areas from no fewer than 10 qualified chips of the same model and calculates the foreign matter density according to the same particle screening rules and the same grid division method. After deleting the highest 5% and lowest 5% of discrete high values ​​and low values, the system takes the 95th percentile of the remaining data as the normal density threshold. If there are records of short circuits, increased thermal resistance, or contamination diffusion caused by particle aggregation in historical failure samples, the system extracts the lowest failure density of the failure samples, uses the median value between the normal density threshold and the lowest failure density as the risk density cutoff value, and writes it into the testing parameter table.

[0121] When extracting the adhesion degree values ​​between foreign objects and the substrate based on the third feature set, the system first identifies the foreign object targets participating in the adhesion analysis. These targets include foreign object particles located in the region with the highest mesh density, foreign object particles whose size reaches the minimum effective particle size threshold, foreign object particles located near pad edges or metal traces, foreign object particles whose embedding depth has been recorded in previous steps, and foreign object particles located within the boundaries of thermal anomaly regions. Other isolated particles only retain their density contribution and are not considered as primary targets for adhesion degree determination.

[0122] The adhesion degree between the foreign object and the substrate is determined by four types of data. The first type is the boundary adhesion ratio. The system reads the continuous boundary length between the particle and the substrate along the outer contour of the foreign object particle, and then compares this continuous boundary length with the total length of the particle contour to obtain the boundary adhesion state. The second type is the boundary shadow width. The system reads the width of the low-brightness shadow band from the particle edge outwards, excluding background shadows caused by the overall lighting direction. The third type is the embedding depth. The system reads the particle focal plane offset, indentation width, and light-dark transition width obtained in previous steps, and converts this information into a depth characterization value representing the particle's embedment into the substrate surface. The fourth type is the contour deformation degree. The system determines whether there are flattened sections, edge recessed sections, blurred boundary bands, or material accumulation edges on the side of the particle near the substrate.

[0123] The specific processing procedure for the adhesion score is as follows: The system reads the boundary fit ratio, boundary shadow width, embedment depth, and contour deformation degree for each particle participating in the adhesion analysis. The higher the boundary fit ratio, the more continuous the shadow width, the greater the embedment depth, and the more obvious the contour flattening or indentation, the higher the adhesion score assigned to that particle. If the particle boundary is clear, there is no continuous shadow around the boundary, no focal plane shift, and no indentation, the system records the particle as a low-adhesion particle. If a continuous contact zone appears between the particle edge and the substrate, and there is also an indentation or embedment depth at that location, the system records the particle as a high-adhesion particle.

[0124] The calibration benchmark for adhesion degree values ​​is established using three types of samples. Type 1 is a clean substrate sample, used to determine the background texture, background brightness, and normal edge fluctuations when no particles are attached. Type 2 is a sample with slight surface adhesion, used to determine the boundary shadow width, adhesion length, and contour state when particles remain on the substrate surface but are not embedded. Type 3 is an embedded or indented sample, used to determine the embedding depth, indentation width, boundary blurring band, and contour deformation range when particles form a stable bond with the substrate. The system acquires low-light images of the above samples and extracts four types of data using the same algorithm to form an adhesion degree calibration table. During detection, the four types of data for the current particle are compared with this calibration table to obtain the corresponding adhesion degree value.

[0125] The adhesion threshold is determined by the calibration table mentioned above. The system first calculates the adhesion value of samples with slight surface adhesion to obtain the adhesion range of cleanable particles; then it calculates the adhesion value of embedded or pressed-in samples to obtain the adhesion range of stably bound particles. If the upper limit of the cleanable particle adhesion range is lower than the lower limit of the stably bound particle adhesion range, the system takes the midpoint between the two as the adhesion threshold. If the two ranges overlap, the system, aiming to avoid missed detections, takes the 5th percentile of the stably bound particle adhesion range as the adhesion threshold. During detection, if the particle adhesion value is greater than this threshold, it indicates a high risk of adhesion between the particle and the substrate.

[0126] When processing the adhesion values ​​between foreign matter and the substrate to obtain the fourth feature set, the system summarizes them according to abnormal regions. The fourth feature set includes: maximum adhesion value, average adhesion value, number of particles with adhesion values ​​greater than the adhesion threshold, percentage of particles with adhesion values ​​greater than the adhesion threshold, maximum embedding depth, average embedding depth, area of ​​continuous adhesion region, coordinates of the main risk particle, foreign matter distribution density of the grid where the main risk particle is located, and distance between the main risk particle and the nearest pad or metal trace. If multiple highly adhered particles exist in the same abnormal region, the system selects the particle with the highest adhesion value as the main risk particle, while retaining the number and location of other highly adhered particles.

[0127] The fourth feature set is not directly compared to a preset risk threshold; instead, it is first converted into an attached risk output value. The system reads each data item in the fourth feature set and compares it with the corresponding upper limit of normal risk and lower limit of high risk. The upper limit of normal risk comes from the statistical results of qualified samples, while the lower limit of high risk comes from embedded samples, pushed-in samples, historical failure samples, or product contamination control specifications. Data below the upper limit of normal risk receives a low-risk score; data between the upper limit of normal risk and the lower limit of high risk receives an intermediate-risk score; and data reaching or exceeding the lower limit of high risk receives a high-risk score. Subsequently, the system synthesizes the various risk scores according to their weights to form the attached risk output value for that abnormal region.

[0128] Weights are determined from historical samples. The system reads the relationship between each feature in the historical samples and the actual failure results, and statistically analyzes the contribution of maximum adhesion degree, continuous adhesion area, distance between the main risk particle and the functional structure, proportion of adhered particles, and maximum embedding depth to short circuits, increased thermal resistance, contamination propagation, and encapsulation failure. Features that contribute significantly to failure are assigned high weights, while features that only cause slight surface contamination and do not induce thermal anomalies are assigned low weights. After weight configuration, the second feature set is converted into an adhesion risk output value, which is used to compare with a preset risk threshold.

[0129] The preset risk threshold is determined during the model building phase. The system selects qualified samples, reworkable clean samples, and high-risk defect samples as threshold calibration data. For each sample, the third and fourth feature sets are extracted according to the same process, and the attachment risk output value is calculated. The system sorts the attachment risk output values ​​of qualified samples from smallest to largest, and takes the 95th percentile as the upper limit of qualified risk; it sorts the attachment risk output values ​​of high-risk defect samples from smallest to largest, and takes the 5th percentile as the lower limit of high risk. If the upper limit of qualified risk is lower than the lower limit of high risk, the system takes the midpoint between the two as the preset risk threshold. If there is overlap, the system determines the threshold according to the detection strategy; when the goal is to avoid missed detections, the 5th percentile of high-risk defect samples is used as the preset risk threshold; when the goal is to reduce false alarms, the 95th percentile of qualified samples is used as the preset risk threshold. The final threshold is written into the detection system after verification with validation samples.

[0130] If the fourth feature set is greater than the preset risk threshold, it indicates that the attachment risk output value obtained by converting the fourth feature set is greater than the preset risk threshold. When this condition is met, the system identifies the abnormal region as a high-risk defect region. If multiple primary risk particle candidates exist within an abnormal region, the system uses the maximum attachment risk output value as the basis for determining the region's high risk; the average attachment risk output value is used for subsequent risk level classification. If the attachment risk output value is not greater than the preset risk threshold, the system marks the abnormal region as a low-risk abnormal region or an abnormal region awaiting review, and does not output it as a high-risk defect region.

[0131] After identifying high-risk defect areas, the system extracts their specific spatial coordinates. The system first reads the coordinates of the closed contour, the coordinates of the circumscribed rectangle, the coordinates of the region center, and the coordinates of the center of the primary risk particle in the low-light image. Then, the system calls the preceding coordinate mapping relationship to convert these coordinates to the visible light surface map coordinate system, the infrared thermal distribution map coordinate system, and the chip physical coordinate system, respectively. The chip physical coordinate system uses the chip's upper left corner positioning mark, center positioning mark, or the reference point specified in the process document as its origin. The output coordinates include the region center coordinates, the coordinates of the upper left corner of the circumscribed rectangle, the coordinates of the lower right corner of the circumscribed rectangle, the closed contour coordinates, the coordinates of the primary risk particle, the coordinates of the smallest enclosing region, and the distance of this region from the nearest pad, metal trace, chip edge, or heat-sensitive area.

[0132] Risk level classification is based on the attached risk output value, foreign object distribution density, distance from functional structure, and preliminary defect classification label. The system classifies risk levels into three levels. Level 1 is a high-risk review area, indicating that the attached risk output value just exceeds the preset risk threshold, the foreign object distribution density does not reach the high-density risk range, and the main risk particle is not close to the functional structure. Level 2 is a high-risk rework area, indicating that the attached risk output value significantly exceeds the preset risk threshold, the foreign object distribution density exceeds the normal density threshold, or the main risk particle enters the functional structure warning distance. Level 3 is a high-risk failure area, indicating that the attached risk output value enters the risk range of historical failure samples, or the foreign object particle comes into contact with the pad, metal trace, chip edge, or heat-sensitive area, or the maximum embedding depth reaches the high-risk lower limit of the stable-bonded particle.

[0133] The risk level threshold is determined by the risk distribution of historical samples. The system calculates the adhesion risk output value for high-risk defect samples, reworked qualified samples, and confirmed failed samples separately. The upper limit of risk for reworked qualified samples serves as the reference boundary between Level 1 and Level 2; the lower limit of risk for confirmed failed samples serves as the reference boundary between Level 2 and Level 3. If there is overlap in sample distribution, the system makes corrections based on the functional structure distance. When the distance between the main risk particle and the pad or metal trace is less than the process warning distance, the risk level is increased by 1 level; if both high foreign matter distribution density and high adhesion degree exist simultaneously, the risk level is increased by another 1 level; the highest level is Level 3.

[0134] The process warning distance is determined by the chip layout and product testing specifications. The system reads the minimum functional structure spacing between adjacent pads, metal traces, chip edges, or heat-sensitive areas in the chip, and takes 10% of this minimum functional structure spacing as the process warning distance. If the product testing specifications already specify the minimum safe distance between foreign particles and functional structures, then that specification value is directly used as the process warning distance. When the distance from the primary risk particle to the functional structure is less than the process warning distance, the system considers that the particle has posed a direct risk to the functional structure, and the risk level is increased accordingly.

[0135] Finally, the system outputs the specific spatial coordinates and risk level classification of high-risk defect areas. The output includes the high-risk defect area number, preliminary defect classification label, third feature set, fourth feature set, attached risk output value, preset risk threshold, risk level, area center coordinates, circumscribed rectangle coordinates, closed contour coordinates, principal risk particle coordinates, nearest functional structure name and distance, density exceeding limit grid number, and corresponding image evidence. This output is used for subsequent local enhancement processing of infrared and visible light images of high-risk defect areas, and provides spatial location and risk level criteria for final defect confirmation.

[0136] S6 includes acquiring an initial infrared thermal distribution map and an initial visible light surface map of the high-risk defect area; using phase-locked amplitude and phase contrast enhancement on the initial infrared thermal distribution map to obtain a detailed magnified infrared image; if the contrast of the detailed magnified infrared image is lower than a preset threshold, histogram equalization is used to obtain a secondary detailed magnified infrared image; using local contrast and edge enhancement on the initial visible light surface map to obtain a feature-enhanced visible light image; if the edge sharpness of the feature-enhanced visible light image is lower than a preset threshold, a Laplacian operator is used to obtain a secondary feature-enhanced visible light image; and fusing the secondary detailed magnified infrared image and the secondary feature-enhanced visible light image to generate a fused high-resolution defect image.

[0137] In this embodiment, the system first acquires the initial infrared thermal distribution map and the initial visible light surface map of the high-risk defect region. Specifically, the system reads the output coordinate information of the high-risk defect region, which includes at least the coordinates of the region center, the closed contour, the circumscribed rectangle, and the main risk particle. Subsequently, the system calls the previous image registration results to map the high-risk defect region into the infrared thermal distribution map coordinate system and the visible light surface map coordinate system, respectively. To ensure that the defect boundary, thermal diffusion boundary, and surrounding reference background simultaneously enter the enhancement processing range, the system adds image cropping margins in the four directions of the circumscribed rectangle. These cropping margins are jointly determined by the average image registration error, the thermal diffusion radius, and the neighborhood width required for edge enhancement. The system first reads the average reprojection error from the previous registration step, then reads the minimum thermal diffusion radius corresponding to the current chip material and packaging structure, and reads the minimum neighborhood width of the local enhancement algorithm. These three values ​​are converted to the current image pixel scale and then added together to obtain the cropping margin. After cropping the image according to the cropping allowance, the initial infrared thermal distribution map and the initial visible light surface map corresponding to the same high-risk defect area are obtained.

[0138] The initial infrared thermal distribution map is not obtained by directly reading a single thermal image, but rather by combining a phase-locked loop (PLL) amplitude map and a PLL phase map. The system first reads the region data corresponding to the high-risk defect area in the PLL amplitude map, then reads the region data at the same coordinate position in the PLL phase map, and adjusts both to have consistent image size and pixel coordinates. If there is a deviation of less than one pixel in pixel arrangement between the infrared thermal image and the phase map, the system performs resampling correction based on the PLL amplitude map, ensuring that the same defect location corresponds to the same pixel position in both types of images. After this processing, the infrared enhancement step can simultaneously access the amplitude and phase information of the same defect area.

[0139] When using phase-locked amplitude and phase contrast enhancement for the initial infrared thermal distribution map, the system first performs phase-locked amplitude enhancement. Specifically, the system reads the phase-locked amplitude of pixels within the defect region and the phase-locked amplitude from the normal reference band outside the defect region. The normal reference band extends outward from the outer boundary of the defect region, with its width consistent with the aforementioned trimming allowance, and must not cross into other defect regions. The system sorts the amplitude data in the normal reference band, deleting the highest 5% and lowest 5% of abnormally high values, then uses the minimum value of the remaining data as the lower limit for amplitude enhancement, and the 95th percentile of the merged data from the defect region and the reference band as the upper limit for amplitude enhancement. The system remaps the amplitude data between these two limits to the full grayscale display range, widening the thermal intensity difference between the normal reference region and the defect thermal response region, thus obtaining the amplitude enhancement result. After this step, the edges of local hot spots, the thermal diffusion path, and the center of thermal response intensity are clearly displayed.

[0140] The system then performs phase contrast enhancement. During processing, the system reads the phase value of each pixel within the defect area point by point, and simultaneously reads the phase values ​​of the adjacent pixels to its left, right, top, and bottom, calculating the phase change between the current pixel and its neighbors. For pixels located near crack endpoints, foreign object boundaries, and thermal diffusion inflection points, if their phase change exceeds the upper limit of the phase change in the normal reference band, the system marks the pixel as a pixel with significant phase contrast. The upper limit of the phase change in the normal reference band is determined by calibration samples: the system first acquires phase-locked phase images of no less than 10 qualified chips under the same operating conditions, selects defect-free areas to calculate the phase change between adjacent pixels, deletes the highest 5% and lowest 5% of discrete values, and takes the 95th percentile as the upper limit of the normal phase change. In the enhancement process, the system increases the display weight of pixels with significant phase contrast, while maintaining the original display level for background pixels with insignificant phase changes, thus forming a clear phase contrast structure in the infrared image for thermal diffusion boundaries, thermal resistance change locations, and defect propagation directions.

[0141] After amplitude enhancement and phase contrast enhancement, the system synthesizes them into a detailed magnified infrared image. During synthesis, the system uses the amplitude enhancement result as the thermal intensity base layer and the phase contrast enhancement result as the thermal boundary correction layer. If a pixel is simultaneously in a high amplitude region and a high phase change region, the system retains it as a key display pixel; if a pixel has a high amplitude but insignificant phase change, the system retains its hotspot intensity but reduces its boundary weight; if a pixel has a low amplitude but significant phase change, the system retains its boundary information to highlight potential propagation paths. Through this processing, the detailed magnified infrared image simultaneously contains thermal intensity information and thermal diffusion structure information.

[0142] After generating the magnified infrared image, the system determines its contrast. This contrast is not based on subjective observation, but rather calculated based on the grayscale difference between the defect area and the background area. Specifically, the system first reads the pixel grayscale values ​​within the defect area of ​​the magnified infrared image, and then reads the background grayscale values ​​from the outer normal reference band. The system calculates the average grayscale value within the defect area and the average grayscale value of the background area, and then calculates the difference between them. Simultaneously, the system also reads the average of the highest 10% grayscale values ​​within the defect area and the average of the lowest 10% grayscale values ​​in the background area, and calculates the difference between them. These two differences are used together as the basis for determining the contrast of the infrared image. If the average grayscale difference and the high-low grayscale difference between the defect area and the background area do not reach the preset contrast threshold, the system determines that the contrast of the magnified infrared image is below the preset threshold.

[0143] The preset threshold for infrared image contrast is determined before detection. The system selects no fewer than 10 known defect samples and no fewer than 10 qualified samples, and performs the same phase-locked amplitude enhancement and phase contrast enhancement on each to generate a magnified infrared image of the sample details. For defect samples, the system calculates the average grayscale difference and the difference between high and low grayscale values ​​between the defect area and the background area to obtain the effective display range of the defect; for qualified samples, the system calculates the corresponding difference between the normal area and the background area to obtain the non-defect display range. The system selects a critical value that can distinguish between the two types of samples as the preset threshold for infrared image contrast. If there is partial overlap between the two types of samples, the system prioritizes not missing any defects and sets the threshold at the 5th percentile of the defect sample display range. This threshold is written into the detection parameter table and used to uniformly determine whether the infrared image enhancement is sufficient.

[0144] When the contrast of the magnified infrared image is lower than a preset threshold, the system uses histogram equalization to obtain a secondary magnified infrared image. Specifically, the system counts the number of pixels corresponding to each gray level in the magnified infrared image, accumulates the pixel distribution from low to high gray level, and then redistributes gray values ​​based on the accumulated distribution, expanding pixels originally concentrated in a narrow gray range to a wider gray range. For images with concentrated gray levels in defect areas and insufficient gray level variation in the background, this processing directly widens the brightness levels between the hot spot core, hot boundary, and background. To prevent histogram equalization from amplifying background noise, the system only performs equalization on valid pixels within the initial infrared cropping area and does not include the image's outer padding values ​​in the statistics. After processing, a secondary magnified infrared image is obtained. If the contrast of the magnified infrared image reaches or exceeds the preset threshold, the system directly outputs the magnified infrared image as the secondary magnified infrared image to maintain consistency in subsequent processing.

[0145] When applying local contrast and edge enhancement to the initial visible light surface image, the system first performs local contrast enhancement. Specifically, the system divides the initial visible light surface image into multiple local sub-regions. The size of each sub-region is determined by the minimum feature width of the defect and the resolution of the microscopic image. The system first reads the minimum identifiable crack width and the minimum identifiable foreign object edge width from the product inspection specifications. Then, based on the actual length corresponding to a single pixel in the visible light image, it converts these feature widths into pixel counts, using twice these pixel counts as the side length of the local sub-region. Subsequently, the system statistically analyzes the brightest, darkest, and intermediate grayscale distributions within each local sub-region and re-stretches the grayscale within that sub-region, resulting in clearer grayscale levels for dark cracks, bright reflective boundaries, surface indentations, and particle edges. By processing all local sub-regions one by one, surface details are not compressed into a low-contrast range due to uneven overall illumination.

[0146] After local contrast enhancement, the system performs edge enhancement. During edge enhancement, the system reads the brightness difference between the current pixel and its surrounding neighboring pixels point by point, identifies locations where there are significant abrupt changes in brightness, and uses these locations as candidate edge pixels. The system then continuously tracks the candidate edge pixels along the crack direction, particle contour direction, indentation boundary direction, and surface texture direction, connecting the continuously distributed candidate edges into structural edges. For edges with continuous boundaries, stable directions, and lengths reaching the minimum effective edge length threshold, the system increases their display weight; for isolated noise edges, short reflective edges, and discontinuous edges, the system decreases their display weight. The minimum effective edge length threshold is calculated from the minimum effective crack length and minimum effective particle boundary length in the product inspection specifications and corrected by the microscopic system pixel calibration. Through this processing, the crack contour, foreign object contour, indentation boundary, and surface microstructure boundary in the initial visible light surface image are enhanced, forming a feature-enhanced visible light image.

[0147] After generating the feature-enhanced visible light image, the system determines its edge sharpness. Edge sharpness is measured by both the intensity of brightness transitions at the edge locations and the edge transition width. Specifically, the system first extracts crack boundaries, particle contour boundaries, and indentation boundaries from the feature-enhanced visible light image, and then reads the brightness change curves along the boundary normal direction. For each boundary, the system records the pixel width experienced by the brightness transition from one side of the boundary to the other, as well as the maximum brightness change during this transition. The narrower the edge transition width and the greater the maximum brightness change, the higher the edge sharpness. The system calculates the average edge sharpness of all valid boundaries as the overall edge sharpness value of the feature-enhanced visible light image. If this overall edge sharpness value is lower than a preset threshold, the system determines that the edge sharpness of the feature-enhanced visible light image is lower than the preset threshold.

[0148] The preset threshold for edge sharpness is calibrated before detection. The system selects no fewer than 10 sample images known to contain cracks, foreign particles, and indentation boundaries, and no fewer than 10 qualified sample images, and uniformly performs local contrast enhancement and edge strengthening processing. Subsequently, the system extracts the edge sharpness of real defect boundaries from defective samples and the edge sharpness of normal texture boundaries from qualified samples. The system selects a critical value that can distinguish between real defect boundaries and ordinary surface texture boundaries as the preset threshold for edge sharpness. If the sharpness distributions of the two types of samples partially overlap, the system sets the preset threshold at the 5th percentile of the edge sharpness distribution of the defective samples, based on the principle that defect boundaries must be stably displayed. This threshold is written into the detection parameter table and used for subsequent edge sharpness judgment.

[0149] When the edge sharpness of the enhanced visible light image is below a preset threshold, the system uses the Laplacian operator to obtain a secondary enhanced visible light image. Specifically, the system reads the brightness value of each pixel in the enhanced visible light image from its surrounding neighborhood and compares the brightness changes of the current pixel with those of its neighbors. For locations with abrupt brightness changes, the Laplacian operator enhances the second-order transformation at that location, making crack edges, foreign object edges, and indentation boundaries clearer. The system then superimposes the Laplacian enhancement result onto the original enhanced visible light image, preserving the surface texture after local contrast enhancement while highlighting the sharpness of the edges. After processing, a secondary enhanced visible light image is obtained. If the edge sharpness of the enhanced visible light image reaches or exceeds the preset threshold, the system directly outputs the enhanced visible light image as the secondary enhanced visible light image.

[0150] After obtaining a secondary detailed magnified infrared image and a secondary feature-enhanced visible light image, the system performs a fusion process to generate a fused high-resolution defect image. Before fusion, the system performs size consistency and pixel alignment checks on the two images. Since both images originate from the same high-risk defect region and have already undergone prior registration and mapping, the system only needs to check for cropping boundary deviations and interpolation deviations. If there is an offset of less than one pixel between the boundaries of the two images, the system resamples and corrects the infrared image based on the visible light image, ensuring that the defect boundaries, particle positions, and thermal anomaly centers fall at uniform coordinate positions.

[0151] The fusion process employs a layered overlay of structural and thermal response information. The system uses the enhanced visible light image as the structural layer, preserving crack contours, particle edges, indentation boundaries, and surface textures; and the magnified infrared image as the thermal information layer, preserving hot spot cores, thermal diffusion boundaries, and areas with significant phase contrast. During fusion, if a pixel is located simultaneously in both the visible light structural boundary and the infrared thermal anomaly region, the system marks it as a key fusion pixel and retains both types of information. If a pixel only has a visible light boundary without significant thermal anomalies, the system primarily displays the structural information; if a pixel only has thermal anomalies without significant surface boundaries, the system primarily displays the infrared thermal information. For defect core regions, the system prioritizes displaying the overlap between the visible light contour and the infrared thermal center; for defect boundary regions, the system prioritizes displaying the spatial relationship between the crack propagation direction and the thermal diffusion direction.

[0152] To ensure high-resolution display of the fused result, the system uses the visible light image resolution as the output resolution benchmark and interpolates and maps the thermal information in the infrared thermal image onto the corresponding visible light pixel grid. After mapping, each output pixel simultaneously possesses structural display attributes and thermal information display attributes. The system then performs a boundary smoothing and information consistency check on the fused result: if the position of a hot spot deviates from the corresponding structural boundary by more than the upper limit of the registration error, the system corrects the position of the hot layer according to the upper limit of the registration error; if there is no infrared thermal information near a structural boundary, the structural boundary is output as is, and a hot layer is not forcibly superimposed. After the above processing, a fused high-resolution defect image is generated.

[0153] The fused high-resolution defect image simultaneously displays surface morphology and thermal anomaly information of high-risk defect regions. The output includes at least the defect region outline, crack boundary, foreign object boundary, indentation boundary, hot spot core location, thermal diffusion boundary, and spatial correspondence with the main risk particle or main crack. This image serves as the direct input for subsequent multimodal feature extraction, defect pattern matching, and final defect confirmation.

[0154] S7 includes acquiring high-resolution defect images, extracting multimodal feature information from the high-resolution defect images to obtain an initial multimodal feature set; extracting the phase-locked thermal response time constant and crack morphology feature values ​​based on the initial multimodal feature set to obtain a multidimensional physical feature vector; extracting optical reflection characteristic data based on the multidimensional physical feature vector to determine a comprehensive defect feature matrix; using the comprehensive defect feature matrix for matching judgment; if the matching degree of the comprehensive defect feature matrix is ​​greater than a preset threshold, then outputting the final defect confirmation result and classification label based on the matching degree.

[0155] In this embodiment, the system first acquires a fused high-resolution defect image. This image is obtained by fusing a secondary detail-enhanced infrared image and a secondary feature-enhanced visible light image from the previous steps. The image simultaneously contains thermal anomaly information, phase contrast information, crack contour information, foreign object particle contour information, indentation boundary information, and surface texture information. The system reads the pixel coordinates, defect region contour, hot spot core location, thermal diffusion boundary, crack boundary, foreign object boundary, and main risk region coordinates of this high-resolution defect image, and uses this data as input for final defect confirmation.

[0156] When extracting multimodal feature information from high-resolution defect images, the system processes information according to three categories: thermal response features, structural morphology features, and optical surface features. For thermal response features, the system reads the hotspot center, thermal anomaly area, thermal diffusion boundary, phase-locked amplitude enhancement region, and phase contrast enhancement region corresponding to the infrared thermal layer in the fused image. For structural morphology features, the system reads the crack centerline, crack boundary, crack endpoint, crack bifurcation point, foreign particle outline, and indentation boundary in the visible light structural layer. For optical surface features, the system reads color changes, brightness changes, reflection intensity changes, and boundary shadow changes within the defect area. All three types of features are merged according to the same defect region number to form an initial multimodal feature set.

[0157] The initial multimodal feature set is generated on a per-defect-region basis. The system first reads the closed defect contours from the high-resolution defect image and uses the interior of the contours as the feature statistical region. If multiple independent defect contours exist in the same image, the system numbers them according to their connectivity and distance relationships. For each defect region, the system sequentially writes the hotspot center coordinates, thermal anomaly area, area of ​​the phase contrast significant region, number of cracks, crack length, crack width, number of crack branches, number of foreign matter particles, foreign matter particle size, foreign matter embedding location, area of ​​the color change region, area of ​​the reflection anomaly region, and the distance between the defect region and the functional structure. The resulting initial multimodal feature set can simultaneously express the thermal response, geometric morphology, and optical performance of the defect.

[0158] When extracting the phase-locked infrared (PLI) thermal response time constant based on the initial multimodal feature set, the system traces back the original time series of the defect region during the PLI acquisition process. Specifically, the system uses the coordinates of the defect region in the high-resolution defect image and calls the pre-registration relationship to map these coordinates into the PLI frame sequence. The system reads the thermal response values ​​of each pixel within the defect region at multiple sampling times and arranges them according to the excitation cycle. Subsequently, the system determines the time elapsed from the start of excitation to the stable rise of the thermal response, and the time elapsed from the weakening of excitation to the stable decay of the thermal response. The system uses the rate of change of the thermal response during the rise and decay processes as the basis for calculating the time constant. The time corresponding to the thermal response reaching 63% of the stable response amplitude is recorded as the heating time constant; the time corresponding to the thermal response decaying from the peak to 37% of the peak is recorded as the cooling time constant. If multiple pixels exist within the same defect region, the system first calculates the time constant for each pixel separately, and then takes the average, maximum, and range of variation within the core defect region as the PLI thermal response time constant data for that defect region.

[0159] The validity of the time constant requires the elimination of abnormal sampling points. The system checks the thermal response curve of each pixel within the same defective region. If the thermal response curve of a pixel exhibits abrupt changes, breaks, or inconsistent changes with adjacent pixels, the system removes that pixel from the time constant statistics. The removal threshold is determined by the normal thermal response fluctuations of qualified samples. The system collects thermal response curves of no fewer than 10 qualified chips under the same excitation conditions, statistically analyzes the normal variation amplitude between adjacent sampling points, and takes the 95th percentile as the upper limit of normal fluctuation; when the variation amplitude of adjacent sampling points of the current pixel exceeds this upper limit, the sampling point is marked as an abnormal sampling point. Through this process, the phase-locked loop thermal response time constant is obtained from a stable thermal response curve.

[0160] Crack morphology features are extracted from the crack profile and crack centerline in the initial multimodal feature set. The system reads the start point, end point, centerline, left and right boundaries, and branch structure of each crack, and calculates the crack length, maximum width, average width, width fluctuation range, number of branches, highest branch level, crack curvature, crack endpoint sharpness, and distance between the crack and the pad or metal trace. The crack length is obtained from the cumulative path length of the centerline from the start point to the end point; the crack width is obtained from the distance between the left and right boundaries perpendicular to the centerline; the crack curvature is determined by the difference between the actual path length of the crack centerline and the straight-line distance from the start point to the end point; the crack endpoint sharpness is determined by the convergence angle of the crack end boundary and the range of gray-scale abrupt changes at the end. The system writes the above values ​​into the crack morphology feature data.

[0161] The effective threshold for crack morphology features is jointly determined by product process specifications, statistical results of qualified samples, and results of failed samples. The system first reads the limits on crack length, crack width, and distance from the crack to the functional structure from the product process specifications; then, it collects no fewer than 10 qualified samples, statistically analyzing the normal surface texture and the length, width, number of branches, and degree of curvature of permissible microcracks; subsequently, it reads crack morphology data from historical failed samples that led to crack propagation, abnormal electrical performance, or increased thermal resistance. For width, length, and number of branches, the system uses the 95th percentile of qualified samples and the strictest value in the process specification limits as the normal upper limit; for values ​​in failed samples that have clearly caused failure, the system uses them as the high-risk lower limit. When the current crack morphology feature exceeds the normal upper limit, the system marks the feature as morphologically abnormal; when it reaches the high-risk lower limit, the system marks the feature as a high-risk morphology feature.

[0162] After obtaining the time constant of the lock-in thermal response and the numerical values ​​of the crack morphology characteristics, the system constructs a multidimensional physical feature vector. This multidimensional physical feature vector is generated on a per-defect-region basis, with each defect-region corresponding to one set of vector data. The vector data includes the heating time constant, cooling time constant, time constant variation range, thermal anomaly area, area of ​​significant phase contrast, maximum crack width, average crack width, crack length, number of crack branches, highest branch level, crack tip sharpness, maximum foreign object size, foreign object embedding depth, foreign object distribution density, and distance from the defect-region to the nearest functional structure. For defect-regions without cracks, the crack-related values ​​are recorded as 0; for defect-regions without foreign objects, the foreign object-related values ​​are recorded as 0; for defect-regions containing both cracks and foreign objects, all corresponding values ​​are written into the same set of vector data.

[0163] When extracting optical reflection characteristic data based on multidimensional physical feature vectors, the system returns to the visible light structure layer and color channel data in the high-resolution defect image. The system reads the brightness values, color channel values, and reflection intensity changes within the defect region, at the defect boundary, and in the normal reference region outside the defect. The normal reference region is formed by extending outward from the defect boundary, with its width determined by 20% of the width of the outer rectangle of the defect region, avoiding other defect regions. The system removes the highest 5% of highly reflective pixels and the lowest 5% of shadow pixels from the normal reference region, and then calculates the reference brightness and reference color. Subsequently, the system compares the brightness differences, color differences, and reflection differences between pixels within the defect region and the reference region point by point to obtain the optical reflection characteristic data.

[0164] Optical reflection characteristic data includes the mean reflection intensity, maximum reflection intensity, reflection non-uniformity, color shift, brightness gradient, boundary shadow width, and the proportion of high-reflection areas. For metallic foreign objects, the system focuses on recording high reflection intensity, strong reflective boundaries, and brightness abrupt changes; for non-metallic foreign objects, the system focuses on recording low-reflection areas, diffuse reflection areas, and color shift; for crack defects, the system focuses on recording dark line boundaries, shadow continuity, and brightness differences on both sides of the crack. The optical reflection threshold is jointly determined by the normal reference area and calibration samples. The system first obtains the upper limit of normal reflection fluctuation from the normal reference area of ​​the current image, and then calls the calibration reflection data of metallic particles, non-metallic particles, contaminants, and crack samples. When the reflection intensity, color shift, or shadow width of the current defect area exceeds the upper limit of normal reflection fluctuation and is consistent with the reflection range of a certain type of calibration sample, the system writes it as a valid optical reflection feature into the comprehensive defect feature matrix.

[0165] The comprehensive defect feature matrix is ​​formed by combining multi-dimensional physical feature vectors and optical reflection characteristic data. The matrix is ​​organized by defect region number as rows and various features as columns. Each row sequentially contains the lock-in thermal response time constant, thermal anomaly area, phase contrast area, crack length, crack width, number of crack branches, crack tip sharpness, foreign object size, foreign object embedding depth, foreign object distribution density, reflection intensity, color shift, boundary shadow width, reflection non-uniformity, and distance from the defect to the functional structure. The system standardizes each feature before writing it into the matrix. The standardization benchmark is established using qualified samples, known crack samples, known foreign object samples, thermal anomaly samples, and composite defect samples. Each feature is converted to a uniform feature level according to its normal upper limit and defect sample risk lower limit, enabling data of different dimensions to participate in the same matching judgment.

[0166] When using a comprehensive defect feature matrix for matching and judgment, the system compares the matrix data of the current defect region with a preset defect pattern library. The defect pattern library is established before detection and includes at least crack-induced thermal anomaly patterns, foreign particle-induced thermal anomaly patterns, foreign object embedding thermal anomaly patterns, localized overheating defect patterns, surface contamination defect patterns, and composite defect patterns. Each defect pattern includes a corresponding range of thermal response time constant, phase contrast feature range, crack morphology range, foreign object size range, foreign object embedding range, and optical reflection characteristics range. The system compares the eigenvalues ​​in the current matrix with the feature ranges of each defect pattern, and counts the number of features falling within the corresponding pattern range, the consistency of key features, and the consistency of spatial location to obtain the matching degree for each defect pattern.

[0167] The matching degree is calculated with priority given to key features. For crack-induced thermal anomaly modes, crack length, crack width, number of crack branches, phase contrast at crack endpoints, and thermal response time constant are key features; for foreign object particle-induced thermal anomaly modes, foreign object size, foreign object embedding depth, foreign object reflection intensity, phase contrast at particle boundaries, and local thermal anomaly area are key features; for local overheating defect modes, thermal response time constant, thermal anomaly area, and phase contrast area are key features; for composite defect modes, crack features, foreign object features, and thermal response features are all considered key features. The system assigns high weight to key features and low weight to auxiliary features. The more key features of the current defect region fall within the range of a certain defect mode, and the more consistent their spatial relationships, the higher the matching degree of that mode.

[0168] The preset matching threshold is determined during the defect pattern library establishment phase. The system selects historical defect samples and qualified samples whose categories have been manually confirmed, generates a comprehensive defect feature matrix according to the same process, and calculates its matching degree with each defect pattern. For each defect pattern, the system sorts the matching degrees of samples confirmed to belong to the pattern from smallest to largest, and takes the 5th percentile as the lower limit of defect matching for that pattern; it sorts the matching degrees of samples not belonging to the pattern from smallest to largest, and takes the 95th percentile as the upper limit of non-pattern matching. When the lower limit of defect matching is higher than the upper limit of non-pattern matching, the system takes the median value between the two as the matching degree threshold for that pattern. When there is overlap, the system sets the threshold at the 5th percentile of the matching degree of defect samples for that pattern, aiming to avoid missed detections. The matching degree thresholds for all patterns are written into the detection system after verification with validation samples.

[0169] When the matching degree between the comprehensive defect feature matrix and a certain defect pattern is greater than the preset threshold corresponding to that pattern, the system confirms that the current defect region conforms to that defect pattern. If only one defect pattern has a matching degree greater than the corresponding threshold, the system directly outputs the final defect confirmation result and classification label corresponding to that pattern. If the matching degree of multiple defect patterns is greater than the corresponding threshold, the system compares the matching degree of each pattern and reads the contribution of key features. The pattern with the highest matching degree and the largest contribution of key features is used as the main classification label; other patterns exceeding the threshold are used as auxiliary labels. If both the crack pattern and the foreign matter pattern exceed the threshold, and thermal anomalies, crack structures, and foreign matter particles exist in the same defect region, the system outputs a composite defect as the main label, and uses crack-induced thermal anomalies and foreign matter particle-induced thermal anomalies as sub-labels.

[0170] When outputting the final defect confirmation result and classification label based on the matching degree, the system simultaneously outputs the confirmation status, classification label, matching degree value, defect pattern name exceeding the threshold, main contributing features, and defect spatial coordinates. The final defect confirmation result includes "confirmed defect" and "unconfirmed defect". Classification labels include crack-induced thermal anomaly, foreign particle-induced thermal anomaly, foreign object embedded thermal anomaly, local overheating defect, surface contamination defect, and composite defect. If the matching degree of all defect patterns is not greater than the corresponding preset threshold, the system outputs an unconfirmed defect and marks the area as an area awaiting manual review, while retaining the comprehensive defect feature matrix and fused image evidence. If the matching degree is greater than the preset threshold, the system outputs a confirmed defect and writes the corresponding classification label into the inspection report for subsequent quality judgment, failure analysis, and process traceability.

[0171] The present invention also discloses a machine-executable program that can be automatically executed by a machine to realize the chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light images as described above.

[0172] The machine (electronic device) mentioned above is, for example, a microcontroller, a single-board computer, a desktop computer, a laptop computer, a server, a programmable controller, or a field-programmable gate array.

[0173] Figure 2 This is a structural block diagram of the local terminal of an exemplary electronic device (machine) of the present invention; as shown... Figure 2 As shown, the electronic device of the present invention includes a processor 11, a memory 12 and a storage space 33 for storing a machine-executable program 14, the machine-executable program 14 being used to execute the above-described control logic.

[0174] Figure 3 This is a structural block diagram of the network end of an exemplary electronic device of the present invention; as shown below. Figure 3 As shown, the present invention also provides an electronic device (machine), which may include at least one processor 210, at least one memory 230 communicatively connected to the processor, and a communication bus 240 connecting different system components (including memory 230 and processor 210); wherein, memory 230 stores a machine executable program that can be executed by the processor, and processor 210 can execute the above-mentioned control logic by calling the machine executable program.

[0175] Communication bus 240 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0176] Electronic devices typically include a variety of computer system readable media, which can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0177] Memory 230 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 230 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the control logic described above.

[0178] A program / utility having a set (at least one) of program modules can be stored in memory 230. Such program modules include, but are not limited to, an operating system, one or more applications, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0179] Machine-executable programs for performing this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, C++, and Python, and may also include specialized engineering languages ​​such as R. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0180] The present invention also discloses a storage medium on which a machine-executable program as described above is stored.

[0181] The aforementioned storage medium may be any combination of one or more computer-readable media. Computer-readable media may be, for example, computer-readable signal media or computer-readable storage media. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium may be, for example, any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0182] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0183] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0184] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A chip defect detection method based on the registration and fusion of infrared thermal imaging and visible light imagery, characterized in that, include: S1. Based on the phase-locked infrared thermal imager, acquire phase-locked infrared thermal image data in the chip's working state. Use an image registration algorithm based on feature point matching to align the infrared thermal image data with the visible light image data in the coordinate system, and obtain the spatially aligned infrared thermal distribution map and visible light surface map. S2. Based on the phase map and amplitude map obtained by phase-locked array imaging, temperature anomaly regions are extracted from the spatially aligned infrared thermal distribution map. Local hot spot locations are identified by phase difference analysis of the phase-locked signal. Combined with color changes and microstructure morphology information in the visible light surface map, the spatial range of potential thermal anomaly sections is determined. S3. Based on the corresponding position of the potential thermal anomaly section in the visible light surface image, use a micro-light microscopy system to perform high-resolution scanning imaging, extract surface detail features, and identify whether there are abnormal crack structures or abnormal foreign particle sizes. S4. When abnormal crack structure or foreign particle abnormality is detected, establish a multimodal risk assessment model, calculate the comprehensive risk index of the abnormal area, and obtain the preliminary defect classification results. S5. Based on the preliminary defect classification results, combined with the foreign matter distribution density, the characteristic parameters of the foreign matter and the substrate bonding degree, and the preset risk threshold standard, determine the specific spatial coordinates and risk level classification of high-risk defect areas.

2. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 1, characterized in that: S1 includes: Obtain the first thermal image matrix and the first surface matrix; Corner features are extracted from the first thermal image matrix and the first surface matrix. If the response value of the corner feature is greater than a preset threshold, a matching point pair is determined based on the corner feature. The affine transformation matrix is ​​calculated based on the matching point pairs, and the first thermal image matrix is ​​processed by the affine transformation matrix to obtain the aligned thermal image matrix; Edge pixel sets are extracted from the aligned thermal image matrix and the first surface matrix. Spatial correction is performed based on the edge pixel sets and texture integrity constraints to obtain the spatially aligned infrared thermal distribution map and visible light surface map.

3. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 1, characterized in that: S2 includes: Temperature anomaly regions are obtained by threshold segmentation of infrared thermal distribution maps; Phase-locked signals are extracted from areas of abnormal temperature, and the phase difference of the phase-locked signals is calculated to determine the location of local hot spots. Color variation features and microstructure morphology features are extracted from visible light surface images containing local hot spots; If the gradient value of the color change feature and the depth value of the microstructure convex-concave morphology feature are greater than the preset threshold, then boundary fusion is performed in combination with the local hot spot location to determine the spatial range of the potential thermal anomaly segment.

4. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 1, characterized in that: S3 includes: Obtain the corresponding location coordinates and scan to obtain a high-resolution low-light image; Microscopic crack contours and foreign particle contours were separated from high-resolution low-light images. Microcrack width and crack hierarchy data are extracted from the microcrack profile, and foreign particle size data are calculated from the foreign particle profile. If the width of the microcrack exceeds the preset width threshold or the crack hierarchy structure data is abnormal, it is determined that there is an abnormal crack structure. If the size data of the foreign object particle is greater than the preset size threshold, it is determined that there is an abnormal foreign object particle size, thus realizing the identification of whether there is an abnormal crack structure or an abnormal foreign object particle size.

5. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 1, characterized in that: S4 includes: The first feature set is obtained by extracting the number of crack branches and the foreign object embedding depth from the image; Based on the first feature set, the abnormal region is located and the phase gradient of the abnormal region is extracted to generate the second feature set; A multimodal data matrix is ​​constructed by fusing stress distribution features extracted from the first feature set and the second feature set; A model is built using a multimodal data matrix to calculate the comprehensive risk index; If the comprehensive risk index is greater than the preset threshold, the preliminary defect classification result is obtained based on the comprehensive risk index.

6. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 1, characterized in that: S5 includes: Extract the foreign object distribution density values ​​from the preliminary defect classification data, and process the foreign object distribution density values ​​to obtain the third feature set; The adhesion degree values ​​between foreign objects and substrate are extracted based on the third feature set, and the fourth feature set is obtained by processing the adhesion degree values ​​between foreign objects and substrate. If the fourth feature set is greater than the preset risk threshold, then a high-risk defect region is determined; Extract the specific spatial coordinates of high-risk defect areas and classify their risk levels. Determine the specific spatial coordinates and risk level classification of high-risk defect areas.

7. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 1, characterized in that, This also includes S6, which involves performing local enhancement processing on the identified high-risk defect areas in both the infrared thermal distribution map and the visible light surface map. The infrared image uses phase-locked amplitude enhancement and phase contrast enhancement for detail magnification, while the visible light image uses local contrast enhancement and edge enhancement algorithms for feature enhancement, generating a fused high-resolution defect image. Specifically, this includes: Obtain the initial infrared thermal distribution map and initial visible light surface map of the high-risk defect area; For the initial infrared thermal distribution map, phase-locked amplitude and phase contrast enhancement is used to obtain a detailed magnified infrared image. If the contrast of the detailed magnified infrared image is lower than a preset threshold, histogram equalization is used to obtain a secondary detailed magnified infrared image.

8. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 7, characterized in that: S6 further includes: For the initial visible light surface image, local contrast and edge enhancement are used to obtain a feature-enhanced visible light image. If the edge sharpness of the feature-enhanced visible light image is lower than a preset threshold, the Laplacian operator is used to obtain a secondary feature-enhanced visible light image. The high-resolution defect image is generated by fusing the secondary detail magnified infrared image and the secondary feature enhanced visible light image.

9. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 7, characterized in that, It also includes S7, which extracts multimodal feature information from the fused high-resolution defect image, combines the lock-in thermal response time constant, crack morphology feature parameters, and optical reflection characteristic data related to the foreign material, performs defect pattern matching judgment, and outputs the final defect confirmation result and classification label, specifically including: Acquire high-resolution defect images, extract multimodal feature information from the high-resolution defect images, and obtain an initial multimodal feature set; Based on the initial multimodal feature set, the phase-locked thermal response time constant and crack morphology feature values ​​are extracted to obtain a multidimensional physical feature vector.

10. The chip defect detection method based on infrared thermal imaging and visible light image registration and fusion according to claim 9, characterized in that: The S7 also includes: Optical reflection characteristic data are extracted based on multidimensional physical feature vectors to determine the comprehensive defect feature matrix; The comprehensive defect feature matrix is ​​used for matching and judgment. If the matching degree of the comprehensive defect feature matrix is ​​greater than the preset threshold, the final defect confirmation result and classification label are output according to the matching degree.

Images