Bipolar residual candidate region generation and statistical screening method for chip surface defects
By generating positive and negative residual maps and combining regional statistical features for screening and polarity determination, the problem of high recall and false alarm of micro-defects in automatic optical inspection of chips and semiconductor devices is solved, and stable candidate region acquisition and polarity discrimination are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 张议文
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-05
AI Technical Summary
In the automated optical inspection of chips and semiconductor devices, existing technologies struggle to reliably obtain high-recall candidate regions for micro-defects without relying on large-scale frame-level defect samples. Furthermore, template difference methods are susceptible to interference leading to false alarms and have difficulty distinguishing anomalies of opposite polarity.
By generating positive and negative residual maps, candidate defect regions are constructed, and regional statistical features are combined for screening and polarity determination. A screening model or rule base is used for secondary discrimination, and the defect location, boundary and polarity are output.
It achieves stable acquisition of micro-defect candidate regions with high recall without relying on large-scale bounding box annotation, reduces false alarms, and provides polarity information to facilitate subsequent analysis and process traceability.
Smart Images
Figure CN122156806A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automated optical inspection of chip and semiconductor device surfaces, and particularly to a method for generating and statistically screening bipolar residual candidate regions for chip surface defects. Background Technology
[0002] In automated optical inspection of chips and semiconductor devices, the targets under inspection typically exhibit characteristics such as high-density repeating patterns, narrow linewidths, tiny pads, and strong edge contrast. Actual defects manifest as open circuits, short circuits, missing components, adhesion defects, edge chipping, localized contamination, pin deformation, positional misalignment, or casing damage. These anomalies are often small in size and are easily affected by uneven illumination, reflections, focal plane drift, and tooling deviations.
[0003] While existing whole-image object detection or semantic segmentation methods have the advantage of end-to-end training, they often require the recollection of a large number of bounding box samples or pixel-level annotations when switching chip objects, process layers, or imaging conditions. For minute defects, whole-image scaling may also compress key structural details, causing small defects to be smoothed out in the backbone network.
[0004] Another approach uses template comparison or grayscale difference to generate suspected regions first. The advantages of this method are high recall, fast deployment, and it is more suitable for scenarios with few samples. However, if relying solely on a single difference map and a simple threshold, it is often affected by edge misalignment, brightness fluctuations, and local reflections, leading to a high number of false alarms. More importantly, a single difference result is insufficient to distinguish between anomalies of opposite polarity: "excess material" and "missing material."
[0005] Therefore, it is necessary to propose an automatic optical defect detection method for chip surfaces that retains the high recall characteristics of template methods while enabling structured screening and polarity determination at the candidate region level. Summary of the Invention
[0006] Purpose of the invention: The purpose of this invention is to provide a method for generating and statistically screening bipolar residual candidate regions for chip surface defects, aiming to solve the following problems: how to stably obtain high-recall candidate regions for micro-defects on the chip surface without relying on large-scale frame-level defect samples, and further suppress false alarms caused by template differentiation, while providing polarity information of the candidate regions for subsequent review, statistical analysis and process traceability.
[0007] Technical solution: A method for generating and statistically screening bipolar residual candidate regions for chip surface defects, including the following steps: S1. Obtain at least one qualified sample image as template image T from the same model chip or semiconductor device with the same package under the same imaging magnification, the same exposure parameters, the same lighting arrangement and the same tooling posture, and obtain the image to be inspected I. S2. Perform grayscale normalization, flat field correction and geometric registration on the image to be inspected I to obtain the registered image Ir corresponding to the template image T; S3. Generate template structure map Bt and structure map Bi to be inspected from template image T and registration image Ir respectively, and construct positive residual map R+ and negative residual map R-, where R+(x,y)=max(Ir(x,y)-T(x,y),0), R-(x,y)=max(T(x,y)-Ir(x,y),0); S4. Perform threshold segmentation, morphological processing, and connected component labeling on the structural difference results of the positive residual map R+, negative residual map R-, template structure map Bt, and the structure map Bi to be inspected, to obtain a set of candidate defect regions. S5. Extract at least the following regional statistical features for each candidate defect region: region width, height, area, aspect ratio, fill rate, mean gray-level difference, maximum gray-level difference, integral gray-level difference, number of positive residual pixels, number of negative residual pixels, template structure ratio, boundary distance, and polarity balance ratio. S6. Input the statistical features of the region into the filtering model or rule base for secondary discrimination, and output the defect location, defect bounding box, defect polarity and defect judgment result.
[0008] Furthermore, the template image T is obtained by directly using a single qualified sample image as a reference template, or by using multiple qualified sample images of the same type through pixel-level median fusion, average fusion, or majority voting fusion.
[0009] Furthermore, the grayscale normalization and flat field correction include estimating the background illumination field based on normal samples from the same batch and performing brightness equalization on the image to be inspected, thereby reducing the impact of uneven illumination, vignetting, and local reflections on the difference results.
[0010] Furthermore, the geometric registration includes coarse registration based on phase correlation or template matching, and fine registration based on enhanced correlation coefficient optimization, local affine transformation or homography correction.
[0011] Furthermore, the template structure image Bt and the structure image Bi to be inspected are obtained from the grayscale image after local contrast enhancement through adaptive threshold segmentation, threshold segmentation after edge enhancement, Otsu's method segmentation, or a combination thereof.
[0012] Furthermore, the candidate defect region is determined by the candidate map M, which satisfies M=1[(R+>τ+)∪(R->τ-)∪(Bt⊕Bi)], where τ+ and τ- are the positive residual threshold and the negative residual threshold, respectively, and ⊕ represents the structural difference operation.
[0013] Furthermore, the morphological processing includes at least one or more of opening operations, closing operations, hole filling, boundary masking, and minimum area filtering to suppress isolated noise, connect broken regions, and eliminate false candidates related to image boundaries.
[0014] Furthermore, the polarity balance ratio is defined as pb=|A+-A-| / (A++A-+ε), where A+ represents the number of positive residual pixels in the candidate region, A- represents the number of negative residual pixels in the candidate region, and ε is a positive number to prevent the denominator from being zero.
[0015] Furthermore, the screening model is a random forest, support vector machine, logistic regression model, shallow convolutional neural network or a combination thereof; when positive residuals dominate in the candidate region, they are preferentially identified as material-addition anomalies, attachment anomalies or short-circuit anomalies; when negative residuals dominate in the candidate region, they are preferentially identified as material-deficient anomalies, open-circuit anomalies or edge-collapse anomalies.
[0016] An electronic device includes a processor, a memory, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described method.
[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.
[0018] Beneficial effects: (1) Separating positive and negative residuals into models can characterize material-increase anomalies and material-shortage anomalies respectively, thereby enhancing the interpretability of subsequent judgments.
[0019] (2) First generate candidate regions and then perform local statistical filtering to avoid the loss of minor defect information caused by scaling the whole image.
[0020] (3) Secondary discrimination can be completed by using regional statistical features, without relying on large-scale box-level relabeling, which facilitates rapid migration when switching production lines.
[0021] (4) The output retains the defect polarity information, which makes it easy to match the detection results with the process semantics such as open circuit, short circuit, missing material, adhesion or edge breakage. Attached Figure Description
[0022] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.
[0023] Figure 2 This is a schematic diagram of AOI imaging, template construction, and a unified coordinate system.
[0024] Figure 3 This is a schematic diagram showing the generation of positive residuals, negative residuals, and candidate regions.
[0025] Figure 4 This is a schematic diagram of statistical feature extraction and polarity discrimination for candidate regions. Detailed Implementation
[0026] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0027] Example 1: General AOI Inspection Process First, obtain the template image T and the image to be inspected I. The template image T can be obtained directly from a single qualified sample as a reference template, or it can be obtained from multiple qualified samples of the same type through median fusion, average fusion, or majority voting fusion. The purpose of template construction is to reduce local random fluctuations in normal samples and maintain the stable structure of the target image.
[0028] The image to be examined, I, is subjected to grayscale normalization and flat-field correction to obtain a normalized image, In. Preferably, grayscale normalization can be expressed by the following formula: In the formula, μI represents the mean gray level of the image to be inspected, σI represents the standard deviation of the gray level of the image to be inspected, and ε represents a positive number to prevent the denominator from being zero. After normalization, the impact of illumination fluctuations in different batches on subsequent threshold segmentation can be reduced.
[0029] The normalized image In is then registered to the coordinate system of the template image T to obtain the registered image Ir. Registration can be summarized by the following formula: In the formula, W(·) represents the geometric transformation operation, and H represents the spatial transformation parameters determined by coarse registration and fine registration. Coarse registration can use phase correlation or template matching; fine registration can use enhanced correlation coefficient optimization, local affine transformation, or homography correction.
[0030] In the template coordinate system, template structure map Bt and the structure map to be detected Bi are generated from the template image T and the registration image Ir, respectively. Structure map generation can employ local contrast enhancement, adaptive thresholding, edge enhancement followed by thresholding, or a combination thereof. Based on the above results, positive and negative residual maps are constructed respectively: In this model, the positive residual map R+ is used to characterize the extra bright structures or materials in the image to be inspected, while the negative residual map R- is used to characterize structures that exist in the template but are missing in the image to be inspected. If a binary structure map is used, the corresponding polarity residual can also be directly generated based on the set difference between Bt and Bi.
[0031] To obtain high-recall candidate regions, a candidate graph M is constructed, and morphological processing and connected component labeling are performed on it. The candidate graph can be represented by the following equation: In the formula, τ+ and τ- represent the positive residual threshold and the negative residual threshold, respectively, and ⊕ represents the structural difference operation between the template structure diagram and the structure diagram to be inspected. The candidate diagram M can be further processed by opening, closing, hole filling, boundary masking, and minimum area filtering to obtain the candidate region set C={c1,c2,...,cn}.
[0032] For each candidate region ci, a region statistical feature vector si is extracted. Preferably, the feature vector includes region width w, height h, area A, aspect ratio r, fill rate ρ, mean gray-level difference μD, maximum gray-level difference mD, integral gray-level difference sD, number of positive residual pixels A+, number of negative residual pixels A-, template structure ratio qt, and boundary distance db. The polarity balance ratio pb can be given by the following formula: The polarity balance ratio pb reflects the dominant direction of the candidate region in terms of positive and negative residuals. When A+ is significantly greater than A-, the candidate region is more likely to be a material-addition anomaly, an adhesion anomaly, or a short-circuit anomaly; when A- is significantly greater than A+, the candidate region is more likely to be a material shortage anomaly, an open-circuit anomaly, or a chipping anomaly.
[0033] The regional statistical feature vector si is input into the screening model or rule base for secondary discrimination, resulting in an output result indicating whether the candidate region belongs to a true defect or a false candidate. The secondary discrimination can be summarized as follows: In the formula, yi represents the discrimination result of the i-th candidate region, and fθ represents random forest, support vector machine, logistic regression model, shallow convolutional neural network or a combination thereof. After two-level discrimination, the defect bounding box, defect polarity and corresponding judgment result are output.
[0034] Example 2: Replacement Example with Disclosed Chip / Semiconductor Device Data When validating the method using publicly available data, a subset of transistors from the MVTec AD dataset can be used as a publicly available example of a semiconductor device. This dataset has an official downloadable page, and its published paper describes that the dataset was acquired under controlled lighting conditions using a 2048×2048 industrial RGB sensor and dual telecentric lenses, with cropped images at resolutions ranging from 700×700 to 1024×1024 pixels. Directly identifiable anomaly categories within the transistor subset include bentlead, cutlead, damaged case, and misplaced. In implementation, a normal template library can be constructed from the train / good subset, and the method flow of this invention can be executed on test / good and test / abnormal samples.
[0035] For publicly available implementations that are closer to the chip object, the MIIC dataset and the publicly available chip surface defect image data corresponding to the literature "Attention-based deep learning for chip-surface-defect detection" can be further used. MIIC is publicly described as an integrated circuit microscopic image anomaly detection dataset; for camera models, exposure times, lens focal lengths, and category statistics not explicitly disclosed in the public metadata, this embodiment does not impose numerical limitations, and these should be determined by the implementer based on the downloaded data description file.
[0036] In practical deployment, the template image and the image to be inspected should meet consistent imaging conditions, including the same lens magnification, the same exposure parameters, the same lighting arrangement, the same tooling posture, and the same background constraints; if necessary, positioning fixtures, field-of-view cropping, and geometric correction should be used to bring the images into a unified coordinate system. For linear material shortages, bridging adhesions, and edge damage on the chip surface, this invention can first obtain high-recall candidate regions through polarity residuals, and then use secondary screening to suppress false alarms caused by brightness fluctuations, edge misalignment, and local reflections.
[0037] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A method for generating and statistically screening bipolar residual candidate regions for chip surface defects, characterized in that, Includes the following steps: S1. Obtain at least one qualified sample image as template image T from the same model chip or semiconductor device with the same package under the same imaging magnification, the same exposure parameters, the same lighting arrangement and the same tooling posture, and obtain the image to be inspected I. S2. Perform grayscale normalization, flat field correction and geometric registration on the image to be inspected I to obtain the registered image Ir corresponding to the template image T; S3. Generate template structure map Bt and structure map Bi to be inspected from template image T and registration image Ir respectively, and construct positive residual map R+ and negative residual map R-, where R+(x,y)=max(Ir(x,y)-T(x,y),0), R-(x,y)=max(T(x,y)-Ir(x,y),0); S4. Perform threshold segmentation, morphological processing, and connected component labeling on the structural difference results of the positive residual map R+, negative residual map R-, template structure map Bt, and the structure map Bi to be inspected, to obtain a set of candidate defect regions. S5. Extract at least the following regional statistical features for each candidate defect region: region width, height, area, aspect ratio, fill rate, mean gray-level difference, maximum gray-level difference, integral gray-level difference, number of positive residual pixels, number of negative residual pixels, template structure ratio, boundary distance, and polarity balance ratio. S6. Input the statistical features of the region into the filtering model or rule base for secondary discrimination, and output the defect location, defect bounding box, defect polarity and defect judgment result.
2. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The template image T is obtained by directly using a single qualified sample image as a reference template, or by using multiple qualified sample images of the same type through pixel-level median fusion, average fusion, or majority voting fusion.
3. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The grayscale normalization and flat field correction include estimating the background illumination field based on normal samples from the same batch and performing brightness equalization on the image to be inspected, thereby reducing the impact of uneven illumination, vignetting, and local reflections on the difference results.
4. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The geometric registration includes coarse registration based on phase correlation or template matching, and fine registration based on enhanced correlation coefficient optimization, local affine transformation or homography correction.
5. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The template structure image Bt and the structure image Bi to be inspected are obtained from the grayscale image after local contrast enhancement through adaptive threshold segmentation, threshold segmentation after edge enhancement, Otsu's method segmentation, or a combination thereof.
6. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The candidate defect region is determined by the candidate map M, which satisfies M=1[(R+>τ+)∪(R->τ-)∪(Bt⊕Bi)], where τ+ and τ- are the positive residual threshold and the negative residual threshold, respectively, and ⊕ represents the structural difference operation.
7. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The morphological processing includes at least one or more of opening, closing, hole filling, boundary masking, and minimum area filtering to suppress isolated noise, connect broken regions, and eliminate false candidates related to image boundaries.
8. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The polarity balance ratio is defined as pb=|A+-A-| / (A++A-+ε), where A+ represents the number of positive residual pixels in the candidate region, A- represents the number of negative residual pixels in the candidate region, and ε is a positive number to prevent the denominator from being zero.
9. The method for generating and statistically screening bipolar residual candidate regions for chip surface defects according to claim 1, characterized in that, The screening model is random forest, support vector machine, logistic regression model, shallow convolutional neural network or a combination thereof; when positive residuals dominate in the candidate region, they are preferentially identified as material-addition anomalies, attachment anomalies or short-circuit anomalies; when negative residuals dominate in the candidate region, they are preferentially identified as material-deficient anomalies, open-circuit anomalies or edge-collapse anomalies.