Defect detection method for macroscopic visual inspection machine

Abstract

The invention discloses a defect detection method for a macroscopic visual inspection machine, which comprises the following steps of: 1) shooting a plurality of local images around a target detection area to obtain a group of local images which cover the target area and are mutually overlapped; 2) fusing the group of local images acquired in the step 1) into a complete spliced image covering the whole target area; 3) processing the spliced image obtained in the step 2) to obtain a reference template representing an ideal state, namely a gold template; 4) carrying out real-time detection, shooting an image of the current detected area, and searching a subimage which is most matched with the image of the current detected area from the gold template; and 5) comparing the image of the detected area with the sub-image searched in the step 4), finding out the difference between the image of the detected area and the sub-image, namely a defect, and then outputting a detection result marked with a defect position, a defect size and a defect type.

Description

Technical Field

[0001] This invention relates to the field of semiconductor inspection technology, and in particular to a defect detection method for a macroscopic visual inspection machine. Background Technology

[0002] In semiconductor manufacturing, defect detection is required. This is typically achieved by capturing images of the semiconductor using a camera, then matching these images with a template to analyze and identify defects. However, existing technologies suffer from the following drawbacks: 1. Blind spots due to inaccurate machine positioning: Traditional inspection relies on precise single-shot positioning. Industrial machines have mechanical tolerances (±0.5mm or more), preventing complete alignment between the image capture area and the template, leading to missed defects. Using high-cost solutions like laser positioning (accuracy ±0.1mm) increases equipment costs by over 50%, resulting in significant investment. 2. Low global imaging efficiency: Existing solutions require capturing high-resolution images of the entire workpiece, which is time-consuming and cannot meet the requirements of online inspection. 3. Insufficient image registration accuracy: Existing golden images are often based on single shots or simple averaging, easily including non-permanent defects such as dust and water stains, resulting in low detection accuracy. Traditional template matching methods (such as NCC) have a mismatch rate >20% in areas with repetitive textures. Summary of the Invention

[0003] In order to solve the above problems, the present invention aims to provide a defect detection method for a macroscopic visual inspection machine.

[0004] This invention provides a defect detection method for a macroscopic visual inspection machine, comprising the following steps:

[0005] 1) Take multiple local images around the target detection area to obtain a set of local images that cover the target area and overlap with each other;

[0006] 2) Merge the set of local images acquired in step 1) into a complete mosaic image that covers the entire target area;

[0007] 3) Process the stitched image obtained in step 2) to obtain a reference template representing the ideal state, namely the golden template;

[0008] 4) Perform real-time detection, capture an image of the current area being tested, and search for the sub-image that best matches the current area image from the gold template;

[0009] 5) Compare the image of the area to be tested with the sub-image searched in step 4) to find the difference between the two, which is the defect. Then, output the detection result, which is marked with the defect location, defect size and defect type.

[0010] Furthermore, in the defect detection method for a macroscopic visual inspection machine provided by the present invention, the number of local images captured in step 1) is ≥4, including at least one image each from the top, bottom, left, and right angles.

[0011] Furthermore, in the defect detection method for a macroscopic visual inspection machine provided by the present invention, the overlapping area of ​​adjacent images in step 1) is ≥30%.

[0012] Furthermore, in the defect detection method for a macroscopic visual inspection machine provided by the present invention, step 2) includes the following steps: 2-1, image registration: find corresponding point pairs in the overlapping area and calculate the transformation matrix to align each local image to the same global coordinate system; 2-2, image fusion: smoothly fuse the aligned images in the overlapping area to eliminate seams; 2-3, illumination compensation: perform illumination compensation on the fused images to make the illumination of each image consistent.

[0013] Furthermore, in the defect detection method for a macroscopic visual inspection machine provided by the present invention, the transformation matrix in step 2-1 is an affine transformation or a homography matrix.

[0014] Furthermore, in the defect detection method for macroscopic visual inspection machine provided by the present invention, the image fusion method in step 2-2 adopts one or more of the following: direct averaging, weighted averaging, multi-band fusion, and optimal seam line.

[0015] Furthermore, in the defect detection method for macroscopic visual inspection machine provided by the present invention, step 3) includes the following steps: 3-1: Repeat steps 1) and 2) N times, N≥3, to obtain N independent complete stitched images; 3-2: Fuse the N stitched images, the fusion rule follows to retain the same parts and remove abnormal parts, and the image obtained after fusion is the gold template.

[0016] Furthermore, in the defect detection method for macroscopic visual inspection equipment provided by the present invention, the fusion algorithm used in step 3-2 preferably adopts median calculation.

[0017] Furthermore, in the defect detection method for macroscopic visual inspection equipment provided by the present invention, the fusion algorithm used in step 3-2 can also employ image segmentation or clustering to identify regions that exist in N images, and regions that do not appear in all N images are considered abnormal and removed.

[0018] Furthermore, in the defect detection method for a macroscopic visual inspection machine provided by this invention, step 4) includes the following steps: 4-1, Image registration: The current image of the area under test is regarded as a template, and template matching is performed in the golden template; 4-2, Template matching: The template matching algorithm adopts an algorithm based on grayscale correlation and feature points; 4-3, Search: According to the theoretical positioning accuracy of the machine, a search area is defined in the golden template, and the sub-image that best matches the current image of the area under test is cropped from the golden template.

[0019] Furthermore, in the defect detection method for macroscopic visual inspection equipment provided by the present invention, step 5) includes the following steps: 5-1, image difference; 5-2, image post-processing, including threshold segmentation, morphological operations, opening operation, closing operation, connected component analysis, and feature screening.

[0020] Furthermore, in the defect detection method for macroscopic visual inspection equipment provided by the present invention, step 5) may also include a machine learning-based classifier operation, in which the difference map or independent defect region Blob is fed into the classifier to determine whether it is a real defect.

[0021] Furthermore, in the defect detection method for macroscopic visual inspection equipment provided by the present invention, step 5) may also include structural similarity calculation.

[0022] The present invention has the following beneficial effects:

[0023] 1. Low cost and anti-offset: Through multi-angle shooting and image fusion technology, it is compatible with machines with a positioning error of ±2mm, reducing hardware costs by 30%.

[0024] 2. Clean template generation: Three-image fusion completely eliminates temporary defects, reducing the false detection rate to <3%.

[0025] 3. Dynamic local comparison: Only a small image of the area to be inspected needs to be captured.

[0026] 4. Multi-level registration guarantee: SIFT feature points + affine transformation are used in the image merging stage, and ORB feature points are combined in the search stage, with a registration error of <0.1 pixels. Attached Figure Description

[0027] Figure 1 These are a set of local images covering the target area and overlapping each other, as described in step 1) of this embodiment of the invention.

[0028] Figure 2 This refers to step 2) of the present invention, where a group of partial images are stitched together into a complete stitched image, Stitched_Image.

[0029] Figure 3This refers to step 3 of the present invention, which involves processing three independent, complete stitched images (Stitched_Image) to obtain the Golden Image template.

[0030] Figure 4 This refers to step 4) of this embodiment of the invention, where the Golden Patch that best matches the current tested region image is cropped from the Golden Image.

[0031] Figure 5 This refers to the detection results obtained from image comparison in step 5) of this embodiment of the invention. Detailed Implementation

[0032] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, will specifically illustrate the technical solution of this invention.

[0033] This embodiment provides a defect detection method for a macroscopic visual inspection machine, including the following steps:

[0034] 1) Image acquisition template definition

[0035] Multiple local images are captured around the target detection area, covering all possible offset ranges within the target area, resulting in a set of overlapping local images that cover the target area. The purpose is to overcome the problem of insufficient repeatability accuracy of the machine and ensure that the template can cover all possible positional deviations.

[0036] The number of partial images taken is ≥4, including at least one each from the top, bottom, left, and right angles.

[0037] The overlap area between adjacent images should be ≥30%. Sufficient overlap between adjacent images is crucial for subsequent registration and fusion.

[0038] See Figure 1 The figure shows four partial images obtained in step 1) of this embodiment.

[0039] 2) Image stitching

[0040] The set of local images acquired in step 1) are merged into a complete stitched image Stitched_Image that covers the entire target area.

[0041] Specifically, the following steps are included:

[0042] 2-1. Image Registration: Find corresponding point pairs (Feature Matching) in overlapping regions and calculate a transformation matrix to precisely align each local image to the same global coordinate system. This transformation matrix can be an affine transformation or a homography matrix.

[0043] 2-2. Image Blending / Compositing: This involves smoothly blending aligned images in overlapping areas to eliminate seams.

[0044] The image fusion method in this step can be one or more of the following, depending on the actual situation: direct averaging, weighted averaging, multi-band fusion, and optimal seam. Direct averaging / weighted averaging is simple but may lead to blurring; multi-band fusion has good results and can better preserve high-frequency details; the optimal seam is to find the path with the least difference in overlapping areas for stitching.

[0045] 2-3. Illumination Compensation: Perform illumination compensation on the fused images to ensure consistent illumination across all images.

[0046] Step 2) of image stitching has the following requirements:

[0047] Accuracy: The registration error must be much smaller than the defect size.

[0048] No artifacts: The fused image should not have ghosting, blurring, or obvious stitching seams.

[0049] Geometric fidelity: The geometric relationships of the stitched image must be consistent with the real object, that is, "consistent with reality".

[0050] See Figure 2 The figure illustrates step 2) of this embodiment, which involves stitching together a set of local images into a complete large image, Stitched_Image.

[0051] 3) Generate a gold template

[0052] The stitched image obtained in step 2) is processed to obtain the Golden Image template. The purpose is to eliminate noise from a single measurement, accidental defects (such as dust or water stains), or minor positional variations, to obtain a clean, defect-free reference template that represents the "ideal state" of the product.

[0053] Specifically, the following steps are included:

[0054] 3-1: Repeat steps 1) and 2) N times, where N ≥ 3, to obtain N independent complete stitched images. In this embodiment, three independent complete stitched images were obtained: Stitched_Image1, Stitched_Image2, and Stitched_Image3.

[0055] 3-2: Merge N stitched images, following the rule of retaining the same parts and removing abnormal parts. The resulting image is the golden template.

[0056] Image fusion algorithms can employ the following conventional algorithms:

[0057] i. Pixel-level statistics: Perform statistics on each pixel position (x, y).

[0058] ii. Median: Golden_Image(x, y) = Median(Stitched_Image1(x, y),Stitched_Image2(x, y), Stitched_Image3(x, y)). This method is the most commonly used and effective, and is preferred. The median algorithm can effectively filter out salt-and-pepper noise, isolated points (defects), and some Gaussian noise.

[0059] iii. Mean / Average: Golden_Image(x, y) = Mean(Stitched_Image1(x,y), Stitched_Image2(x, y), Stitched_Image3(x, y)). The mean algorithm is fast to calculate and sensitive to outliers (large defects).

[0060] iv. Maximum / Minimum Values: The brightest or darkest outliers will be retained.

[0061] Furthermore, the image fusion algorithm can also employ image segmentation or clustering algorithms to identify regions (background / normal structures) present in the N images, and regions that do not appear in all N images are considered abnormal and removed. For example, in this embodiment, regions that are stably present in all three images are identified, and regions that appear only in one or two images are considered abnormal and removed.

[0062] Step 3) The three Stitched_Images in this embodiment must be precisely registered in space, otherwise the fusion result will be blurry. The solution of this invention has ensured the consistency of the global coordinate system during the stitching process in step 2).

[0063] See Figure 3The figure illustrates step 3) of this embodiment, which involves processing three independent, complete stitched images (Stitched_Image) to obtain the Golden Image template.

[0064] 4) Image search matching

[0065] Real-time detection is performed by capturing an image of the current region under test (RUT). During real-time detection, the machine's positioning remains inaccurate, requiring a search within the Golden_Image template to find the most similar corresponding region (Golden_Patch) to the current RUT image.

[0066] The search and matching process includes the following steps:

[0067] 4-1. Image registration: Treat the current RUT image as a template and perform template matching in the Golden_Image template.

[0068] 4-2. Template matching: Template matching algorithms include gray-scale-based algorithms or feature point-based algorithms.

[0069] Based on grayscale correlation: for example, Normalized CrossCorrelation (NCC) can be used. The similarity between the RUT and the sliding window at each possible location on the Golden Image is calculated, and the location with the highest similarity is taken as the matching location. This method has some robustness to changes in illumination.

[0070] Feature-based approach: Feature points (SIFT, SURF, ORB, etc.) are extracted from both the RUT and Golden Image, feature matching is performed, and then the transformation matrix is ​​calculated. This method is suitable for cases with rotation and scaling, and its robustness is generally better than grayscale correlation.

[0071] 4-3. Search: Based on the theoretical positioning accuracy of the machine, a reasonable search area (Search ROI) is defined in the golden template to improve speed and accuracy. After the search, the sub-image (Golden_Patch) that best matches the current RUT is cropped from the golden template.

[0072] See Figure 4 The figure illustrates step 4) of this embodiment, which involves cropping the Golden Patch from the Golden Image that best matches the current tested region image (shown by the red rectangle in the figure).

[0073] 5) Image comparison

[0074] The tested region image RUT is compared with the sub-image Golden_Patch found in step 4). The differences between the two are identified as defects. Then, the detection results are output, which are marked with the defect location, defect size, and defect type. Specifically, the steps include:

[0075] 5-1. Image Differencing: Difference_Map = |RUT - Golden_Patch| (or squared difference)

[0076] 5-2. Image Post-processing:

[0077] i. Thresholding: A threshold T is set for the Difference_Map. Pixels greater than T are considered potential defect points;

[0078] ii. Morphological Operations.

[0079] iii. Opening: Erosion followed by dilation removes small noise points.

[0080] iv. Closing: First expand, then erode, filling small voids.

[0081] v. Connected Component Analysis: This method identifies connected pixel regions in a binarized difference image as independent defect blobs.

[0082] vi. Feature filtering: Filter blobs based on actual defect features (such as area, aspect ratio, shape, and location) to remove false defects that do not meet the criteria (such as residual noise or edge differences caused by registration errors).

[0083] Furthermore, the following methods can also be used for image post-processing operations:

[0084] Machine learning-based classifiers: Feed the difference map or blob region into a classifier (such as SVM, RandomForest, CNN) to determine whether it is a real defect.

[0085] Structural similarity (SSIM): It reflects perceptual differences better than simple difference, but requires slightly more computation.

[0086] See Figure 5The figure illustrates the detection results obtained by image comparison in step 5) of this embodiment. The left figure is the image to be detected, and the right figure is the sub-image that best matches it (Golden_Patch). Defects are marked in the figure.

[0087] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A defect detection method for a macroscopic visual inspection machine, characterized by, The method comprises the following steps: 1) Taking multiple local images around the target detection area to obtain a set of local images covering the target area and overlapping with each other; 2) Fusing the set of local images collected in step 1) into a complete spliced image covering the entire target area; 3) Processing the spliced image obtained in step 2) to obtain a reference template representing an ideal state, i.e. a golden template; 4) Real-time detection, taking a current image of the measured area, searching for a sub-image most matching the current image of the measured area from the golden template; 5) Comparing the image of the measured area with the sub-image searched in step 4) to find the difference between the two, i.e. the defect, and then outputting the detection result marked with the defect position, defect size and defect type.

2. The defect detection method for a macro visual inspection machine according to claim 1, wherein the number of local images taken in step 1) is ≥4, including at least one image for each of the upper, lower, left and right angles.

3. The defect detection method for a macro visual inspection machine according to claim 1, wherein the overlapping area of adjacent images in step 1) is ≥30%.

4. The defect detection method for a macro visual inspection machine according to claim 1, wherein step 2) comprises the following steps: 2-1, image registration: finding corresponding point pairs in the overlapping area and calculating a transformation matrix to align each local image to the same global coordinate system; 2-2, image fusion: smoothly transitioning and fusing the aligned images in the overlapping area to eliminate the joint; 2-3, illumination compensation: compensating the fused image to make the illumination of each image consistent.

5. The defect detection method for a macro visual inspection machine according to claim 4, wherein the transformation matrix in step 2-1 is an affine transformation or a homography matrix.

6. The defect detection method for a macro visual inspection machine according to claim 4, wherein the image fusion method in step 2-2 adopts one or more of direct averaging, weighted averaging, multi-band fusion and optimal joint line.

7. The defect detection method for a macro visual inspection machine according to claim 1, wherein step 3) comprises the following steps: 3-1: repeating steps 1) and 2) N times, N≥3, to obtain N independent complete spliced images; 3-2: fusing the N spliced images, the fusion rule being to retain the same parts and eliminate abnormal parts, the fused image being the golden template.

8. The defect detection method for a macro visual inspection machine according to claim 1, wherein step 4) comprises the following steps: 4-1, image registration: taking the current image of the measured area as a template and performing template matching in the golden template; 4-2, template matching: the template matching algorithm adopts a gray correlation-based algorithm and a feature point-based algorithm; 4-3, searching: according to the theoretical positioning accuracy of the machine, defining a search area in the golden template and cutting out a sub-image most matching the current image of the measured area from the golden template.

9. The defect detection method for a macro visual inspection machine according to claim 1, wherein ​ ​ ​ ​ ​ ​ ​ Step 5) includes the following steps: 5-1, image difference; 5-2, image post-processing, including threshold segmentation, morphological operation, opening operation, closing operation, connected domain analysis, feature screening.

10. The defect detection method for macroscopic inspection machine according to claim 9, characterized in that: Step 5) further includes the following steps: 5-3, a classifier based on machine learning, the difference map or the independent defect area Blob is sent into the classifier to judge whether it is a real defect.

Images