A method for extracting the minimum grain unit based on template side length ratio matching

Abstract

The present invention discloses a method for extracting the minimum crystal grain unit based on template side length ratio matching, comprising the following steps: 1) using a CCD industrial camera to acquire an image; 2) acquiring the center point of the crystal grain template based on normalized similarity matching; 3) extracting the minimum unit based on a vertical vector method; 4) adapting the template size based on the template side length ratio. The present invention extracts image features based on two-dimensional convolution of the image; acquires the center point of the crystal grain template based on similarity matching; uses connected components to mark bright spot targets; extracts the minimum unit based on a vertical vector method; and adapts the template size based on the template side length ratio, thereby improving the efficiency of wafer template matching.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for extracting a minimum grain unit based on template side length ratio matching. Background Art

[0002] During the wafer defect detection process, since the wafer surface has neatly patterned circuits, the neat circuit patterns will become a very complex background when the wafer is visually inspected. Therefore, most methods used for defect detection are to choose template matching, because for regularly arranged images, template matching detection is faster.

[0003] The main technology that needs to be solved this time is the template extraction method in the existing wafer inspection technology using template matching. Whether the template can be accurately extracted is important for the subsequent detection of the similarity between the image and the template of the inspection grain, and a template extraction method is proposed.

[0004] In 2019, Chen Zhishan et al. also studied grain defects in the article "Wafer Surface Defect Detection Based on Machine Vision". The article did not mention the acquisition of template images, so whether the template image can be applied to contour detection may have a certain impact on the subsequent contour matching.

[0005] In 2019, Zhang Yidong and others invented a method and system for detecting wafer defects (publication number: CN108831844B). This invention provides a wafer detection method and a wafer detection system, which uses infrared imaging to identify the defects of the wafer to be detected. However, the wafer surface has a complex background, so the efficiency will be improved by using template matching for defect detection.

[0006] In 2020, Zhao Chundong et al. conducted research on grain defects in the article "Research on a Method to Improve the Accuracy of Grain Surface Defect Detection". Using K-means clustering based on intra-class and inter-class deviation matrices to segment images may cause the segmented grains to be of different sizes, affecting the accuracy of subsequent template matching.

[0007] In summary, although the existing template matching method can perform image registration, the registration efficiency problem has not been solved. When using template matching detection, the selection of grain template size is usually based on the smallest grain, which may lead to inaccurate matching results and increase the time of later template matching. The influence of grain size on image registration accuracy has not been studied, so the grain template that is most suitable for the image registration should be selected for matching. Summary of the invention

[0008] The purpose of the present invention is to overcome the defects of the prior art and provide a method for extracting the minimum grain unit based on template side length ratio matching, which can select the grain template most suitable for image registration for matching, and the efficiency and effect of template matching will be improved.

[0009] The object of the present invention is achieved by: a method for extracting the minimum grain unit based on template side length ratio matching, comprising the following steps:

[0010] Step 1) using a CCD industrial camera to acquire images;

[0011] Step 2) obtaining the center point of the grain template based on normalized similarity matching;

[0012] Step 3) extracting the minimum unit based on the vertical vector method;

[0013] Step 4) Adapt the template size based on the template side length ratio.

[0014] As a further limitation of the present invention, the step 2) specifically comprises:

[0015] Step 2.1) The original image is converted to a grayscale image with a pixel resolution of w p ×h p The image is randomly cropped from the grayscale image to the central region with a pixel resolution of w Tp ×h Tp ;

[0016] Step 2.2) Normalize using squared difference matching to be in the interval [0, 1]; perform template matching to find the portion of image P that is similar to template T, slide the initial template image over the patch, and calculate the following metrics to compare template T with the local area of ​​the patch;

[0017]

[0018] Where (x',y') means 0≤x'<w Tp and 0≤y'<h Tp One of the pixels covered by the template, w Tp ,h Tp are the pixels randomly cropped to the center area in step 2.1); where R(x, y) are the two evaluation images I T and I L The normalized cross-correlation between ; therefore, the pixels R(x,y) form 0≤x≤w p -w Tp and 0≤y≤h p -h Tp The related image R of p ,h pis a pixel of the grayscale image;

[0019] Step 2.3) Apply binary thresholding to the similarity map obtained in the above step to obtain a binary map, which is mathematically expressed as follows:

[0020]

[0021] If the similarity is higher than the set threshold T, it is marked as 1, otherwise it is 0;

[0022] Step 2.4) Apply the connected component method to label all the bright targets in the graph R B and then calculate the centroid of each target. Here, c i =(x i , y i ) represents the center of the I-th object. By considering every possible object of (i, j), a set of permutation tuples is found;

[0023]

[0024] In the formula: D represents the moving tuple, and d i,j represents the displacement vector.

[0025] As a further limitation of the present invention, the said step 4) specifically includes:

[0026] Step 4.1) Extract the minimum grain template, translate the minimum unit in all directions to form an image composed of 9 minimum units;

[0027] Step 4.2) Extend 1 / 2, 1 / 3, and 1 / 4 of the side length of the minimum unit outward respectively, and calculate the histogram similarity with the corresponding position of the original image;

[0028]

[0029] In the formula: l 1 , l 2 , l 3 respectively represent the ratios corresponding to extending 1 / 2, 1 / 3, and 1 / 4 outward; h is the side length of the grain template; s is the shorter side length of the original image;

[0030] Step 4.3) Calculate the histogram similarity R n ;

[0031] Step 4.4) Calculate the average similarity The metric will present an array composed of four similarity values R (-1 < R < 1). The extraction result is evaluated by the average value of this array, as shown in formula (6),

[0032]

[0033] Iterate the ratio of the side length of the rectangular template in the interval (0,1) and select the mean similarity The ratio of the rectangle side length when it is the largest is used to calculate the size of the template at this time.

[0034] The present invention adopts the above technical solution, and compared with the prior art, the beneficial effect is as follows: in order to study the influence of the template selection size on the final matching result, the side length of the original grain template is extended to the outside of the four sides, and then the histogram similarity between it and the corresponding position of the original image is calculated. The proportion of the rectangular side length when the similarity mean is the largest is selected to calculate the size of the template at this time. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0036] Figure 2 A flow chart for obtaining the center point of the grain template of the present invention.

[0037] Figure 3 It is a schematic diagram of the iteration of the optimal ratio of template side lengths of the present invention.

[0038] Figure 4 It is a schematic diagram of the relationship between the mean similarity value and the template side length ratio of the present invention.

[0039] Figure 5 This is a schematic diagram of the minimum unit extraction of the vector method of the present invention. DETAILED DESCRIPTION

[0040] like Figure 1 A method for extracting the minimum grain unit based on template side length ratio matching is shown, comprising the following steps:

[0041] Step 1) Using a CCD industrial camera to obtain an image; Using a CCD camera to take an image of the wafer to be inspected.

[0042] Step 2) obtaining the center point of the grain template based on normalized similarity matching; Figure 2 As shown, step 2) specifically includes:

[0043] Step 2.1) The original image is converted to a grayscale image with a pixel resolution of w p ×h p The image is randomly cropped from the grayscale image near the center area;

[0044] Step 2.2) Perform a template matching process to find parts of the image P that are similar to the template T, slide the initial template image over the patch as in a 2D convolution, and calculate the following metrics for comparing the template T with the local area of ​​the patch;

[0045]

[0046] Where (x',y') means 0≤x'<w Tp and 0≤y'<h Tp One of the pixels covered by the template, w Tp ,h Tp are the pixels randomly cropped to the center area in step 2.1); where R(x, y) are the two evaluation images I T and I L The normalized cross-correlation between ; therefore, the pixels R(x,y) form 0≤x≤w p -w Tp and 0≤y≤h p -h Tp The related image R of p ,h p are the pixels of the grayscale image; it is noteworthy that bright pixels indicate high similarity at these locations.

[0047] Step 2.3) Apply binary thresholding to the similarity graph obtained in the above step to obtain a binary graph, which is mathematically represented as follows:

[0048]

[0049] If the similarity is higher than the set threshold T, it is marked as 1, otherwise it is 0;

[0050] Step 2.4) Apply the connected component method to the graph R B All bright targets in are marked, and then the center of mass of each target is calculated, where c i =(x i ,y i ) represents the center of the I-th object, and a set of permutation tuples is found by considering every possible object of (i, j).

[0051]

[0052] Where: D represents the mobile tuple, d i,j Represents a displacement vector.

[0053] Step 3) extracting the minimum unit based on the vertical vector method;

[0054] After completing the similarity matching through template matching, locate the center point of the rectangle, and use the center point of the rectangle to replace the matched rectangle to form a center point set. The point closest to the center of the image is determined as the origin O. From the origin to the radius of the image range, use the vertical vector method to search for a point A closest to the origin to determine the OA vector. Using the OA vector as a reference, search for a point B that is perpendicular to the OA vector and closest to the origin. That is, OA and OB form a vertical angle. The angle calculation formula is shown in formula (4):

[0055]

[0056] In the point set, there may be a certain error in the point position, so the error must be considered when judging the vertical angle (the allowable error is: 90°±1.5°). Figure 5 As shown, the OA and OB vectors can determine an approximate rectangle. According to the angle and distance between OA and OB, an approximate rectangle is obtained. In actual activities, the smallest unit is mostly a rectangle, so the approximate rectangle is further processed to make its angle a right angle. The corresponding position in the original image is determined by the rectangle, and the minimum unit image is obtained, thereby realizing the minimum unit extraction.

[0057] Step 4) adapting the template size based on the template side length ratio;

[0058] In order to explore the influence of the size of the rectangular template on the results in similarity matching, the relationship between the size of the rectangular template and the results is obtained. The accuracy of the result is actually the similarity between the spliced ​​part and the corresponding position in the sample when two minimum units are spliced ​​together; the histogram similarity is used to measure the similarity, such as Figure 3 As shown, step 4) specifically includes:

[0059] Step 4.1) extract the minimum grain template, translate the minimum unit to all sides, and form an image composed of 9 minimum units;

[0060] Step 4.2) Extend the side length of the smallest unit to the outside by 1 / 2, 1 / 3, and 1 / 4 respectively, and calculate the histogram similarity between the side length of the smallest unit and the original image;

[0061]

[0062] Where: l 1 ,l 2 ,l 3 They represent the proportions corresponding to 1 / 2, 1 / 3, and 1 / 4 extensions respectively; h is the side length of the grain template; s is the shorter side length of the original image;

[0063] Step 4.3) Calculate the histogram similarity R between the downward extended part and the corresponding position of the original image n ;

[0064] Step 4.4) Calculate the average similarity The metric will present an array composed of four similarity values R (-1 < R < 1). The average value of this array is used to evaluate the extraction result. As shown in formula (6), the average similarity The closer it is to 1, the better the extraction effect; conversely, the worse the extraction effect

[0065]

[0066] Iterate on the ratio of the side length of the rectangular template in the interval (0, 1), and select the ratio of the side length of the rectangle when the average similarity is the largest, and calculate the size of the template at this time. Draw a corresponding relationship graph, with the ratio of the side length of the template on the abscissa and the average similarity on the ordinate as Figure 4 shown

[0067] When the ratio of the side length of the rectangular template is less than 0.1 or greater than 0.8, it exceeds the threshold of template matching and the matching cannot be carried out; the iteration result of the ratio of the rectangular template shows that the ratio that makes the average similarity the largest is about l 2 , so this ratio is the optimal ratio

[0068] The present invention performs feature extraction of an image based on two-dimensional convolution of the image; obtains the center point of the crystal grain template based on similarity matching; labels the bright spot target by using connected components; extracts the minimum unit based on the vertical vector method; and adapts the template size based on the ratio of the side length of the template, thereby improving the efficiency of wafer template matching

[0069] The present invention is not limited to the above embodiments. Based on the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and deformations to some technical features without creative labor according to the disclosed technical content, and these substitutions and deformations are all within the protection scope of the present invention

Claims

1. A method for extracting the minimum grain unit based on template side length ratio matching, characterized in that: The following steps are involved: Step 1) using a CCD industrial camera to acquire an image; Step 2) obtaining the center point of the grain template based on normalized similarity matching; The step 2) specifically includes: Step 2.1) The original image is converted to a grayscale image with a pixel resolution of w p ×h p The image is randomly cropped from the grayscale image to the central region with a pixel resolution of w Tp ×h Tp ; Step 2.2) Normalize using squared difference matching to be in the interval [0, 1]; perform template matching to find the portion of image P that is similar to template T, slide the initial template image over the patch, and calculate the following metric to compare template T with the local region of the patch to obtain pixel R(x, y); Where (x, y) means 0≤x<w p and 0≤y<h p One of the covered pixels, (x', y') means 0≤x'<w Tp and 0≤y'<h Tp One of the pixels covered by the template, w Tp ,h Tp is the pixel randomly cropped to the center area in step 2.1); where the pixel R(x, y) is the pixel of the two evaluation images I T and I L The normalized cross-correlation value between ; therefore, the pixels R(x,y) are combined to form 0≤x≤w p -w Tp and 0≤y≤h p -h Tp The related image R of p ,h p is the pixel of the grayscale image; Step 2.3) Apply binary thresholding to the correlation image R obtained in the above step to obtain a binary map, which is mathematically expressed as follows: If the pixel R(x,y) is higher than the set threshold T, it is marked as 1, otherwise it is 0; Step 2.4) Apply the connected component method to the graph R B All bright targets in are marked, and then the center of mass of each target is calculated, where c i =(x i ,y i ) represents the center of the I-th object, and find a set of permutation tuples by considering every possible object of (i, j); Where: D represents the mobile tuple, d i,j represents the displacement vector; x i ,y i are the pixel values ​​in the X and Y directions of the center of the I-th object; j ,y j are the pixel values ​​in the X direction and Y direction of the center of the Jth object respectively; i and j represent the Ith object and the Jth object respectively; Step 3) extracting the minimum unit based on the vertical vector method; After completing the similarity matching through template matching, locate the center point of the rectangle, and use the center point of the rectangle to replace the matched rectangle to form a center point set; determine the point closest to the image center as the origin O; use the vertical vector method to search for a point A closest to the origin within the radius from the origin to the image range to determine the OA vector; use the OA vector as a reference to search for a point B that is perpendicular to the OA vector and closest to the origin; that is, OA and OB form a vertical angle; the angle calculation formula is shown in formula (4): in, and Represents two vectors, |A| and |B| represent vectors respectively. and vector The modulus of OA and OB vectors can determine an approximate rectangle. According to the angle and distance between OA and OB, an approximate rectangle is obtained. In actual activities, the smallest unit is mostly a rectangle, so the approximate rectangle is further processed to make its angle a right angle. The corresponding position in the original image is determined by the rectangle, and the smallest unit image is obtained, thereby realizing the extraction of the smallest unit. Step 4) adapting the template size based on the template side length ratio; The step 4) specifically includes: Step 4.1) extract the minimum grain template, translate the minimum unit to all sides, and form an image composed of 9 minimum units; Step 4.2) Extend the side length of the smallest unit to the outside by 1 / 2, 1 / 3, and 1 / 4 respectively, and calculate the histogram similarity between the side length of the smallest unit and the original image; Where: l1, l2, l3 represent the proportions corresponding to the extension of 1 / 2, 1 / 3, and 1 / 4 respectively; h is the side length of the grain template; s is the shorter side length of the original image; Step 4.3) Calculate the histogram similarity R between the extended part and the corresponding position of the original image n ; Step 4.4) Calculate the average similarity The metric will present an array consisting of four similarity values R, where -1 < R < 1. The extraction result is evaluated using the average value of this array, as shown in formula (6). Iterate the ratio of the side length of the rectangular template in the interval (0,1) and select the mean similarity The ratio of the rectangle side length when it is the largest is used to calculate the size of the template at this time.

Images