An image matching method

Abstract

The application discloses an image matching method, comprising the following steps: S1, acquiring a template image, and collecting a to-be-matched image which is larger than the size of the template image; S2, calculating the similarity of the to-be-matched image and the template image to obtain a similarity coefficient matrix S, and acquiring the maximum value in all elements of the similarity coefficient matrix; S3, establishing a judgment index based on the similarity coefficient matrix and the maximum value; S4, calculating the judgment index to obtain its result, and the result of the judgment index is true or false; S5, if the result is true, acquiring a matching score based on a set weight w and the maximum value, so that the matching score is greater than the maximum value; S6, comparing the matching score with a set score threshold, and judging whether the matching is successful according to the comparison result. By setting the judgment index for the similarity coefficient matrix, the true or false is calculated, and then the matching score is adjusted accordingly.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an image matching method. Background Technology

[0002] Image matching methods are used to match objects or scenes in different images. In existing image matching methods, template images and images to be matched are typically used, that is, searching for feature patterns in the image to be matched that are basically the same as the feature patterns in the template image (the matching score of the template image and the image to be matched is greater than or equal to the score threshold).

[0003] In practical applications, a template image can be obtained by acquiring an image of a sample (such as a wafer) when the feature pattern is standard and clear. However, for the image to be matched, the feature pattern may be blurred or distorted due to factors such as drift in the operating parameters of the image acquisition equipment or changes in the sample's process. When the template image is used to match the image to be matched containing blurred or distorted feature patterns, a low matching score may occur, affecting the comparison result between the matching score and the score threshold, and consequently affecting the accuracy of the matching result (successful or unsuccessful matching), easily leading to misjudgment of the matching result. For example, for an image to be matched with blurred or distorted feature patterns, if the degree of blurring or distortion is low, the matching result between the template image and the image to be matched should be a successful match. However, because the matching score is low, it is less than the score threshold, resulting in a failed match and a misjudgment of the matching result.

[0004] Therefore, it is necessary to provide an image matching method to solve the above-mentioned problems existing in the prior art. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned defects in the prior art and provide an image matching method.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] An image matching method includes the following steps:

[0008] S1. Obtain template image t, and acquire a matching image f that is larger than the size of the template image t;

[0009] S2. Calculate the similarity between the image to be matched f and the template image t to obtain a similarity coefficient matrix S, and obtain the maximum value S among all elements in the similarity coefficient matrix S. max ;

[0010] S3. Based on the similarity coefficient matrix S and its maximum value S max Establish judgment indicators;

[0011] S4. Calculate the judgment index to obtain its result, and the result of the judgment index is true or false.

[0012] S5. If the result is false, then the maximum value S is... max As the matching score, if the result is true, it is based on the set weight w and the maximum value S. max Obtain a matching score Score such that the matching score Score is greater than the maximum value S. max ;

[0013] S6. Compare the matching score with the set score threshold, and determine whether the match is successful based on the comparison result.

[0014] Preferably, in step S3, based on the similarity coefficient matrix S and its maximum value S... max Establish at least two judgment indicators.

[0015] Preferably, in step S5, if all results are false, then the maximum value S is set to... max As a matching score, Score is used; otherwise, it is based on the set weight w and the maximum value S. max Obtain a matching score Score such that the matching score Score is greater than the maximum value S. max .

[0016] Preferably, in step S5, the matching score is obtained using the following formula:

[0017] Score = S max ×w

[0018] Where w > 1.

[0019] Preferably, in step S4, the number of true results (cnt) among all results is obtained; in step S5, if all results are false, the maximum value S is set to... max As the matching score, otherwise, based on the quantity cnt, weight w, and maximum value S max Obtain a matching score Score such that the matching score Score is greater than the maximum value S. max .

[0020] Preferably, in step S5, the matching score is obtained using the following formula:

[0021] Score = S max ×w cnt

[0022] Where w > 1.

[0023] Preferably, of which 1 <w<1.5。

[0024] Preferably, both the template image t and the image to be matched f are images of wafers; in step S2, the similarity is calculated based on the reciprocal algorithm of mean square error or the normalized cross-correlation coefficient algorithm.

[0025] Preferably, in step S2, the maximum value S is obtained. max The position p in the similarity coefficient matrix S; in step S6, when a match is successful, the matching position of the template image t in the image to be matched f is obtained according to the position p.

[0026] Preferably, the judgment index is at least two of the indices Idx1, Idx2, Idx3 and Idx4 defined by the following formulas (1)-(4):

[0027]

[0028] Where k1 represents a preset first threshold and is a positive number greater than 1, and avg(S>0) represents the average value of all elements greater than 0 in the similarity coefficient matrix S.

[0029]

[0030] Where k2 represents a preset second threshold and is a positive number greater than 1, and mid(S>0) represents the median of all elements greater than 0 in the similarity coefficient matrix S;

[0031]

[0032] Where k3 represents a preset third threshold, and is a positive number, std(S p±j ) indicates that the calculation is performed with the maximum value S. max The position p in the similarity coefficient matrix S is centered at (2j+1). 2 The standard deviation of each element, 1≤j≤3, is an integer;

[0033]

[0034] The ratio is calculated using equation (5);

[0035]

[0036] Among them, grad x (S p±l ) indicates that the calculation is performed with the maximum value S. max The position p is centered at (2l+1) 2 The gradient of each element along the x-direction; grad y (Sp±l ) indicates that the calculation is performed with the maximum value S. max The position p is centered at (2l+1) 2 The gradient of each element along the y-direction, where the x and y directions are perpendicular to each other; l ≥ 5 and is an integer, and sum is the summation function.

[0037] The beneficial effect of this invention is that by setting a similarity coefficient matrix S and its maximum value S... max The judgment index is used to calculate its truth value. Based on the calculation result, the matching score is increased, which can increase the probability of successful matching, reduce the false judgment rate of matching results, and make the tolerance for blurring or deformation of feature patterns in the matching image higher and the robustness better. Attached Figure Description

[0038] Figure 1 This is a flowchart of an embodiment of the present invention;

[0039] Figure 2 The image T is a clear and standard image of a feature pattern acquired in one embodiment of the present invention;

[0040] Figure 3 This refers to a template image t extracted from image T in one embodiment of the present invention.

[0041] Figure 4a This is an image to be matched acquired in one embodiment of the present invention;

[0042] Figure 4b This is another image to be matched, acquired in one embodiment of the present invention;

[0043] Figure 4c This is another image to be matched, acquired in one embodiment of the present invention;

[0044] Figure 5a It indicated that Figure 4a The matching score and the searched feature pattern are obtained by directly matching the image to be matched with the template image.

[0045] Figure 5b It indicated that Figure 4b The matching score and the searched feature pattern are obtained by directly matching the image to be matched with the template image.

[0046] Figure 5c It indicated that Figure 4c The matching score and the searched feature pattern are obtained by directly matching the image to be matched with the template image.

[0047] Figure 6a This illustrates the use of the image matching method provided in the embodiments of the present invention to... Figure 4aThe matching score and the searched feature pattern obtained by matching the image to be matched with the template image;

[0048] Figure 6b This illustrates the use of the image matching method provided in the embodiments of the present invention to... Figure 4b The matching score and the searched feature pattern obtained by matching the image to be matched with the template image;

[0049] Figure 6c This illustrates the use of the image matching method provided in the embodiments of the present invention to... Figure 4c The matching score and the searched feature pattern are obtained by matching the image to be matched with the template image. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but does not exclude other elements or objects.

[0051] like Figure 1 As shown, this embodiment of the invention provides an image matching method, including the following steps:

[0052] S1. Obtain the template image t, and acquire the matching image f, which is larger than the template image t.

[0053] S2. Calculate the similarity between the image to be matched f and the template image t, obtaining a similarity coefficient matrix S. Then, obtain the maximum value S among all elements in the similarity coefficient matrix S. max ; where the element is the similarity coefficient.

[0054] S3, Based on the similarity coefficient matrix S and its maximum value S max Establish judgment indicators;

[0055] S4. Calculate the judgment indicator to obtain its result, and judge whether the result of the judgment indicator is true or false.

[0056] S5. If the result is false, then the maximum value S is... max As a matching score, if the result is true, it is based on the set weight w and the maximum value S. maxGet the matching score Score such that the matching score Score is greater than the maximum value S. max ;

[0057] S6. Compare the matching score with the set score threshold, and determine whether the match is successful based on the comparison result.

[0058] The beneficial effect of this invention is that by setting a similarity coefficient matrix S and its maximum value S... max The judgment index is used to calculate its truth value, and the matching score is increased based on the calculation result (in the existing technology, the maximum value S). max To match scores, in this embodiment, the scores are based on weight w and maximum value S. max Obtaining a higher matching score (Score) can increase the probability of successful matching, reduce the false positive rate of matching results, and make the matching image more tolerant of blurring or deformation of feature patterns, resulting in better robustness.

[0059] For example, both the template image t and the image to be matched f are images of a wafer.

[0060] In some embodiments, the template image t is pre-stored in the working menu of the image acquisition device (such as an optical microscope or a scanning electron microscope), so the template image t can be obtained from the working menu.

[0061] In some embodiments, please refer to Figure 2 and Figure 3 The method involves acquiring an image T of size a×b on a wafer using an image acquisition device. This image contains standard and clear feature patterns. A smaller region (smaller than the size of image T) including these feature patterns is then cropped from image T as a template image t. It should be noted that the smaller the size of the cropped template image t, while including the feature patterns, the better, as this reduces computational complexity and improves matching accuracy. The feature patterns are unique patterns within image T. For example, a feature pattern might be a cross-shaped alignment mark on the wafer.

[0062] In some embodiments, in step S3, based on the similarity coefficient matrix S and its maximum value S max Establish at least two judgment indicators.

[0063] In some embodiments, in step S5, if all results are false, then the maximum value S is set to... max The score is used as the matching score; otherwise, it is based on the set weight w and the maximum value S. max Get the matching score Score such that the matching score Score is greater than the maximum value S. max .

[0064] In some embodiments, in step S5, the matching score is obtained using the following formula:

[0065] Score = w max ×w

[0066] Where w > 1.

[0067] In some embodiments, in step S4, the number of true results (cnt) among all results is obtained; for example, if the judgment indicators are set to 4, and three judgment indicators are true, then cnt = 3; in step S5, if all results are false, then the maximum value S is set to... max As a matching score, Score is used; otherwise, it is based on the number cnt, weight w, and maximum value S. max Get the matching score Score such that the matching score Score is greater than the maximum value S. max .

[0068] In some embodiments, in step S5, the matching score is obtained using the following formula:

[0069] Score = S max ×w cnt

[0070] Where w > 1. For example, if the judgment indicators are set to 4, and the calculation results of three judgment indicators are true, then cnt = 3, then:

[0071] Score = S max ×w 3

[0072] In some embodiments, to avoid the matching score from being too high, an upper limit is also set for the weight w, where 1 <w<1.5。

[0073] In some embodiments, in step S2, similarity is calculated based on the reciprocal algorithm of mean square error or the normalized cross-correlation coefficient algorithm.

[0074] In some embodiments, in step S2, the maximum value S is obtained. max The position p in the similarity coefficient matrix S; in step S6, when a match is successful, the matching position of the template image t in the image to be matched f is obtained based on the position p. For example, if the similarity coefficient matrix S and the image to be matched f have the same number of rows and columns, then the corresponding position in the image to be matched f obtained based on the position p is the matching position.

[0075] In some embodiments, the judgment index is at least two of the indices Idx1, Idx2, Idx3 and Idx4 defined by the following formulas (1)-(4):

[0076]

[0077] Where k1 represents a preset first threshold and is a positive number greater than 1, and avg(S>0) represents the average value of all elements greater than 0 in the similarity coefficient matrix S.

[0078]

[0079] Where k2 represents a preset second threshold and is a positive number greater than 1, and mid(S>0) represents the median of all elements greater than 0 in the similarity coefficient matrix S;

[0080]

[0081] Where k3 represents a preset third threshold, and is a positive number, std(S p±j ) indicates that the calculation is performed with the maximum value S. max The position p in the similarity coefficient matrix S is centered at (2j+1). 2 The standard deviation of each element, 1≤j≤3, is an integer;

[0082]

[0083] The ratio is calculated using equation (5);

[0084]

[0085] Among them, grad x (S p±l ) indicates that the calculation is performed with the maximum value S. max The position p is centered at (2l+1) 2 The gradient of each element along the x-direction; grad y (S p±l ) indicates that the calculation is performed with the maximum value S. max The position p is centered at (2l+1) 2 The gradient of each element along the y direction; l≥5 and is an integer, sum is the summation function, the x and y directions are two mutually perpendicular directions, for example, the x and y directions are the directions of the rows and columns of the matrix S respectively, else represents the two cases of ratio≤0.5 and ratio≥2, that is, ratio≤0.5 or ratio≥2. In both cases, the judgment result of formula (4) is false.

[0086] In the formulas (1)-(4) above, true means true and false means false.

[0087] In some embodiments, combined with Figures 2-6cFurther details on the steps involved in the image matching method:

[0088] S1. Obtain the template image t, and acquire a matching image f that is larger than the template image t; in this embodiment, as... Figure 2 and Figure 3 As shown, an image T of size a×b is acquired, which contains standard and clear feature patterns. The feature patterns are... Figure 2 The cross-shaped pattern shown in the image is represented by a smaller template image t, with dimensions a1×b1, extracted from image T, which includes the feature pattern. <a,b1<b;

[0089] For the image f to be matched, in this embodiment, as follows Figure 4a , Figure 4b and Figure 4c As shown, Figure 4a and Figure 4b The images f to be matched contain slightly deformed feature patterns and more deformed feature patterns, respectively. Figure 4c The image to be matched, f, does not contain a feature pattern that corresponds to the feature pattern in the template image, t.

[0090] S2. Calculate the similarity between the image to be matched f and the template image t, obtaining a similarity coefficient matrix S, and obtain the maximum value S in the matrix S. max And its position p; In this embodiment, the normalized cross-correlation coefficient algorithm is used to calculate the similarity;

[0091] S3, Based on the similarity coefficient matrix S and its maximum value S max Establish judgment indicators. In this embodiment, four judgment indicators are established and selected, specifically the indicators Idx1, Idx2, Idx3 and Idx4 defined by formulas (1)-(4) respectively;

[0092] S4: Calculate the judgment index to obtain its result, and judge whether the result of the judgment index is true or false; In this embodiment, for each judgment index with a true result, the count value cnt is incremented by 1. The initial value of cnt is 0. Calculate the results of all judgment indices and obtain the value of cnt. At this time, cnt is the number of true results among all results. The value range of cnt is [0,4] and is an integer within this range.

[0093] S5: Based on the value of cnt, a specific function is used to improve the matching score. In this embodiment, the specific function is:

[0094] Score = S max ×w cnt

[0095] Where Score is the final matching score, S maxis the maximum value in the similarity coefficient matrix S, cnt is the number of true results among all results, and w is a set weight; in this embodiment, the value of w is 1.1.

[0096] S6. Compare the matching score with the set score threshold, and determine whether the match is successful based on the comparison result.

[0097] refer to Figure 4a , Figure 4b and Figure 4c It can be seen Figure 4a The characteristic pattern in the image has been slightly deformed. Figure 4b The deformation of the characteristic patterns in the middle is relatively large, while Figure 4c If there is no corresponding feature pattern, and it is the same as the existing technology, the template image t is directly used and compared with... Figure 4a , Figure 4b and Figure 4c To calculate the matching score for the image f to be matched, please refer to [the provided text]. Figure 5a , Figure 5b and Figure 5c , Figure 5a , Figure 5b and Figure 5c The diagram illustrates the regions (represented by smaller rectangles) found by the template image t in the image to be matched f. These regions include feature patterns. The score in the diagram represents the matching score, which is the maximum value S in the similarity coefficient matrix S. max The matching scores are 0.83 (approximately 0.826205 to 0.83, and the same applies to other scores), 0.52, and 0.32, respectively. Figure 5b The displayed score is in a somewhat ambiguous position; judging from the matching score, it could be a mismatch or a correct match. For example, if the preset score threshold is 0.6, then for... Figure 4b and Figure 5b The image shown is intended for matching. Because 0.52 is less than 0.6, a match is deemed unsuccessful. However, the accuracy of this result is questionable, and a misjudgment may exist. In fact, from... Figure 4b As can be seen from the image f to be matched, although the feature pattern in the image f is greatly deformed, it is still similar to the feature pattern in the template image t. After image matching, the result should be a successful match, not a failed match.

[0098] In some embodiments, if the image matching method of this embodiment is used to match the template image t with... Figure 4a , Figure 4b and Figure 4c The matching score of the image f to be matched shown is calculated as follows: Figure 6a , Figure 6b and Figure 6cAs shown, Figure 6a , Figure 6b and Figure 6c The diagrams illustrate the regions (represented by smaller rectangles) found by the template image t in the image f to be matched. These regions include feature patterns. The scores in the diagrams represent the matching scores, which are 1.21, 0.76, and 0.32 respectively. Specifically, for... Figure 6c In the image to be matched, all judgment indicators are false, and the matching score is still the maximum value S. max The fraction remains unchanged; for Figure 6a and Figure 6b In the image to be matched, several judgment indicators are true, based on weight w and maximum value S respectively. max The matching score is obtained, therefore all matching scores are higher than the corresponding maximum value S. max Especially Figure 6b The matching score shown increased from 0.52 to 0.76. After comparing it with the score threshold (such as 0.6), a clearer judgment can be made. Figure 6b The feature pattern in the image is a correct match, meaning that a successful match has been obtained.

[0099] While embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations fall within the scope and spirit of the invention as set forth in the claims. Furthermore, the invention described herein may have other embodiments and can be implemented or carried out in various ways.

Claims

1. An image matching method, characterized in that, Includes the following steps: S1. Obtain template image t, and acquire a matching image f that is larger than the size of the template image t; S2. Calculate the similarity between the image to be matched f and the template image t to obtain a similarity coefficient matrix S, and obtain the maximum value S among all elements in the similarity coefficient matrix S. max ; S3. Based on the similarity coefficient matrix S and its maximum value S max Establish judgment indicators; S4. Calculate the judgment index to obtain its result, and the result of the judgment index is true or false; S5. If the result is false, then the maximum value S is... max As the matching score, if the result is true, it is based on the set weight w and the maximum value S. max Obtain a matching score Score such that the matching score Score is greater than the maximum value S. max ; S6. Compare the matching score with the set score threshold, and determine whether the match is successful based on the comparison result; The judgment index is at least two of the indices Idx1, Idx2, Idx3 and Idx4 defined by the following formulas (1)-(4): （1） Where k1 represents a preset first threshold and is a positive number greater than 1, and avg(S>0) represents the average value of all elements greater than 0 in the similarity coefficient matrix S. （2） Where k2 represents a preset second threshold and is a positive number greater than 1, and mid(S>0) represents the median of all elements greater than 0 in the similarity coefficient matrix S; （3） Where k3 represents a preset third threshold, and is a positive number. This indicates that the calculation is based on the maximum value S. max The position p in the similarity coefficient matrix S is centered The standard deviation of each element, 1≤j≤3, is an integer; （4） The ratio is calculated using equation (5); (5) in, This indicates that the calculation is based on the maximum value S. max The position p is centered The gradient of each element along the x-direction; This indicates that the calculation is based on the maximum value S. max The position p is centered The gradient of each element along the y-direction, where the x and y directions are perpendicular to each other; l ≥ 5 and is an integer, and sum is the summation function.

2. The method according to claim 1, characterized in that, In step S3, based on the similarity coefficient matrix S and its maximum value S... max Establish at least two judgment indicators.

3. The method according to claim 2, characterized in that, In step S5, if all results are false, then the maximum value S is set to... max As a matching score, Score is used; otherwise, it is based on the set weight w and the maximum value S. max Obtain a matching score Score such that the matching score Score is greater than the maximum value S. max .

4. The method according to claim 3, characterized in that, In step S5, the matching score is obtained using the following formula: Where w > 1.

5. The method according to claim 2, characterized in that, In step S4, the number of true results (cnt) is obtained out of all results; in step S5, if all results are false, the maximum value S is set to... max As the matching score, otherwise, based on the quantity cnt, weight w, and maximum value S max Obtain a matching score Score such that the matching score Score is greater than the maximum value S. max .

6. The method according to claim 5, characterized in that, In step S5, the matching score is obtained using the following formula: Where w > 1.

7. The method according to claim 6, characterized in that, in, 1<w<1.5。 8. The method according to claim 1, characterized in that, The template image t and the image to be matched f are both images of wafers; in step S2, the similarity is calculated based on the reciprocal algorithm of mean square error or the normalized cross-correlation coefficient algorithm.

9. The method according to claim 1, characterized in that, In step S2, the maximum value S is obtained. max The position p in the similarity coefficient matrix S; in step S6, when a match is successful, the matching position of the template image t in the image to be matched f is obtained according to the position p.

Images