Image registration and alignment method and device, equipment and storage medium

Abstract

The invention discloses an image registration and alignment method, device and equipment and a storage medium, and can be applied to the technical field of image processing, and the method comprises the steps: carrying out the partitioning of a template image, and obtaining n template blocks; n is greater than or equal to 2; partitioning a to-be-matched image according to the n template blocks to obtain n to-be-matched blocks; each template block is provided with a unique corresponding block to be matched; determining valid blocks in the n template blocks based on the texture degree; matching the effective block with a corresponding block to be matched, and obtaining a matching result value; and aligning the image to be matched with the template image according to the matching result value. Thus, the image is partitioned, the effective blocks with clearer textures are selected according to the texture degree to be matched with the to-be-matched blocks, the obtained more accurate matching result value is used for image registration and alignment, and the accuracy of image registration and alignment is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image registration and alignment method, apparatus, device, and storage medium. Background Art

[0002] In the manufacturing process of semiconductor devices, registration and alignment is a core link to ensure circuit functionality and yield, especially in processes such as lithography, interconnection and three-dimensional integration.

[0003] Traditional registration and alignment methods use whole-image matching, directly comparing a template image corresponding to a semiconductor die with the image to be matched. This then determines the offset between the two and ultimately achieves registration and alignment. However, with the increasing variety of semiconductor devices, the frequency of localized periodic textures or weak textures appearing in the semiconductor die images used for matching is increasing. Such textures are prone to mismatching during image registration. When such textured areas occupy a large proportion of the die, they can easily lead to mismatches across the entire image, reducing the accuracy of image registration and alignment.

[0004] Therefore, how to improve the accuracy of image registration and alignment is an urgent problem that those skilled in the art need to solve. Summary of the Invention

[0005] Based on the above problems, the present application provides an image registration and alignment method, device, equipment and storage medium. By dividing the image into blocks, and then selecting valid blocks with clearer textures to match with the blocks to be matched according to the texture degree, the more accurate matching result values obtained are used for image registration and alignment, thereby improving the accuracy of image registration and alignment.

[0006] In a first aspect, an embodiment of the present application provides an image registration and alignment method, comprising:

[0007] Divide the template image into blocks to obtain n template blocks, where n is greater than or equal to 2;

[0008] Divide the image to be matched into blocks according to the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched;

[0009] Determining a valid block among the n template blocks based on texture;

[0010] Matching the valid block with the corresponding block to be matched, and obtaining a matching result value;

[0011] The image to be matched is aligned with the template image according to the matching result value.

[0012] Optionally, dividing the image to be matched according to the n template blocks includes:

[0013] Determining a reference position and a reference size corresponding to each template block in the n template blocks;

[0014] Expanding the reference size corresponding to each template block based on a preset search range to obtain a target matching size corresponding to each template block;

[0015] The image to be matched is divided into blocks based on the reference position and the target matching size.

[0016] Optionally, determining a valid block among the n template blocks based on texture includes:

[0017] Divide each template block in the n template blocks into blocks, and obtain m*m calculation blocks corresponding to each template block; m≥2;

[0018] Determining the texture degree of each calculation block corresponding to each template block;

[0019] Determining the number of computing blocks whose texture degree is greater than a first preset threshold and corresponds to the same template block;

[0020] The template blocks corresponding to the calculation blocks whose number is greater than the second preset threshold are regarded as valid blocks;

[0021] The second preset threshold is greater than or equal to m.

[0022] Optionally, determining the texture of each calculation block corresponding to each template block includes:

[0023] Determine the maximum grayscale value, minimum grayscale value and grayscale mean value of each calculation block corresponding to each template block;

[0024] Determining the texture of each calculation block using a texture metric formula based on the maximum grayscale value, the minimum grayscale value, and the grayscale mean;

[0025] The texture measurement formula is: texture degree=(maximum grayscale value-minimum grayscale value) / grayscale mean.

[0026] Optionally, matching the valid block with the corresponding block to be matched and obtaining a matching result value includes:

[0027] Matching the valid block with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score;

[0028] The matching score greater than the third preset threshold is used as a target matching value, and the offset corresponding to the target matching value is recorded.

[0029] Optionally, aligning the image to be matched with the template image according to the matching result value includes:

[0030] Using the valid block corresponding to the target matching value as the alignment block;

[0031] If each alignment block corresponds to only one target matching value, cluster the alignment blocks according to the offset, take the category with the largest number as the valid result category, and use the alignment block corresponding to the valid result category as the target block;

[0032] The image to be matched is aligned with the template image based on the offset corresponding to the target block.

[0033] Optionally, aligning the image to be matched with the template image based on the offset corresponding to the target block includes:

[0034] Determine the weighted average of the offsets corresponding to each target block;

[0035] The image to be matched is aligned with the template image based on the weighted average value.

[0036] Optionally, aligning the image to be matched with the template image according to the matching result value includes:

[0037] Using the valid block corresponding to the target matching value as the alignment block;

[0038] If there are at least two target matching values corresponding to the alignment block, determine the target matching value closest to the center, and use the offset corresponding to the target matching value closest to the center as the target offset value;

[0039] Performing clustering processing on the alignment blocks according to the target offset values, taking the category with the largest number as the valid result category, and taking the alignment blocks corresponding to the valid result category as the target blocks;

[0040] The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.

[0041] In a second aspect, an embodiment of the present application provides an image registration and alignment device, comprising:

[0042] The first block division module is used to divide the template image into blocks to obtain n template blocks, where n is greater than or equal to 2;

[0043] A second block division module is used to divide the to-be-matched image into blocks according to the n template blocks to obtain n to-be-matched blocks; each template block has a unique corresponding to-be-matched block;

[0044] A valid determination module, configured to determine a valid block among the n template blocks based on texture;

[0045] A matching calculation module, used to match the valid block with the corresponding block to be matched and obtain a matching result value;

[0046] An alignment module is used to align the image to be matched with the template image according to the matching result value.

[0047] In a third aspect, an embodiment of the present application provides an image registration and alignment device, comprising:

[0048] memory for storing computer programs;

[0049] A processor is configured to implement the steps of the image registration and alignment method described above when executing the computer program.

[0050] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the image registration and alignment method as described above are implemented.

[0051] It can be seen from the above technical solutions that compared with the existing technology, this application has the following advantages:

[0052] This application first divides the template image into blocks to obtain n template blocks. Where n ≥ 2. Then the image to be matched is divided into blocks according to the n template blocks to obtain n blocks to be matched. Wherein, each template block has a unique corresponding block to be matched. Then, based on the texture degree, the valid blocks in the n template blocks are determined; the valid blocks are matched with the corresponding blocks to be matched, and the matching result values are obtained. Finally, the image to be matched is aligned with the template image according to the matching result values. In this way, by dividing the image into blocks, and then selecting the valid blocks with clearer textures to match with the blocks to be matched according to the texture degree, the more accurate matching result values obtained are used for image registration and alignment, thereby improving the accuracy of image registration and alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a local texture provided in an embodiment of the present application;

[0054] Figure 2 A flowchart of an image registration and alignment method provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a template image divided into blocks provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a camera capturing an image provided in an embodiment of the present application;

[0057] Figure 5 A schematic structural diagram of an image registration and alignment device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] As mentioned above, existing registration methods suffer from low image registration accuracy. Specifically, with the increasing variety of semiconductor devices, the frequency of localized periodic textures or weak textures appearing in semiconductor die images used for matching is increasing. Such textures are prone to mismatching during image registration. When such textured areas occupy a large proportion of the die, they can easily lead to mismatches across the entire image, further reducing image registration accuracy.

[0059] To address the above-mentioned issues, an embodiment of the present application provides an image registration and alignment method. The method first divides a template image into blocks to obtain n template blocks, where n ≥ 2. The image to be matched is then divided into blocks based on the n template blocks to obtain n blocks to be matched. Each template block has a unique corresponding block to be matched. A valid block among the n template blocks is then determined based on texture; the valid block is matched with the corresponding block to be matched, and a matching result value is obtained. Finally, the image to be matched is aligned with the template image based on the matching result value.

[0060] In this way, by dividing the image into blocks, and then selecting valid blocks with clearer textures to match with the blocks to be matched according to the texture degree, the more accurate matching result values obtained are used for image registration and alignment, thereby improving the accuracy of image registration and alignment.

[0061] It should be noted that the image registration and alignment method, apparatus, device, and storage medium provided in this application can be applied to the field of image processing technology. The above is only an example and does not limit the application field of the image registration and alignment method, apparatus, device, and storage medium provided in this application.

[0062] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] Figure 1 A local texture diagram provided in an embodiment of the present application. Figure 1 As shown, Figure 1In the figure, a represents a local periodic texture, and b represents a local weak texture. These two textures are easily confused and have unclear distinctions between light and dark. When either of these textures appears on a semiconductor die image used for matching, mismatching is likely to occur. When such textured areas occupy a large proportion of the die, mismatching of the entire image can occur, reducing the accuracy of image registration.

[0064] Figure 2 A flowchart of an image registration and alignment method provided in an embodiment of the present application. Figure 2 As shown, an image registration and alignment method provided in an embodiment of the present application may include:

[0065] S201: Divide the template image into blocks to obtain n template blocks; where n≥2.

[0066] In practical applications, a template image is an image of a semiconductor die (bare crystal) captured by a camera. This template image serves as a reference image, and the coordinates of each point on it serve as reference coordinates. Any image to be matched subsequently needs to be aligned with this template image. To improve the accuracy of image registration, this embodiment of the application proposes a block alignment method. First, the selected template image is divided into blocks using pre-set block widths and block heights, resulting in n template blocks. Figure 3 A schematic diagram of a template image provided in an embodiment of the present application. Figure 3 As shown, the red line divides the template image into 20 blocks, and the size of each block may vary. It's understandable that the number of template blocks is related to the size of the template image: smaller images correspond to fewer blocks, while larger images correspond to more blocks. The size of each template block is also solely dependent on the pre-set block width and height: larger template blocks are generated when the pre-set block width and height are larger.

[0067] S202: Divide the image to be matched into blocks according to the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched.

[0068] In practical applications, the image to be matched and the template image should correspond to the same die model. The template image is divided into several template blocks, and the image to be matched must be divided into several corresponding matching blocks. To achieve a corresponding match, each template block must correspond to only one matching block.

[0069] In addition, since the methods of dividing the image to be matched are different, the embodiment of the present application can illustrate a possible dividing method.

[0070] In one embodiment, dividing the image to be matched according to the n template blocks specifically includes:

[0071] Determining a reference position and a reference size corresponding to each template block in the n template blocks;

[0072] Expanding the reference size corresponding to each template block based on a preset search range to obtain a target matching size corresponding to each template block;

[0073] The image to be matched is divided into blocks based on the reference position and the target matching size.

[0074] In actual applications, the position of the camera in the machine is generally fixed. When the process is in progress, we want the bare crystal to stay in the "required position", and the camera captures the image of the bare crystal in this position, and the result is a template image. The bare crystal that subsequently enters the machine may not be in the "required position", but because the camera position remains unchanged, the area captured is still the area of the template image. It is understandable that due to the change in the position of the bare crystal, the features in the later captured image are far different from the features in the template image, which in turn affects the matching. To this end, the present application can make the features of the captured image richer by changing the focal length of the lens. Figure 4 A schematic diagram of a camera capturing an image provided in an embodiment of the present application. Figure 4 As shown, Figure 4 In the figure, a represents the image to be matched, and b represents the template image. The blue frame in a represents the scaled area of the template image, meaning that the texture features in the blue frame are identical to those in the template image. It should be understood that if the die corresponding to the image to be matched is aligned with the die corresponding to the template image, the center of the blue frame should align with the center of the image to be matched. Misalignment indicates a certain offset between the two images. Scaling the template image on the image to be matched means that the image to be matched is expanded by a certain size on the basis of the original template image. This size constitutes the preset search range for matching. When segmenting the image to be matched, the blue frame area is first segmented in the same proportion as the template image, based on the reference positions and reference sizes of each template block on the template image. This results in n initial blocks, each of which corresponds to a unique template block. The texture features of the two corresponding blocks should be identical, differing only in the scaling ratio. Furthermore, assuming the preset search range is 100 pixels, this means that the texture features of the template image are expanded by 50 pixels on the basis of the original template image. Assuming the base size of each template block is 100*50 pixels, the corresponding size of the expanded block in the expanded template image is 200*150 pixels, i.e., the target matching size is 200*150 pixels. The image to be matched is then divided into blocks based on the target matching size. It is understood that when dividing the image to be matched into blocks, each block to be matched corresponds to an expanded block, i.e., the texture features of each block to be matched should be consistent with its corresponding expanded block.

[0075] S203: Determine a valid block among the n template blocks based on the texture.

[0076] In practical applications, a valid block refers to a template block with more obvious features. The embodiment of the present application introduces the concept of texture, and determines the valid block among the obtained n template blocks by texture.

[0077] In addition, since the methods for determining valid blocks are different, the embodiments of the present application can illustrate a possible determination method.

[0078] In one case, S203: determining a valid block among the n template blocks based on texture, specifically includes:

[0079] Divide each template block in the n template blocks into blocks, and obtain m*m calculation blocks corresponding to each template block; m≥2;

[0080] Determining the texture degree of each calculation block corresponding to each template block;

[0081] Determining the number of computing blocks whose texture degree is greater than a first preset threshold and corresponds to the same template block;

[0082] The template blocks corresponding to the calculation blocks whose number is greater than the second preset threshold are regarded as valid blocks;

[0083] The second preset threshold is greater than or equal to m.

[0084] In practical applications, the block-based approach can be continued, further dividing each template block into several smaller blocks (calculation blocks), and then determining the texture degree of each calculation block. Among the multiple calculation blocks corresponding to a template block, if the number of calculation blocks with a texture degree greater than a first preset threshold is greater than a second preset threshold, then the template block is considered a valid block. The second preset threshold is greater than or equal to m, and is generally set to m. For example, taking n=9, the template image is divided into nine equal-sized template blocks. Assuming m=3 (each template block corresponds to nine calculation blocks), the first preset threshold is 0.3, and the second preset threshold is 3. If only three or fewer calculation blocks in a template block have a texture degree greater than 0.3, and the remaining six or more calculation blocks have a texture degree less than 0.3, then the template image is marked as an invalid block. Similarly, if four or more calculation blocks in a template block have a texture degree greater than 0.3, and the remaining six or fewer calculation blocks have a texture degree less than 0.3, then the template image is marked as a valid block. It will be understood that the first preset threshold mentioned in the embodiments of this application is an empirical value. The first preset threshold is obtained by pre-selecting image regions with various textures and conducting experiments. Specifically, the texture value of each pre-selected image region with various textures is calculated using a texture degree calculation formula. Extensive experiments have shown that image regions with a texture degree greater than 0.3 have a low probability of template matching failure. However, when the texture degree is less than 0.3 or less, the texture is no longer noticeable, the image becomes flat, and template matching failures increase significantly.

[0085] In addition, since there are different ways to calculate texture, the embodiment of the present application can illustrate a possible calculation method.

[0086] In one embodiment, determining the texture of each calculation block corresponding to each template block includes:

[0087] Determine the maximum grayscale value, minimum grayscale value and grayscale mean value of each calculation block corresponding to each template block;

[0088] Determining the texture of each calculation block using a texture metric formula based on the maximum grayscale value, the minimum grayscale value, and the grayscale mean;

[0089] The texture measurement formula is: texture degree=(maximum grayscale value-minimum grayscale value) / grayscale mean.

[0090] In practical applications, texture can be calculated using a texture metric formula. Specifically, first, a template block is taken and the maximum, minimum, and mean grayscale values of each calculation block are calculated. Then, the texture metric formula (texture = (maximum grayscale - minimum grayscale) / mean grayscale) is used to determine the texture of each calculation block. The resulting texture values are then combined with the valid block determination method described above to determine whether each template block is valid.

[0091] S204: Match the valid block with the corresponding block to be matched, and obtain a matching result value.

[0092] In practical applications, the template image is divided into n template blocks, and the image to be matched is divided into n matching blocks. Each template block has a unique corresponding matching block, and each matching block should include all the texture features of the corresponding template block as well as the texture features expanded from the template block. A valid block is a block with more distinct texture features selected from all template blocks. Matching this valid block with its corresponding matching block can result in a more accurate matching result.

[0093] In addition, since the methods for obtaining the matching result value are different, the embodiment of the present application can illustrate one possible method for obtaining it.

[0094] In one case, S204: matching the valid block with the corresponding block to be matched and obtaining a matching result value specifically includes:

[0095] Matching the valid block with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score;

[0096] The matching score greater than the third preset threshold is used as a target matching value, and the offset corresponding to the target matching value is recorded.

[0097] In practical applications, the NCC (Normalized Cross Correlation Coefficient) matching algorithm can be used to match template blocks with their corresponding valid blocks, although other matching algorithms are also possible. The resulting matching result includes a match score and an offset. Thus, a third preset threshold can be set to filter valid blocks with high match scores and mark them as successfully matched. Specifically, using the example of n=9, there are nine template blocks. Assume blocks 1, 3, 6, 7, and 9 are valid blocks. After matching, the matching scores for block 1 are 0.8, 0.3, 0.7, 0.7, and 0.9, respectively. If the third preset threshold is set to 0.6, then blocks 1, 6, 7, and 9 are marked as successfully matched, and their corresponding values of 0.8, 0.7, 0.7, and 0.9 are used as target match values. The corresponding offsets are also recorded. For example, when the first valid block is matched with its corresponding block to be matched, the offset between the area where its target matching value is 0.8 and the first valid block is x: -2, y: -2 (that is, the block to be matched is moved two pixels to the left and two pixels downward compared to the first valid block).

[0098] S205: Align the image to be matched with the template image according to the matching result value.

[0099] In practice, the matching result value indicates the matching score and offset corresponding to the successfully matched block in the valid block. Combined with the offset, the image to be matched can be aligned with the template image.

[0100] In addition, since the methods of aligning the image to be matched with the template image are different, the embodiment of the present application can illustrate a possible alignment method.

[0101] In one case, S205: aligning the image to be matched with the template image according to the matching result value may specifically include:

[0102] Using the valid block corresponding to the target matching value as the alignment block;

[0103] If each alignment block corresponds to only one target matching value, cluster the alignment blocks according to the offset, take the category with the largest number as the valid result category, and use the alignment block corresponding to the valid result category as the target block;

[0104] The image to be matched is aligned with the template image based on the offset corresponding to the target block.

[0105] In practical applications, when a valid block is matched with a block to be matched, it matches multiple regions within the block to be matched and obtains corresponding matching scores. When the matching score of a region with the valid block exceeds a third preset threshold, the valid block is marked as a successfully matched block, the matching score is recorded as the target matching value, and the offset corresponding to the target matching value is recorded. Successfully matched valid blocks can be recorded as alignment blocks, and each alignment block has a corresponding target matching value and offset. Generally, an alignment block only corresponds to one target matching value and offset. At this point, alignment of the image to be matched with the template image is achieved based on clustering of matching results. Specifically, continuing with the example of n=9, assuming that blocks 1, 3, 6, 7, and 9 are valid blocks, of which blocks 1, 6, 7, and 9 are successfully matched and recorded as alignment blocks, their corresponding target matching values are 0.8, 0.7, 0.7, and 0.9, respectively, and their corresponding offsets are x:-2, y:-2, x:-3, y:-1, x:1, y:1, and x:-1, y:-3, respectively. It can be understood that the blocks to be matched corresponding to the 1st, 6th and 9th template blocks are offset to the lower left, and the blocks to be matched corresponding to the 7th template block are offset to the upper right, and then through k-means clustering, the category with the largest number is taken as the valid result category, that is, the offsets x: -2, y: -2, x: -3, y: -1 and x: -1, y: -3 of the pieces indicating that the blocks to be matched are offset to the lower left are retained, and the other results are marked as outliers. At this time, the 1st, 6th and 9th template blocks are recorded as target blocks. Finally, the center of the valid cluster is calculated by the offset corresponding to the target block, that is, the center values of the offsets x: -2, y: -2, x: -3, y: -1 and x: -1, y: -3 are calculated, and the image to be matched is aligned with the template image based on the center value. The center value can be directly obtained by averaging, so the average value obtained by combining the above offsets x:-2, y:-2, x:-3, y:-1 and x:-1, y:-3 is x:-2, y:-2.

[0106] In addition, since there are different ways to use offsets to align images, the embodiments of the present application can illustrate a possible alignment method.

[0107] In one embodiment, aligning the image to be matched with the template image based on the offset corresponding to the target block includes:

[0108] Determine the weighted average of the offsets corresponding to each target block;

[0109] The image to be matched is aligned with the template image based on the weighted average value.

[0110] In practical applications, the center value can also be obtained through weighted averaging. Continuing with the above example, the weight value corresponding to each template block can be pre-set. Then, after determining the 1st, 6th, and 9th template blocks as target blocks, the offsets corresponding to the 1st, 6th, and 9th template blocks are weighted averaged. Finally, the weighted average is used as the center value to align the image to be matched with the template image.

[0111] In addition, since the methods of aligning the image to be matched with the template image are different, the embodiment of the present application can illustrate another possible alignment method.

[0112] In one case, S205: aligning the image to be matched with the template image according to the matching result value may further include:

[0113] Using the valid block corresponding to the target matching value as the alignment block;

[0114] If there are at least two target matching values corresponding to the alignment block, determine the target matching value closest to the center, and use the offset corresponding to the target matching value closest to the center as the target offset value;

[0115] Performing clustering processing on the alignment blocks according to the target offset values, taking the category with the largest number as the valid result category, and taking the alignment blocks corresponding to the valid result category as the target blocks;

[0116] The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.

[0117] In actual applications, since the preset search range can be set, its value can be large or small, and the texture on the template image is as possible. It is understandable that when the preset search range is larger than the reference size of the template block, and there is a periodic texture in the template image, a valid block may have a matching score with multiple areas thereof that exceeds the third preset threshold when matching its corresponding block to be matched. In other words, each alignment block may correspond to more than one target matching value and offset. At this time, it is necessary to select the target matching value closest to the center from these multiple target matching values. Continuing with the above example of n=9, blocks 1, 3, 6, 7, and 9 are valid blocks, of which blocks 1, 6, 7, and 9 are successfully matched and are recorded as alignment blocks. Among them, block 1 corresponds to two target matching values, and the others correspond to only one. Suppose that when matching an alignment block with its corresponding unmatched block, it needs to match nine regions on the unmatched block. The matching scores with the second and fifth regions both exceed the third preset threshold, 0.8 and 0.8, respectively, and the corresponding offsets are x: -1, y: -2 and x: -2, y: -2, respectively. Assume that the center point of the unmatched block is (2, 2), and the center points of the second and fifth regions are (2, 1) and (2, 2), respectively. The fifth region is closest to the center point of the unmatched block, and its corresponding matching score is 0.8. Its corresponding offsets, x: -2, y: -2, are recorded as the target offsets. The unmatched image and the template image are then aligned based on the target offsets. Specifically, the target matching scores for the sixth, seventh, and ninth template blocks are 0.7, 0.7, and 0.9, respectively, and the corresponding offsets are x: -3, y: -1, x: 1, y: 1, and x: -1, y: -3, respectively. Through the above screening, the target matching value corresponding to the first template block is 0.8, and the corresponding offset is x: -2, y: -2, recorded as the target offset. Then, through k-means clustering, the first, sixth, and ninth template blocks are determined to be target blocks. Finally, the center of the valid cluster is calculated using the offsets corresponding to the target blocks (the target offset corresponding to the first template block and the offsets corresponding to the sixth, seventh, and ninth template blocks). That is, the center values of the offsets x: -2, y: -2, x: -3, y: -1, and x: -1, y: -3 are calculated. Based on these center values, the image to be matched is aligned with the template image. This center value can be obtained directly by averaging or by weighted averaging.

[0118] In summary, this solution, applied to image block alignment in semiconductor inspection, differs from conventional block alignment in image processing. This solution primarily targets images with locally periodic textures and weakly textured images. The main approach employed is to separate the template image and the image to be matched into blocks, resulting in n template blocks and n matching blocks. Where n ≥ 2, and each template block has a unique corresponding matching block. Then, valid blocks within the n template blocks are identified based on texture intensity. These valid blocks are matched with the corresponding matching blocks, and matching results are obtained. Regions with no or minimal texture are not matched. The matching results of the successfully matched blocks are then clustered to remove outliers and eliminate interference caused by mismatched local blocks. Finally, the cluster centers are used as the final matching results, and the image to be matched is then aligned with the template image. In this way, by segmenting the image and selecting valid blocks with clearer textures based on texture intensity for matching with the matching blocks, the resulting more accurate matching results are used for image registration, improving the accuracy of image registration.

[0119] Based on the image registration and alignment method provided in the above embodiment, the present application also provides an image registration and alignment device. The image registration and alignment device is described below in conjunction with the embodiments and drawings.

[0120] Figure 5 A schematic diagram of the structure of an image registration and alignment device provided in an embodiment of the present application. Figure 5 As shown, the image registration and alignment device 500 provided in the embodiment of the present application includes:

[0121] The first block division module 501 is used to divide the template image into blocks to obtain n template blocks, where n≥2;

[0122] A second block division module 502 is configured to divide the image to be matched into blocks according to the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched;

[0123] A valid determination module 503, configured to determine a valid block among the n template blocks based on texture;

[0124] A matching calculation module 504 is used to match the valid block with the corresponding block to be matched and obtain a matching result value;

[0125] The alignment module 505 is configured to align the image to be matched with the template image according to the matching result value.

[0126] As an implementation manner, regarding how to divide the matching image into blocks, the second block division module 502 may be specifically used to:

[0127] Determining a reference position and a reference size corresponding to each template block in the n template blocks;

[0128] Expanding the reference size corresponding to each template block based on a preset search range to obtain a target matching size corresponding to each template block;

[0129] The image to be matched is divided into blocks based on the reference position and the target matching size.

[0130] As an implementation manner, regarding how to determine the valid blocks in the template blocks, the valid determination module 503 may specifically include: a third block division module, a first determination submodule, a second determination submodule, and a screening module;

[0131] A third block division module is used to divide each template block in the n template blocks into blocks, and each template block corresponds to m*m calculation blocks; m≥2;

[0132] A first determining submodule is configured to determine the texture of each calculation block corresponding to each template block;

[0133] A second determining submodule is configured to determine the number of calculation blocks whose texture degree is greater than a first preset threshold and which correspond to the same template block;

[0134] The screening module is configured to select the template blocks corresponding to the calculation blocks whose number is greater than a second preset threshold as valid blocks; the second preset threshold is greater than or equal to m.

[0135] As an implementation manner, regarding how to determine the texture of each calculation block corresponding to each template block, the first determination submodule can be specifically used to:

[0136] Determine the maximum grayscale value, minimum grayscale value and grayscale mean value of each calculation block corresponding to each template block;

[0137] Determining the texture of each calculation block using a texture metric formula based on the maximum grayscale value, the minimum grayscale value, and the grayscale mean;

[0138] The texture measurement formula is: texture degree=(maximum grayscale value-minimum grayscale value) / grayscale mean.

[0139] As an implementation manner, regarding how to match a valid block with a corresponding block to be matched, the matching calculation module 504 may be specifically used to:

[0140] Matching the valid block with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score;

[0141] The matching score greater than the third preset threshold is used as a target matching value, and the offset corresponding to the target matching value is recorded.

[0142] As an implementation method, regarding how to align the image to be matched with the template image according to the matching result value, the alignment module 505 can be specifically used to:

[0143] Using the valid block corresponding to the target matching value as the alignment block;

[0144] If each alignment block corresponds to only one target matching value, cluster the alignment blocks according to the offset, take the category with the largest number as the valid result category, and use the alignment block corresponding to the valid result category as the target block;

[0145] The image to be matched is aligned with the template image based on the offset corresponding to the target block.

[0146] The aligning of the image to be matched with the template image based on the offset corresponding to the target block includes:

[0147] Determine the weighted average of the offsets corresponding to each target block;

[0148] The image to be matched is aligned with the template image based on the weighted average value.

[0149] As an implementation manner, regarding how to align the image to be matched with the template image according to the matching result value, the alignment module 505 may be further used to:

[0150] Using the valid block corresponding to the target matching value as the alignment block;

[0151] If there are at least two target matching values corresponding to the alignment block, determine the target matching value closest to the center, and use the offset corresponding to the target matching value closest to the center as the target offset value;

[0152] Performing clustering processing on the alignment blocks according to the target offset values, taking the category with the largest number as the valid result category, and taking the alignment blocks corresponding to the valid result category as the target blocks;

[0153] The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.

[0154] In summary, the present application first divides the template image into blocks to obtain n template blocks. Where n ≥ 2. Then, the image to be matched is divided into blocks according to the n template blocks to obtain n blocks to be matched. Wherein, each template block has a unique corresponding block to be matched. Then, the valid blocks in the n template blocks are determined based on the texture degree; the valid blocks are matched with the corresponding blocks to be matched, and the matching result values are obtained. Finally, the image to be matched is aligned with the template image according to the matching result values. In this way, by dividing the image into blocks, and then selecting the valid blocks with clearer textures to match with the blocks to be matched according to the texture degree, the more accurate matching result values obtained are used for image registration and alignment, thereby improving the accuracy of image registration and alignment.

[0155] In addition, the present application also provides an image registration and alignment device, comprising: a memory for storing a computer program; and a processor for implementing the steps of the image registration and alignment method described above when executing the computer program.

[0156] In addition, the present application also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image registration and alignment method described above are implemented.

[0157] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for image registration and alignment, characterized in that: The method comprises: Divide the template image into blocks to obtain n template blocks, where n is greater than or equal to 2; Divide the image to be matched into blocks according to the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched; Determining a valid block among the n template blocks based on texture; Matching the valid block with the corresponding block to be matched, and obtaining a matching result value; The image to be matched is aligned with the template image according to the matching result value.

2. The method according to claim 1, characterized in that The dividing the image to be matched into blocks according to the n template blocks comprises: Determining a reference position and a reference size corresponding to each template block in the n template blocks; Expanding the reference size corresponding to each template block based on a preset search range to obtain a target matching size corresponding to each template block; The image to be matched is divided into blocks based on the reference position and the target matching size.

3. The method according to claim 1, characterized in that The determining of a valid block among the n template blocks based on texture comprises: Divide each template block in the n template blocks into blocks, and obtain m*m calculation blocks corresponding to each template block; m≥2; Determining the texture degree of each calculation block corresponding to each template block; Determining the number of computing blocks whose texture degree is greater than a first preset threshold and corresponds to the same template block; The template blocks corresponding to the calculation blocks whose number is greater than the second preset threshold are regarded as valid blocks; The second preset threshold is greater than or equal to m.

4. The method according to claim 3, characterized in that Determining the texture of each calculation block corresponding to each template block includes: Determine the maximum grayscale value, minimum grayscale value and grayscale mean value of each calculation block corresponding to each template block; Determining the texture of each calculation block using a texture metric formula based on the maximum grayscale value, the minimum grayscale value, and the grayscale mean; The texture measurement formula is: texture degree=(maximum grayscale value-minimum grayscale value) / grayscale mean.

5. The method according to claim 1, wherein Matching the valid block with the corresponding block to be matched and obtaining a matching result value includes: Matching the valid block with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score; The matching score greater than the third preset threshold is used as a target matching value, and the offset corresponding to the target matching value is recorded.

6. The method according to claim 5, characterized in that The aligning the image to be matched with the template image according to the matching result value includes: Using the valid block corresponding to the target matching value as the alignment block; If each alignment block corresponds to only one target matching value, cluster the alignment blocks according to the offset, take the category with the largest number as the valid result category, and use the alignment block corresponding to the valid result category as the target block; The image to be matched is aligned with the template image based on the offset corresponding to the target block.

7. The method according to claim 6, characterized in that The aligning the image to be matched with the template image based on the offset corresponding to the target block includes: Determine the weighted average of the offsets corresponding to each target block; The image to be matched is aligned with the template image based on the weighted average value.

8. The method according to claim 5, characterized in that The aligning the image to be matched with the template image according to the matching result value includes: Using the valid block corresponding to the target matching value as the alignment block; If there are at least two target matching values corresponding to the alignment block, determine the target matching value closest to the center, and use the offset corresponding to the target matching value closest to the center as the target offset value; Performing clustering processing on the alignment blocks according to the target offset values, taking the category with the largest number as the valid result category, and taking the alignment blocks corresponding to the valid result category as the target blocks; The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.

9. An image registration and alignment device, characterized in that: include: The first block division module is used to divide the template image into blocks to obtain n template blocks; Said n≥2; A second block division module is used to divide the to-be-matched image into blocks according to the n template blocks to obtain n to-be-matched blocks; each template block has a unique corresponding to-be-matched block; A valid determination module, configured to determine a valid block among the n template blocks based on texture; A matching calculation module, used to match the valid block with the corresponding block to be matched and obtain a matching result value; An alignment module is used to align the image to be matched with the template image according to the matching result value.

10. An image registration and alignment device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the image registration and alignment method according to any one of claims 1 to 8 when executing the computer program.

11. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image registration and alignment method according to any one of claims 1 to 8.

Images