Image registration alignment method, device, equipment and storage medium
By segmenting the template image and selecting its texture, combined with the NCC matching algorithm and clustering, the mismatch problem caused by local textures in semiconductor devices is solved, thus improving the accuracy of image registration and alignment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU ZHONGKE FEICE TECHNOLOGY CO LTD
- Filing Date
- 2025-05-15
- Publication Date
- 2026-06-09
Smart Images
Figure CN120471895B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image registration and alignment method, apparatus, device, and storage medium. Background Technology
[0002] In the manufacturing process of semiconductor devices, registration and alignment are the core steps to ensure circuit functionality and yield, and are especially indispensable in processes such as photolithography, interconnection and 3D integration.
[0003] Traditional registration and alignment methods employ whole-image matching, directly comparing the template image corresponding to the semiconductor die with the image to be matched to determine the offset between them, ultimately achieving registration and alignment. However, with the increasing variety of semiconductor devices, the frequency of localized periodic textures or weak textures appearing on the semiconductor die images used for matching is also increasing. Such textures are prone to mismatches during image registration and alignment. When such textured regions occupy a large proportion of the die, it can easily lead to incorrect matching of the entire image, thereby reducing the accuracy of image registration and alignment.
[0004] Therefore, improving the accuracy of image registration and alignment is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] To address the aforementioned issues, this application provides an image registration and alignment method, apparatus, device, and storage medium. By dividing the image into blocks, and then selecting a more clearly textured effective block to be matched with the block to be matched based on the texture degree, the more accurate matching result value is used for image registration and alignment, thereby improving the accuracy of image registration and alignment.
[0006] In a first aspect, embodiments of this application provide an image registration and alignment method, comprising:
[0007] The template image is divided into blocks to obtain n template blocks; where n ≥ 2.
[0008] The image to be matched is divided into blocks based on the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched.
[0009] The effective blocks among the n template blocks are determined based on texture degree;
[0010] The valid block is matched with the corresponding block to be matched, and the matching result value is obtained;
[0011] The image to be matched is aligned with the template image based on the matching result value.
[0012] Optionally, the step of dividing the image to be matched into blocks based on the n template blocks includes:
[0013] Determine the reference position and reference size of each template block among the n template blocks;
[0014] Based on a preset search range, the reference size corresponding to each template block is expanded outward to obtain the target matching size corresponding to each template block;
[0015] The image to be matched is divided into blocks based on the reference position and the target matching size.
[0016] Optionally, determining the valid blocks among the n template blocks based on texture degree includes:
[0017] Each of the n template blocks is divided into blocks, and each template block corresponds to m*m calculation blocks; where m≥2;
[0018] Determine the texture degree of each computational block corresponding to each template block;
[0019] Determine the number of computation blocks whose texture degree is greater than a first preset threshold and which correspond to the same template block;
[0020] The template blocks corresponding to the number of calculation blocks that are greater than the second preset threshold are taken as valid blocks;
[0021] The second preset threshold is greater than or equal to m.
[0022] Optionally, determining the texture degree of each computational block corresponding to each template block includes:
[0023] Determine the maximum gray value, minimum gray value, and average gray value of each calculation block corresponding to each template block;
[0024] Based on the maximum gray value, the minimum gray value, and the average gray value, the texture degree of each computation block is determined using a texture metric formula;
[0025] The texture measurement formula is: Texture = (Maximum gray value - Minimum gray value) / Average gray value.
[0026] Optionally, the step of matching the valid block with the corresponding block to be matched and obtaining a matching result value includes:
[0027] The valid block is matched with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score;
[0028] The matching score that is greater than the third preset threshold is taken as the target matching value, and the offset corresponding to the target matching value is recorded.
[0029] Optionally, aligning the image to be matched with the template image based on the matching result value includes:
[0030] Use the valid block corresponding to the target matching value as the alignment block;
[0031] If each alignment block corresponds to only one target matching value, then the alignment blocks are clustered according to the offset, the category with the largest number is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block;
[0032] The image to be matched is aligned with the template image based on the offset corresponding to the target block.
[0033] Optionally, aligning the image to be matched with the template image based on the offset corresponding to the target block includes:
[0034] Determine the weighted average of the offsets corresponding to each target block;
[0035] The image to be matched is aligned with the template image based on the weighted average value.
[0036] Optionally, aligning the image to be matched with the template image based on the matching result value includes:
[0037] Use the valid block corresponding to the target matching value as the alignment block;
[0038] If there are alignment blocks corresponding to at least two target matching values, then the target matching value closest to the center is determined, and the offset corresponding to the target matching value closest to the center is used as the target offset value;
[0039] The alignment blocks are clustered according to the target offset value. The category with the largest number of clusters is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block.
[0040] The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.
[0041] Secondly, embodiments of this application provide an image registration and alignment apparatus, comprising:
[0042] The first segmentation module is used to segment the template image into n template blocks; where n ≥ 2.
[0043] The second segmentation module is used to segment the image to be matched into n blocks based on the n template blocks, so as to obtain n blocks to be matched; each template block has a unique corresponding block to be matched.
[0044] A valid determination module is used to determine the valid blocks among the n template blocks based on texture degree;
[0045] The matching calculation module is used to match the valid block with the corresponding block to be matched and obtain the matching result value;
[0046] An alignment module is used to align the image to be matched with the template image based on the matching result value.
[0047] Thirdly, embodiments of this application provide an image registration and alignment device, comprising:
[0048] Memory, used to store computer programs;
[0049] A processor for executing the computer program to implement the registration and alignment method for the image as described above.
[0050] Fourthly, embodiments of this application provide a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image registration and alignment method described above.
[0051] As can be seen from the above technical solutions, compared with the prior art, this application has the following advantages:
[0052] This application first divides the template image into blocks, obtaining n template blocks, where n ≥ 2. Then, based on these n template blocks, the image to be matched is divided into blocks, obtaining n blocks to be matched. Each template block has a unique corresponding block to be matched. Next, valid blocks are determined from the n template blocks based on texture quality; these valid blocks are matched with their corresponding blocks to be matched, and a matching result value is obtained. Finally, the image to be matched is aligned with the template image based on the matching result value. Thus, by dividing the image into blocks and then selecting valid blocks with clearer textures to match the blocks to be matched based on texture quality, more accurate matching result values are used for image registration and alignment, improving the accuracy of image registration and alignment. Attached Figure Description
[0053] Figure 1 A partial texture diagram provided for an embodiment of this application;
[0054] Figure 2 A flowchart illustrating an image registration and alignment method provided in an embodiment of this application;
[0055] Figure 3 This is a block diagram of a template image provided in an embodiment of this application;
[0056] Figure 4 A schematic diagram illustrating image acquisition by a camera, provided as an embodiment of this application;
[0057] Figure 5 This is a schematic diagram of the structure of an image registration and alignment device provided in an embodiment of this application. Detailed Implementation
[0058] As mentioned earlier, existing registration and alignment methods suffer from low accuracy in image registration and alignment. Specifically, with the increasing variety of semiconductor devices, the frequency of localized periodic textures or weak textures appearing on the semiconductor die images used for matching is also increasing. Such textures are prone to mismatches during image registration and alignment. When such textured regions occupy a large proportion of the die, it can easily lead to mismatches of the entire image, thereby reducing the accuracy of image registration and alignment.
[0059] To address the aforementioned problems, this application provides an image registration and alignment method. This method first divides a template image into blocks, obtaining n template blocks, where n ≥ 2. Then, based on the n template blocks, the image to be matched is divided into blocks, obtaining n blocks to be matched. Each template block has a unique corresponding block to be matched. Next, valid blocks are determined from the n template blocks based on texture; these valid blocks are matched with their corresponding blocks to be matched, and a matching result value is obtained. Finally, the image to be matched is aligned with the template image based on the matching result value.
[0060] In this way, by dividing the image into blocks, and then selecting the more textured effective blocks to be matched with the blocks to be matched based on the texture, the more accurate matching result values are used for image registration and alignment, thereby improving the accuracy of image registration and alignment.
[0061] It should be noted that the image registration and alignment method, apparatus, device, and storage medium provided in this application can be applied to the field of image processing technology. The above are merely examples and do not limit the application field of the image registration and alignment method, apparatus, device, and storage medium provided in this application.
[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0063] Figure 1 This is a schematic diagram of a local texture provided for an embodiment of this application. (In conjunction with...) Figure 1 As shown, Figure 1In the image, 'a' represents a local periodic texture, and 'b' represents a local weak texture. These two textures are easily confused and have poor contrast between light and dark areas. When either of these textures appears on the semiconductor die image used for matching, mismatches are likely to occur. Furthermore, when such textured areas occupy a large proportion of the die, it can lead to incorrect matching of the entire image, thereby reducing the accuracy of image registration and alignment.
[0064] Figure 2 This is a flowchart illustrating an image registration and alignment method provided in an embodiment of this application. (In conjunction with...) Figure 2 As shown in the embodiments of this application, an image registration and alignment method may include:
[0065] S201: Divide the template image into blocks to obtain n template blocks; where n≥2.
[0066] In practical applications, the template image is a semiconductor die (bare die) image captured by a camera. This template image serves as the reference image, and the coordinates of each point on it are the reference coordinates. Any subsequent image to be matched needs to be aligned with this template image. To improve the accuracy of image registration and alignment, this application proposes a block alignment method. First, the selected template image is divided into blocks using pre-set block widths and heights, resulting in n template blocks. Figure 3 This is a block diagram of a template image provided in an embodiment of this application. (In conjunction with...) Figure 3 As shown, the red line divides the template image into 20 blocks, each of which may vary in size. It's understandable that the number of template blocks is related to the size of the template image; a smaller image corresponds to fewer blocks, and a larger image corresponds to more blocks. Furthermore, the size of each template block is only related to pre-defined values; a larger pre-defined block width and height result in a larger template block.
[0067] S202: Divide the image to be matched into blocks according to the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched.
[0068] In practical applications, the image to be matched and the template image should correspond to the same type of bare die. The template image is divided into several template blocks, and the image to be matched must be divided into several corresponding blocks to be matched. Furthermore, in order to achieve a corresponding match, each template block can only correspond to a unique block to be matched.
[0069] Furthermore, since there are different ways to segment the image to be matched, this application embodiment can describe one possible segmentation method.
[0070] In one scenario, the step of dividing the image to be matched into blocks based on the n template blocks specifically includes:
[0071] Determine the reference position and reference size of each template block among the n template blocks;
[0072] Based on a preset search range, the reference size corresponding to each template block is expanded outward to obtain the target matching size corresponding to each template block;
[0073] The image to be matched is divided into blocks based on the reference position and the target matching size.
[0074] In practical applications, the camera position in the machine is generally fixed. During the process, we want the die to be in the "required position," and the camera captures an image of the die at that position, resulting in the template image. Subsequent dies entering the machine may not be in the "required position," but because the camera position remains unchanged, the image still captures the area from the template image. Understandably, due to the change in the die's position, the features in the later captured image differ significantly from those in the template image, thus affecting matching. Therefore, this application addresses this by changing the lens focal length to enrich the features of the captured image. Figure 4 This is a schematic diagram illustrating image acquisition by a camera, provided as an embodiment of this application. (In conjunction with...) Figure 4 As shown, Figure 4 In this diagram, 'a' represents the image to be matched, and 'b' represents the template image. The area within the blue frame in 'a' is the scaled-up template image, meaning the texture features in the blue area are identical to those in the template image. It's understood that if the die corresponding to the image to be matched aligns with the die corresponding to the template image, the center of the blue frame should align with the center of the image to be matched. If they are not aligned, it indicates a certain offset between the two images. Scaled-up template images onto the image to be matched means that the image to be matched is expanded outwards by a certain size based on the original template image; this size is the preset search range for matching. When dividing the image to be matched into blocks, the blue frame area is first divided proportionally according to the reference position and size of each template block on the template image, resulting in n initial blocks. Each initial block corresponds to a unique template block, and the texture features of the two corresponding blocks should be identical, differing only in scaling ratio. Furthermore, assuming the preset search range is set to 100 pixels, this represents an expansion of the texture features by 50 pixels outwards from the template image. Assuming the base size of each template block is 100*50 pixels, the size of the expanded block in the expanded template image is 200*150 pixels, meaning the target matching size is 200*150 pixels. The image to be matched is then divided into blocks according to the target matching size. It can be understood that when dividing the image to be matched into blocks, each block corresponds to an expanded block; that is, the texture features in each block should be consistent with its corresponding expanded block.
[0075] S203: Determine the valid blocks among the n template blocks based on texture.
[0076] In practical applications, a valid block can be considered as a template block with more obvious features. This application introduces the concept of texture degree to determine the valid blocks among the n obtained template blocks.
[0077] Furthermore, since the methods for determining valid blocks are not entirely the same, this application embodiment can describe one possible determination method.
[0078] In one scenario, S203: determining the valid blocks among the n template blocks based on texture density, specifically including:
[0079] Each of the n template blocks is divided into blocks, and each template block corresponds to m*m calculation blocks; where m≥2;
[0080] Determine the texture degree of each computational block corresponding to each template block;
[0081] Determine the number of computation blocks whose texture degree is greater than a first preset threshold and which correspond to the same template block;
[0082] The template blocks corresponding to the number of calculation blocks that are greater than the second preset threshold are taken as valid blocks;
[0083] The second preset threshold is greater than or equal to m.
[0084] In practical applications, the block-based approach can be further adopted, dividing each template block into several smaller blocks (computation blocks), and then determining the texture degree of each computation block. Among the multiple computation blocks corresponding to a template block, if the number of computation blocks with a texture degree greater than a first preset threshold is greater than a second preset threshold, then the template block is considered a valid block. The second preset threshold is greater than or equal to m, and is generally taken as m. Taking n=9 as an example, the template image is divided into 9 equally sized template blocks. Assuming m=3 (each template block corresponds to 9 computation blocks), the first preset threshold is 0.3, and the second preset threshold is 3. If only three or fewer computation blocks in a template block have a texture degree greater than 0.3, while the texture degree of the remaining six or more computation blocks is less than 0.3, then the template image is recorded as an invalid block; similarly, if four or more computation blocks in a template block have a texture degree greater than 0.3, while the texture degree of the remaining six or fewer computation blocks is less than 0.3, then the template image is recorded as a valid block. It is understood that the first preset threshold mentioned in the embodiments of this application is an empirical value. The first preset threshold was obtained by pre-selecting image regions with various textures and conducting experiments. Specifically, for the pre-selected image regions with various textures, their texture values were calculated using the texture degree calculation formula. Through numerous experiments, it was found that image regions with a texture degree greater than 0.3 had a lower probability of failure when performing template matching, while image regions with a texture degree lower than 0.3 or smaller had less noticeable texture, tended to be flat, and significantly increased the chances of template matching failure.
[0085] Furthermore, since the methods for calculating texture are not entirely the same, this application embodiment can describe one possible calculation method.
[0086] In one case, determining the texture degree of each computational block corresponding to each template block includes:
[0087] Determine the maximum gray value, minimum gray value, and average gray value of each calculation block corresponding to each template block;
[0088] Based on the maximum gray value, the minimum gray value, and the average gray value, the texture degree of each computation block is determined using a texture metric formula;
[0089] The texture measurement formula is: Texture = (Maximum gray value - Minimum gray value) / Average gray value.
[0090] In practical applications, texture degree can be calculated using a texture metric formula. Specifically, first, a template block is taken, and the maximum gray value, minimum gray value, and average gray value of each computational block are calculated. Then, according to the texture metric formula: Texture degree = (Maximum gray value - Minimum gray value) / Average gray value, the texture degree of each computational block is determined. Finally, based on the obtained texture degree, the valid block judgment method given above is used to determine whether each template block is a valid block.
[0091] S204: Match the valid block with the corresponding block to be matched, and obtain the matching result value.
[0092] In practical applications, the template image is divided into n template blocks, and the image to be matched is divided into n matching blocks. Each template block has a unique corresponding matching block, and each matching block should include all texture features of its corresponding template block as well as the texture features extending beyond that template block. A valid block is a block with more prominent texture features selected from all template blocks. Matching this valid block with its corresponding matching block makes the matching result more accurate.
[0093] Furthermore, since the methods for obtaining matching result values are not entirely the same, this application embodiment can describe one possible method of obtaining the result.
[0094] In one scenario, S204: Match the valid block with the corresponding block to be matched, and obtain a matching result value, specifically including:
[0095] The valid block is matched with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score;
[0096] The matching score that is greater than the third preset threshold is taken as the target matching value, and the offset corresponding to the target matching value is recorded.
[0097] In practical applications, the NCC (Normalized Cross-Correlation Coefficient) matching algorithm can be used to match template blocks with their corresponding valid blocks, although other matching algorithms can also be used. The resulting matching value includes a matching score and an offset. A third preset threshold can be set to filter valid blocks with higher matching scores and mark them as successfully matched. Specifically, continuing with the example of n=9, there are nine template blocks. Assuming blocks 1, 3, 6, 7, and 9 are valid blocks, after matching, the matching scores are: block 1: 0.8, block 3: 0.3, block 6: 0.7, block 7: 0.7, and block 9: 0.9. If the third preset threshold is set to 0.6, then blocks 1, 6, 7, and 9 are marked as successfully matched blocks, and their corresponding values of 0.8, 0.7, 0.7, and 0.9 are used as target matching values, with their respective offsets recorded. For example, when the first valid block is matched with its corresponding block to be matched, the offset between the region where the target matching value is 0.8 and the first valid block is x: -2, y: -2 (that is, the block to be matched is moved two pixels to the left and two pixels down relative to the first valid block).
[0098] S205: Align the image to be matched with the template image according to the matching result value.
[0099] In practical applications, the matching result value indicates the matching score and offset of the successfully matched block within the valid block. Combining the offset allows for the alignment of the image to be matched with the template image.
[0100] Furthermore, since the methods of aligning the image to be matched and the template image are not entirely the same, this application embodiment can describe one possible alignment method.
[0101] In one case, S205: Aligning the image to be matched with the template image according to the matching result value, specifically including:
[0102] Use the valid block corresponding to the target matching value as the alignment block;
[0103] If each alignment block corresponds to only one target matching value, then the alignment blocks are clustered according to the offset, the category with the largest number is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block;
[0104] The image to be matched is aligned with the template image based on the offset corresponding to the target block.
[0105] In practical applications, when a valid block is matched with a block to be matched, it is matched with multiple regions within the block to be matched, and corresponding matching scores are obtained. When the matching score of a certain region with the valid block exceeds a third preset threshold, the valid block is marked as a successfully matched block, the matching score is recorded as the target matching value, and the offset corresponding to the target matching value is also recorded. A successfully matched valid block can be recorded as an aligned block, and each aligned block has a corresponding target matching value and offset. Generally, an aligned block corresponds to only one target matching value and offset. At this time, clustering based on the matching results is used to align the image to be matched with the template image. Specifically, continuing with the example of n=9, assuming that blocks 1, 3, 6, 7, and 9 are valid blocks, and blocks 1, 6, 7, and 9 are successfully matched and recorded as aligned blocks, their corresponding target matching values are 0.8, 0.7, 0.7, and 0.9, respectively, and their corresponding offsets are x: -2, y: -2, x: -3, y: -1, x: 1, y: 1, and x: -1, y: -3, respectively. Understandably, the matching blocks corresponding to template blocks 1, 6, and 9 are shifted to the lower left, while the matching block corresponding to template block 7 is shifted to the upper right. Then, through k-means clustering, the category with the largest number of occurrences is selected as the valid result category. That is, the offsets x: -2, y: -2, x: -3, y: -1, and x: -1, y: -3 indicating the lower left shift of the matching block are retained, and other results are marked as outliers. At this point, template blocks 1, 6, and 9 are recorded as target blocks. Finally, the centers of the valid clusters are calculated using the offsets corresponding to the target blocks, i.e., the center values of offsets x: -2, y: -2, x: -3, y: -1, and x: -1, y: -3 are calculated, and the image to be matched is aligned with the template image based on these center values. The central value can be obtained directly by averaging. Therefore, by averaging the above offsets x:-2, y:-2, x:-3, y:-1 and x:-1, y:-3, the value obtained is x:-2, y:-2.
[0106] Furthermore, since there are different ways to align images using offsets, this application embodiment can describe one possible alignment method.
[0107] In one case, aligning the image to be matched with the template image based on the offset corresponding to the target block includes:
[0108] Determine the weighted average of the offsets corresponding to each target block;
[0109] The image to be matched is aligned with the template image based on the weighted average value.
[0110] In practical applications, the center value can also be obtained by weighted averaging. Continuing with the example above, a weight value can be pre-set for each template block. Then, after determining the 1st, 6th, and 9th template blocks as target blocks, the offsets corresponding to the 1st, 6th, and 9th template blocks are weighted and averaged. Finally, the weighted average value is used as the center value to align the image to be matched with the template image.
[0111] Furthermore, since the methods of aligning the image to be matched and the template image are not entirely the same, this application embodiment can describe another possible alignment method.
[0112] In one case, S205: Aligning the image to be matched with the template image according to the matching result value, which may further include:
[0113] Use the valid block corresponding to the target matching value as the alignment block;
[0114] If there are alignment blocks corresponding to at least two target matching values, then the target matching value closest to the center is determined, and the offset corresponding to the target matching value closest to the center is used as the target offset value;
[0115] The alignment blocks are clustered according to the target offset value. The category with the largest number of clusters is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block.
[0116] The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.
[0117] In practical applications, the preset search range can be set, with its value varying from large to small, and the texture on the template image can be anything. Understandably, when the preset search range is large compared to the baseline size of the template block, and the template image contains periodic textures, a valid block may have matching scores exceeding the third preset threshold with multiple regions when matching its corresponding target block. That is, each alignment block may correspond to more than one target matching value and offset. In this case, it is necessary to select the target matching value closest to the center from these multiple target matching values. Continuing with the example of n=9, blocks 1, 3, 6, 7, and 9 are valid blocks, with blocks 1, 6, 7, and 9 successfully matched and denoted as alignment blocks. Block 1 corresponds to two target matching values, while the others correspond to only one. Suppose that when the alignment block is matched with its corresponding block to be matched, it needs to match with 9 regions on the block to be matched. The matching scores with the 2nd and 5th regions both exceed the third preset threshold, being 0.8 and 0.8 respectively, with corresponding offsets of x: -1, y: -2 and x: -2, y: -2 respectively. Assuming the center point coordinates of the block to be matched are (2, 2), and the center point coordinates of the 2nd and 5th regions are (2, 1) and (2, 2) respectively, then the 5th region is closest to the center point of the block to be matched, and its corresponding matching score of 0.8 is taken. Its corresponding offset x: -2, y: -2 is recorded as the target offset value. Then, the target offset value is used to align the image to be matched with the template image. Specifically, the target matching values for the 6th, 7th, and 9th template blocks are 0.7, 0.7, and 0.9 respectively, with corresponding offsets of x: -3, y: -1, x: 1, y: 1, and x: -1, y: -3 respectively. After the above filtering, the target matching value corresponding to the first template block is 0.8, and the corresponding offsets are x: -2, y: -2, denoted as the target offsets. Then, k-means clustering is used to determine the first, sixth, and ninth template blocks as target blocks. Finally, the centers of the effective clusters are calculated using the offsets corresponding to the target blocks (the target offsets corresponding to the first template block and the offsets corresponding to the sixth, seventh, and ninth template blocks), i.e., the center values of offsets x: -2, y: -2, x: -3, y: -1, and x: -1, y: -3 are found, and the image to be matched is aligned with the template image based on these center values. These center values can be obtained directly by averaging or by weighted averaging.
[0118] In summary, this proposed solution for image block alignment in semiconductor inspection differs from block alignment in conventional image processing techniques. This solution primarily targets block matching for images with localized periodic textures and weak textures. The main method involves dividing the template image and the image to be matched into blocks, resulting in n template blocks and n matching blocks. Here, n ≥ 2, and each template block has a unique corresponding matching block. Then, based on texture density, valid blocks are determined from the n template blocks. These valid blocks are matched with their corresponding matching blocks, and the matching result value is obtained. Regions with no texture or very little texture are not matched. Next, the matching result values of the successfully matched blocks are clustered to remove outliers and eliminate interference from local mismatches. Finally, the cluster centers are taken as the final matching result, and the image to be matched is aligned with the template image. Thus, by dividing the image into blocks and selecting more clearly textured valid blocks for matching based on texture density, the more accurate matching result value is used for image registration and alignment, improving the accuracy of image registration and alignment.
[0119] Based on the image registration and alignment method provided in the above embodiments, this application also provides an image registration and alignment device. The image registration and alignment device will be described below with reference to the embodiments and accompanying drawings.
[0120] Figure 5 This is a schematic diagram of the structure of an image registration and alignment device provided in an embodiment of this application. (In conjunction with...) Figure 5 As shown, the image registration and alignment device 500 provided in this application embodiment includes:
[0121] The first segmentation module 501 is used to segment the template image into n template blocks; where n ≥ 2.
[0122] The second segmentation module 502 is used to segment the image to be matched according to the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched.
[0123] The effective determination module 503 is used to determine the effective blocks among the n template blocks based on the texture degree.
[0124] The matching calculation module 504 is used to match the valid block with the corresponding block to be matched and obtain the matching result value;
[0125] Alignment module 505 is used to align the image to be matched with the template image according to the matching result value.
[0126] As one implementation method, regarding how to segment the image to be matched into blocks, the second segmentation module 502 described above can specifically be used for:
[0127] Determine the reference position and reference size of each template block among the n template blocks;
[0128] Based on a preset search range, the reference size corresponding to each template block is expanded outward to obtain the target matching size corresponding to each template block;
[0129] The image to be matched is divided into blocks based on the reference position and the target matching size.
[0130] As one implementation method, the above-mentioned valid determination module 503 may specifically include: a third block division module, a first determination sub-module, a second determination sub-module, and a filtering module for determining the valid blocks in the template block;
[0131] The third block-segmentation module is used to divide each of the n template blocks into blocks, and each template block corresponds to m*m calculation blocks; where m≥2;
[0132] The first determining submodule is used to determine the texture degree of each computation block corresponding to each template block;
[0133] The second determining submodule is used to determine the number of computation blocks whose texture degree is greater than the first preset threshold and which correspond to the same template block;
[0134] The filtering module is used to select template blocks corresponding to calculation blocks whose quantity is greater than a second preset threshold as valid blocks; the second preset threshold is greater than or equal to m.
[0135] As one implementation method, regarding how to determine the texture degree of each computational block corresponding to each template block, the aforementioned first determining submodule can specifically be used for:
[0136] Determine the maximum gray value, minimum gray value, and average gray value of each calculation block corresponding to each template block;
[0137] Based on the maximum gray value, the minimum gray value, and the average gray value, the texture degree of each computation block is determined using a texture metric formula;
[0138] The texture measurement formula is: Texture = (Maximum gray value - Minimum gray value) / Average gray value.
[0139] As one implementation method, regarding how to match the valid block with the corresponding block to be matched, the matching calculation module 504 described above can be specifically used for:
[0140] The valid block is matched with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score;
[0141] The matching score that is greater than the third preset threshold is taken as the target matching value, and the offset corresponding to the target matching value is recorded.
[0142] As one implementation method, regarding how to align the image to be matched with the template image based on the matching result value, the alignment module 505 can specifically be used for:
[0143] Use the valid block corresponding to the target matching value as the alignment block;
[0144] If each alignment block corresponds to only one target matching value, then the alignment blocks are clustered according to the offset, the category with the largest number is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block;
[0145] The image to be matched is aligned with the template image based on the offset corresponding to the target block.
[0146] The step of aligning the image to be matched with the template image based on the offset corresponding to the target block includes:
[0147] Determine the weighted average of the offsets corresponding to each target block;
[0148] The image to be matched is aligned with the template image based on the weighted average value.
[0149] As one implementation method, regarding how to align the image to be matched with the template image based on the matching result value, the alignment module 505 described above can also be used for:
[0150] Use the valid block corresponding to the target matching value as the alignment block;
[0151] If there are alignment blocks corresponding to at least two target matching values, then the target matching value closest to the center is determined, and the offset corresponding to the target matching value closest to the center is used as the target offset value;
[0152] The alignment blocks are clustered according to the target offset value. The category with the largest number of clusters is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block.
[0153] The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.
[0154] In summary, this application first divides the template image into blocks, obtaining n template blocks, where n ≥ 2. Then, based on these n template blocks, the image to be matched is divided into blocks, obtaining n blocks to be matched. Each template block has a unique corresponding block to be matched. Next, valid blocks are determined from the n template blocks based on texture quality; these valid blocks are matched with their corresponding blocks to be matched, and a matching result value is obtained. Finally, the image to be matched is aligned with the template image based on the matching result value. Thus, by dividing the image into blocks and then selecting valid blocks with clearer textures to match the blocks to be matched based on texture quality, a more accurate matching result value is obtained for image registration and alignment, improving the accuracy of image registration and alignment.
[0155] In addition, this application also provides an image registration and alignment device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the image registration and alignment method described above.
[0156] In addition, this application also provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image registration and alignment method described above.
[0157] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for image registration and alignment, characterized in that, The method includes: The template image is divided into blocks to obtain n template blocks; where n ≥ 2. The image to be matched is divided into blocks based on the n template blocks to obtain n blocks to be matched; each template block has a unique corresponding block to be matched. The effective blocks among the n template blocks are determined based on texture degree; The valid block is matched with the corresponding block to be matched, and the matching result value is obtained; Align the image to be matched with the template image based on the matching result value; The determination of the valid blocks among the n template blocks based on texture degree includes: Each of the n template blocks is divided into blocks, and each template block corresponds to m*m calculation blocks; where m≥2; Determine the texture degree of each computational block corresponding to each template block; Determine the number of computation blocks whose texture degree is greater than a first preset threshold and which correspond to the same template block; The template blocks corresponding to the number of calculation blocks that are greater than the second preset threshold are taken as valid blocks; The second preset threshold is greater than or equal to m; Determining the texture degree of each computational block corresponding to each template block includes: Determine the maximum gray value, minimum gray value, and average gray value of each calculation block corresponding to each template block; Based on the maximum gray value, the minimum gray value, and the average gray value, the texture degree of each computation block is determined using a texture metric formula; The texture measurement formula is: Texture = (Maximum gray value - Minimum gray value) / Average gray value.
2. The method according to claim 1, characterized in that, The step of dividing the image to be matched into blocks based on the n template blocks includes: Determine the reference position and reference size of each template block among the n template blocks; Based on a preset search range, the reference size corresponding to each template block is expanded outward to obtain the target matching size corresponding to each template block; The image to be matched is divided into blocks based on the reference position and the target matching size.
3. The method according to claim 1, characterized in that, The step of matching the valid block with the corresponding block to be matched and obtaining the matching result value includes: The valid block is matched with the corresponding block to be matched based on the NCC matching algorithm to obtain a matching score; The matching score that is greater than the third preset threshold is taken as the target matching value, and the offset corresponding to the target matching value is recorded.
4. The method according to claim 3, characterized in that, The step of aligning the image to be matched with the template image based on the matching result value includes: Use the valid block corresponding to the target matching value as the alignment block; If each alignment block corresponds to only one target matching value, then the alignment blocks are clustered according to the offset, the category with the largest number is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block; The image to be matched is aligned with the template image based on the offset corresponding to the target block.
5. The method according to claim 4, characterized in that, Aligning the image to be matched with the template image based on the offset corresponding to the target block includes: Determine the weighted average of the offsets corresponding to each target block; The image to be matched is aligned with the template image based on the weighted average value.
6. The method according to claim 3, characterized in that, The step of aligning the image to be matched with the template image based on the matching result value includes: Use the valid block corresponding to the target matching value as the alignment block; If there are alignment blocks corresponding to at least two target matching values, then the target matching value closest to the center is determined, and the offset corresponding to the target matching value closest to the center is used as the target offset value; The alignment blocks are clustered according to the target offset value. The category with the largest number of clusters is taken as the valid result category, and the alignment block corresponding to the valid result category is taken as the target block. The image to be matched is aligned with the template image based on the target offset value corresponding to the target block.
7. An image registration and alignment device, characterized in that, include: The first block-segmentation module is used to divide the template image into blocks to obtain n template blocks; Where n≥2; The second segmentation module is used to segment the image to be matched into n blocks based on the n template blocks, so as to obtain n blocks to be matched; each template block has a unique corresponding block to be matched. A valid determination module is used to determine the valid blocks among the n template blocks based on texture degree; The matching calculation module is used to match the valid block with the corresponding block to be matched and obtain the matching result value; An alignment module is used to align the image to be matched with the template image based on the matching result value; The effective determination module specifically includes: a third block module, a first determination sub-module, a second determination sub-module, and a filtering module; The third block-segmentation module is used to divide each of the n template blocks into blocks, with each template block corresponding to m*m calculation blocks; where m≥2; The first determining submodule is used to determine the texture degree of each computation block corresponding to each template block; The second determining submodule is used to determine the number of computation blocks whose texture degree is greater than the first preset threshold and which correspond to the same template block; The filtering module is used to select the template blocks corresponding to the calculation blocks whose quantity is greater than the second preset threshold as valid blocks; The second preset threshold is greater than or equal to m; The first determining submodule is specifically used for: Determine the maximum gray value, minimum gray value, and average gray value of each calculation block corresponding to each template block; Based on the maximum gray value, the minimum gray value, and the average gray value, the texture degree of each computation block is determined using a texture metric formula; The texture measurement formula is: Texture = (Maximum gray value - Minimum gray value) / Average gray value.
8. An image registration and alignment device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the registration and alignment method for an image as described in any one of claims 1 to 6 when executing the computer program.
9. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image registration and alignment method as described in any one of claims 1 to 6.