Image alignment method and device, and storage medium

Abstract

This application discloses an image alignment method, apparatus, and storage medium. The method includes: acquiring a template image and a real-time image to be searched; processing the template image using a weight generation algorithm to obtain a weight map corresponding to the template image; processing the real-time image to be searched using the weight generation algorithm to obtain a weight map corresponding to the real-time image to be searched; storing a response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula to obtain a stored response map; and performing image alignment between the real-time image to be searched and the template image based on the stored response map. This application not only achieves high-precision image alignment but also provides information about the matching quality.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image alignment method, apparatus and storage medium. Background Technology

[0002] In modern semiconductor manufacturing processes, wafer inspection is a core step in ensuring chip yield and product reliability. This process automatically scans the wafer surface using high-magnification optical systems (such as electron microscopes or optical microscopes), acquiring massive amounts of high-resolution images. These images are then compared with defect-free reference images to identify manufacturing defects at the micrometer or even nanometer scale, such as particle contamination, circuit breaks, short circuits, and pattern misalignment. The accuracy of image alignment is the cornerstone of the entire wafer inspection process. Only when the real-time acquired images achieve sub-pixel-level precise alignment with the reference template image can subsequent defect identification algorithms function effectively. Any minute alignment deviation may be incorrectly identified as a defect (false defect) or cause genuine minute defects to be missed, severely impacting the accuracy and reliability of the inspection. Summary of the Invention

[0003] To address the aforementioned issues, this application provides an image alignment method, apparatus, and storage medium, with the aim of improving the accuracy and robustness of image alignment.

[0004] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides an image alignment method, including: Get the template image and the real-time image to be searched; The template image is processed using a weight generation algorithm to obtain a weight map corresponding to the template image; The weight generation algorithm is used to process the real-time image to be searched to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is used to configure a corresponding weight value for each pixel in the template image or the real-time image to be searched. The response map is stored based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula, and the stored response map is obtained. Image alignment is performed on the real-time image to be searched and the template image based on the stored response graph.

[0005] Optionally, the step of processing the template image using a weight generation algorithm to obtain a weight map corresponding to the template image specifically includes: The template image is recursively calculated to obtain the horizontal gradient map and the vertical gradient map corresponding to the template image. Based on the horizontal gradient map and the vertical gradient map, the first feature value and the second feature value corresponding to each pixel in the template image are obtained; Based on the first feature value and the second feature value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

[0006] Optionally, obtaining the first feature value and the second feature value corresponding to each pixel in the template image based on the horizontal gradient map and the vertical gradient map specifically includes: Based on the horizontal gradient map and the vertical gradient map, a horizontal element map, a vertical element map, and a combined element map are obtained; the horizontal element map is obtained by squaring the horizontal gradient map element by element; the vertical element map is obtained by squaring the vertical gradient map element by element; and the combined element map is obtained by multiplying the horizontal gradient map and the vertical gradient map element by element. The horizontal element map, the vertical element map, and the combined element map are smoothed respectively to obtain smoothed horizontal element map, smoothed vertical element map, and smoothed combined element map; By traversing each pixel in the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map, the structure tensor of each pixel in the template image is dynamically constructed. The eigenvalue decomposition is performed on the structural tensor of each pixel in the template image to obtain the first eigenvalue and the second eigenvalue corresponding to each pixel in the template image.

[0007] Optionally, the smoothing process performed on the horizontal element map, the vertical element map, and the combined element map to obtain smoothed horizontal element map, smoothed vertical element map, and smoothed combined element map specifically includes: The horizontal element map, the vertical element map, and the combined element map are smoothed using a Gaussian smoothing filter to obtain the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map.

[0008] Optionally, obtaining the weight map corresponding to the template image based on the first feature value and the second feature value corresponding to each pixel in the template image specifically includes: The weight value corresponding to each pixel in the template image is obtained by using the weight formula based on the first feature value and the second feature value corresponding to each pixel in the template image; Based on the weight value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

[0009] Optionally, the weighting formula is as follows: W(x,y)=(1-exp(-(λ(x,y)1-λ(x,y)2) 2 / σ 2 ))×(λ(x, y)1+λ(x, y)2); Where W(x, y) represents the weight value corresponding to the pixel at coordinate (x, y) in the template image; λ(x, y)1 represents the first feature value corresponding to the pixel at coordinate (x, y) in the template image; λ(x, y)2 represents the second feature value corresponding to the pixel at coordinate (x, y) in the template image; σ represents the scale parameter; and exp() represents the exponential function.

[0010] Optionally, the method further includes: A response graph is constructed based on the real-time image to be searched and the template image; the width value of the response graph is configured as the sum of the width difference and a first preset value; the width difference is configured as the difference between the width value of the real-time image to be searched and the width value of the template image; the length value of the response graph is configured as the sum of the length difference and a second preset value; the length difference is configured as the difference between the length value of the real-time image to be searched and the width length value of the template image; The response graph is initialized.

[0011] Optionally, the step of using the improved normalized cross-correlation formula to store the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched, to obtain the stored response map, specifically includes: The improved normalized cross-correlation formula is used to obtain the matching value of each candidate sub-region in the real-time image to be searched based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched; the improved normalized cross-correlation formula is configured to introduce the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched for modulation when calculating the weight value. The matching value of each candidate sub-region in the real-time image to be searched is stored in the response map to obtain the stored response map; the stored response map is used to store the matching values ​​of multiple candidate sub-regions in the real-time image to be searched.

[0012] Optionally, the step of using the improved normalized cross-correlation formula to obtain the matching value of each candidate sub-region in the real-time image to be searched based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched specifically includes: Based on the weight map corresponding to the template image and the gray value of each pixel in the template image, the weighted average gray value of the template image is obtained; Based on the weight map corresponding to the real-time image to be searched and the gray value of each pixel in the real-time image to be searched, the weighted average gray value of each candidate sub-region in the real-time image to be searched is obtained. Multiple candidate sub-regions in the real-time image to be searched are extracted by traversing through a sliding window. For any candidate sub-region: Based on the weight value of each pixel in the candidate sub-region and the weight value of each pixel in the template image, a combined weight value for each pixel in the candidate sub-region is obtained; The matching value of the candidate sub-region is obtained based on the combined weight value of each pixel in the candidate sub-region, the gray value of each pixel in the template image, the weighted average gray value of the template image, the weighted average gray value of the candidate sub-region, and the gray value of each pixel in the candidate sub-region.

[0013] Optionally, the step of aligning the real-time image to be searched and the template image based on the stored response graph specifically includes: A global search is performed on the stored response graph, and the maximum value point in the stored response graph is taken as the main peak. Based on the coordinates of the pixel corresponding to the main peak, the optimal matching position of the real-time image to be searched is obtained; Image alignment is performed between the real-time image to be searched and the template image based on the optimal matching position.

[0014] Optionally, obtaining the optimal matching position of the real-time image to be searched based on the coordinates of the pixel corresponding to the main peak specifically includes: The target region is obtained based on the coordinates of the pixels corresponding to the main peak; A fitting algorithm is used to obtain the coordinates of the sub-pixels corresponding to the main peak based on the target region; the fitting algorithm includes quadratic surface fitting or centroid calculation. The coordinates of the sub-pixel corresponding to the main peak are used as the best matching position of the real-time image to be searched.

[0015] A second aspect of this application provides an image alignment device, comprising: The acquisition module is used to acquire template images and real-time images to be searched; The weight map determination module is used to process the template image using a weight generation algorithm to obtain the weight map corresponding to the template image; The weight map determination module is further configured to process the real-time image to be searched using the weight generation algorithm to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is configured to assign a corresponding weight value to each pixel in the template image or the real-time image to be searched. The normalization module is used to store the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula, so as to obtain the stored response map. The image alignment module is used to perform image alignment between the real-time image to be searched and the template image based on the stored response map.

[0016] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image alignment method described in the first aspect of this application.

[0017] This application includes acquiring a template image and a real-time image to be searched; processing the template image using a weight generation algorithm to obtain a weight map corresponding to the template image; processing the real-time image to be searched using the weight generation algorithm to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is used to configure a corresponding weight value for each pixel in the template image or the real-time image to be searched; storing the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula to obtain a stored response map; and performing image alignment between the real-time image to be searched and the template image based on the stored response map.

[0018] This application assigns a weight value to each pixel in the template image using a weight generation algorithm. This can emphasize or weaken the importance of specific regions, allowing the image alignment process to focus more on key features rather than noise or other irrelevant information, thereby improving the accuracy and robustness of image alignment. Similarly, weight calculation on the real-time image to be searched can focus more on parts with similar features to the template image during image alignment, improving alignment accuracy and reducing the impact of background interference. While traditional normalized cross-correlation methods can effectively evaluate the matching degree between two images, they perform poorly in complex scenes such as changes in lighting or partial occlusion.

[0019] The improved normalized cross-correlation formula incorporates weight maps corresponding to the two images for modulation during the calculation process. This allows it to consider the differences in importance of different pixels while calculating the matching values, enhancing the algorithm's adaptability under various conditions and further improving the reliability and accuracy of the matching results. Finally, by analyzing the matching values ​​recorded in the response map, the position of the closest template image in the real-time image to be searched can be determined. This not only achieves high-precision image alignment but also provides information about the matching quality. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an image alignment method provided in this application embodiment; Figure 2 This is a structural diagram of an image alignment device provided in an embodiment of this application. Detailed Implementation

[0022] As mentioned earlier, with the dramatic increase in the complexity of integrated circuit design, modern wafer surfaces exhibit highly precise and extremely repetitive pattern structures. Especially in the core regions of devices such as memory chips and image sensors, there are often millions of unit structures (such as memory cells and pixel units) arranged in a strictly two-dimensional array, with highly similar or even identical geometric and grayscale features. This strongly periodic and highly self-similar pattern characteristic poses a severe challenge to traditional image alignment algorithms.

[0023] Both of the current mainstream alignment methods have significant limitations in this type of scenario: On the one hand, based on normalized cross-correlation, during the matching process, the presence of numerous nearly identical repeating instances of the template in the image can generate multiple densely distributed local peaks (i.e., "side lobes") with similar amplitudes on the response map. These pseudo-peaks are difficult to distinguish from the main peak at the true matching location, causing the algorithm to be unable to stably lock the unique correct pose. This can easily lead to "jittering" or "jumping" in the matching results between adjacent units, and in severe cases, directly locating the wrong repeating unit, causing sub-pixel-level alignment failure, and thus triggering false defects or missed detections.

[0024] On the other hand, matching methods based on local feature points also fail to perform well. In highly regular wafer patterns, while there are abundant extractable key points such as corners and edges, their local neighborhood textures are highly consistent, resulting in a lack of discriminative power in the generated feature descriptors. This leads to a large number of ambiguous correspondences during the feature matching stage, with correct matches being submerged in a massive number of false matches. Even with robust estimation strategies such as the Random Sample Consensus (RANSAC) algorithm, the low proportion of interior points makes it difficult to converge to the true geometric transformation model, ultimately resulting in insufficient registration accuracy or complete failure.

[0025] In view of the above problems, this application provides an image alignment generation method, apparatus, and storage medium. The method includes: acquiring a template image and a real-time image to be searched; processing the template image using a weight generation algorithm to obtain a weight map corresponding to the template image; processing the real-time image to be searched using the weight generation algorithm to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is used to configure a corresponding weight value for each pixel in the template image or the real-time image to be searched; storing the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula to obtain a stored response map; and performing image alignment between the real-time image to be searched and the template image based on the stored response map.

[0026] This application introduces a weight generation algorithm to assign corresponding weight values ​​to each pixel in the template image and the real-time image to be searched. This effectively highlights pixels with significant features or importance while suppressing highly repetitive and non-unique regions that are prone to confusion. The improved normalized cross-correlation formula incorporates the weight maps corresponding to the two images for modulation during the calculation process, making the peak of the correct matching position in the response image more obvious, while effectively suppressing the sidelobe peaks. This not only helps to stably lock the unique and correct matching peak but also significantly reduces the probability of false matches and improves the reliability of the matching results.

[0027] By employing a weighted mechanism to adjust the similarity measurement between images, high alignment accuracy and robustness are maintained even when dealing with complex patterns exhibiting strong periodicity and high self-similarity. This is particularly beneficial when processing wafer images with numerous repeating cells, such as those found in memory chips. It effectively prevents matching results from "jittering" between adjacent cells or jumping to incorrect cells, ensuring sub-pixel-level accurate alignment. This significantly improves the efficiency of image alignment, especially crucial when processing high-resolution images in large-scale integrated circuit design.

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0029] Figure 1 A flowchart of an image alignment method provided in an embodiment of this application is shown below. Figure 1 As shown, an image alignment method includes: S101: Obtain the template image and the real-time image to be searched.

[0030] This application does not limit the method of acquiring the template image and the real-time image to be searched. For example, in a wafer inspection scenario, multiple high-resolution images can be acquired by imaging the same physical location of multiple dies on the same wafer. Based on this, one of the images can be designated as the template image, usually the first frame acquired, and it can be slightly cropped (e.g., removing a few pixels from the edges) to ensure that its size is slightly smaller than the other images, thereby reserving boundary space for subsequent sliding matching in the image to be searched; any other image can be designated as the real-time image to be searched for alignment comparison with the template image.

[0031] This application balances the flexibility of image acquisition in actual processes with the robustness of alignment algorithms: on the one hand, it allows dynamic selection of templates and real-time images from multiple frames of images in the same batch and the same region, adapting to different modes such as online detection or offline re-inspection; on the other hand, by controlling the template image size to be slightly smaller, it effectively avoids matching out-of-bounds problems caused by image edge distortion, focus deviation or mechanical positioning error, and improves the stability and reliability of sliding window algorithms such as normalized cross-correlation.

[0032] S102: The template image is processed using a weight generation algorithm to obtain a weight map corresponding to the template image.

[0033] S103: The weight generation algorithm is used to process the real-time image to be searched to obtain the weight map corresponding to the real-time image to be searched.

[0034] This application does not limit the weight generation algorithm, but may use the weight generation algorithm to configure a corresponding weight value for each pixel in the template image or the real-time image to be searched.

[0035] S104: Using the improved normalized cross-correlation formula, the response map is stored based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched, to obtain the stored response map.

[0036] This application does not limit the specific form of the improved normalized cross-correlation formula. Given that this application generates weight maps for both the template image and the real-time image to be searched, the proposed improved normalized cross-correlation formula aims to organically combine these two types of weight maps with the traditional normalized cross-correlation formula calculation process to enhance the accuracy and robustness of the matching algorithm.

[0037] Specifically, the improved normalized cross-correlation formula is configured to modulate the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched when calculating the weight values. This ensures that during image matching, not only the grayscale or color information of the pixels themselves is considered, but also the importance weights of each pixel position are integrated. This adjustment effectively suppresses matching ambiguity caused by highly repetitive texture backgrounds, thereby improving the algorithm's performance in complex application scenarios.

[0038] This application does not limit the response map, as described above, the stored response map is used to store the matching values ​​of multiple candidate sub-regions in the real-time image to be searched.

[0039] S105: Based on the stored response map, perform image alignment between the real-time image to be searched and the template image.

[0040] This application does not limit the specific image alignment method. For example, based on the matching values ​​of each candidate position in the response map, the position with the highest similarity (or optimal score) can be determined as the best matching position. Subsequently, based on this position, spatial alignment operations (such as translation, rotation, or affine transformation) are performed on the real-time image to be searched and the template image to achieve precise alignment at the pixel or sub-pixel level. This alignment process can employ techniques such as interpolation, optimization search, or geometric correction to ensure that the final alignment result meets the requirements of high-precision detection or comparison tasks. The alignment strategy guided by the response map effectively improves the accuracy and robustness of registration.

[0041] The above describes the main technical solution of this application. Further implementations of the main technical solution are now introduced. Details are as follows: Regarding S102, which uses a weight generation algorithm to process the template image to obtain a weight map corresponding to the template image, this application provides an optional embodiment: The template image is recursively calculated to obtain the horizontal gradient map and vertical gradient map corresponding to the template image.

[0042] This application does not limit the specific recursive method. For example, in an optional embodiment, the Sobel operator can be used to calculate the gradient of the template image: specifically, a 3×3 or 5×5 Sobel convolution kernel is used to perform first-order partial derivative operations on the template image in the horizontal (x) direction and the vertical (y) direction, respectively, thereby generating two gradient maps with the same size as the original image, namely the horizontal gradient map Ix (each pixel value reflects the gray-level change rate at that position in the x direction) and the vertical gradient map Iy (each pixel value reflects the gray-level change rate at that position in the y direction).

[0043] It should be emphasized that the above-described scheme using the Sobel operator is merely illustrative and not intended to limit the technical solution of this application. Any method that can effectively generate horizontal and vertical gradient images representing local structural changes from a template image (e.g., using the Scharr operator, Prewitt operator, central difference, Gaussian derivative filter, or other differential operators) falls within the scope of protection of this application. The core focus is on obtaining directional gradient information to support subsequent processing, rather than specific gradient calculation methods.

[0044] Based on the horizontal gradient map and the vertical gradient map, the first feature value and the second feature value corresponding to each pixel in the template image are obtained.

[0045] For example, based on the horizontal gradient map and the vertical gradient map, a horizontal element map, a vertical element map, and a combined element map are obtained.

[0046] This application does not limit the method of determining the horizontal element map, vertical element map, and combined element map. For example, the horizontal element map is obtained by squaring the horizontal gradient map element by element (e.g., horizontal element map I_xx = Ix × Ix (i.e., element-wise squared Ix)); the vertical element map is obtained by squaring the vertical gradient map element by element (e.g., vertical element map I_yy = Iy × Iy (i.e., element-wise squared Iy)); the combined element map is obtained by multiplying the horizontal gradient map and the vertical gradient map element by element (e.g., combined element map I_xy = Ix × Iy (i.e., element-wise product of Ix and Iy)). Although this application does not limit the method of determining the horizontal element map, vertical element map, and combined element map, in order to ensure the accurate execution of subsequent steps, the horizontal element map, vertical element map, and combined element map in this embodiment are all the same size as the template image.

[0047] The horizontal element map, the vertical element map, and the combined element map are smoothed to obtain smoothed horizontal element map, smoothed vertical element map, and smoothed combined element map.

[0048] This application does not limit the specific method of smoothing. For example, a feasible implementation is to use a Gaussian smoothing filter to smooth the horizontal element map, the vertical element map, and the combined element map respectively, to obtain the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map, thereby reducing image noise and preserving important features. In this process, the size of the Gaussian kernel (e.g., 5×5 or 7×7) and the standard deviation (e.g., 1.5) are adjustable parameters used to control the neighborhood size and its spatial weighting method, thereby generating the smoothed horizontal element map, vertical element map, and combined element map.

[0049] It is worth noting that the method using a Gaussian smoothing filter described above is merely illustrative and does not limit this application to using only this specific method for smoothing. In fact, any smoothing technique that can effectively reduce image noise while preserving key features is applicable to this application, including but not limited to mean filtering, bilateral filtering, median filtering, or other adaptive filtering methods. The core of this application lies in improving image quality through appropriate smoothing processing, rather than the implementation of a specific smoothing algorithm. Therefore, any smoothing method can be adopted as long as it achieves the same technical effect.

[0050] By traversing each pixel in the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map, a structure tensor for each pixel in the template image is dynamically constructed.

[0051] This application does not limit the method for determining the structure tensor. For example, the structure tensor components corresponding to the pixels in the smoothed horizontal element image S_xx, the smoothed vertical element image S_yy, and the smoothed combined element image S_xy can be combined to dynamically construct the structure tensor of each pixel in the template image. For example, the structure tensor J(x,y) of the pixel at coordinates (x,y) in the template image is as follows: J(x,y)=[S_xx(x,y),S_xy(x,y);S_xy(x,y),S_yy(x,y)] (1); Where S_xx(x, y) represents the structural tensor component corresponding to the pixel at coordinate (x, y) in the smoothed horizontal element image; S_xy(x, y) represents the structural tensor component corresponding to the pixel at coordinate (x, y) in the smoothed combined element image; and S_yy(x, y) represents the structural tensor component corresponding to the pixel at coordinate (x, y) in the smoothed vertical element image.

[0052] The eigenvalue decomposition is performed on the structural tensor of each pixel in the template image to obtain the first eigenvalue and the second eigenvalue corresponding to each pixel in the template image.

[0053] For example, by performing eigenvalue decomposition on the structure tensor J(x,y) of the pixel at coordinates (x,y) in the template image, two real eigenvalues ​​can be obtained: the first eigenvalue λ(x,y)1 (representing the largest eigenvalue of the structure tensor at coordinates (x,y) in the template image, reflecting the intensity of the local gradient energy in the principal direction) and the second eigenvalue λ(x,y)2 (representing the smallest eigenvalue of the pixel at coordinates (x,y) in the template image, characterizing the gradient response intensity in the orthogonal direction), and λ(x,y)1 ≥ λ(x,y)2. )2 ≥0.

[0054] By traversing all pixels, two eigenvalue maps of the same size as the original image can be constructed: one storing the first eigenvalue of each point, and the other storing the second eigenvalue. These eigenvalue maps can be used for subsequent analysis of local image structure (such as distinguishing flat regions, edges, or corners), providing a discriminative basis for weight generation, enhancement of defect-sensitive regions, or robust registration.

[0055] Based on the first feature value and the second feature value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

[0056] This application does not limit the method of determining the weight map based on the first eigenvalue and the second eigenvalue, but as one optional embodiment: The weight value for each pixel in the template image is obtained by using a weight formula based on the first and second feature values ​​corresponding to each pixel.

[0057] Based on the weight value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

[0058] The weight map corresponding to the template image is a final structural feature weight map of the same size as the template image. The value of each pixel in the weight map is the weight value corresponding to each pixel after final calculation.

[0059] This application does not impose any restrictions on the weighting formula, such as the following weighting formula: W(x,y)=(1-exp(-(λ(x,y)1-λ(x,y)2) 2 / σ 2 ))×(λ(x, y)1+λ(x, y)2) (2); Where W(x, y) represents the weight value corresponding to the pixel at coordinates (x, y) in the template image; λ(x, y)1 represents the first feature value corresponding to the pixel at coordinates (x, y) in the template image; λ(x, y)2 represents the second feature value corresponding to the pixel at coordinates (x, y) in the template image; σ represents an adjustable scale parameter used to control the sensitivity to anisotropy; exp() represents an exponential function.

[0060] The term (λ(x,y)1+λ(x,y)2) represents the total gradient energy factor of the neighborhood of the pixel at coordinates (x,y) in the template image. The total gradient energy factor is characterized by the sum of eigenvalues ​​(λ(x,y)1+λ(x,y)2) or its equivalent form (such as the trace(J) of the structure tensor). This term ensures that only regions with sufficiently large gradient strength can obtain high weights.

[0061] (λ(x,y)1-λ(x,y)2) 2The term represents the anisotropy factor of the pixel at coordinates (x, y) in the template image. The anisotropy factor is the squared difference of the eigenvalues ​​(λ(x, y)1-λ(x, y)2) obtained through a nonlinear function (such as an exponential function). 2 Mapped to a predetermined interval, the anisotropy factor represents the degree of anisotropy of the neighborhood gradient of the pixel at coordinates (x, y) in the template image. For strong edges with a single orientation, this value is large; for corners or flat areas with complex orientations, this value is small.

[0062] (1-exp(-(λ(x,y)1-λ(x,y)2) 2 / σ 2 The term is an anisotropy selection factor that maps the degree of anisotropy to the interval [0, 1]. When the anisotropy is strong (simple edges), the factor is close to 1; when the anisotropy is weak (corner points, intersections of repeating structures), the factor is close to 0.

[0063] This embodiment performs feature-based analysis, calculating a final weight value for each pixel that reflects its value as a "reliable matching feature".

[0064] Regarding S103, which uses the weight generation algorithm to process the real-time image to be searched and obtain a weight map corresponding to the real-time image to be searched, this application provides an optional embodiment: The real-time image to be searched is recursively calculated to obtain the horizontal gradient map and vertical gradient map corresponding to the real-time image to be searched.

[0065] Since the methods for determining the horizontal and vertical gradient maps of the real-time image to be searched are similar to those for the template image, and their calculation methods follow the same principles and steps, they will not be elaborated here.

[0066] Based on the horizontal gradient map and the vertical gradient map corresponding to the real-time image to be searched, the horizontal element map, the vertical element map, and the combined element map corresponding to the real-time image to be searched are obtained.

[0067] Since the methods for determining the horizontal element map, vertical element map, and combined element map of the real-time image to be searched are similar to the corresponding process for the template image, and the calculation methods of the two follow the same principles and steps, they will not be repeated here.

[0068] The horizontal element map, the vertical element map, and the combined element map corresponding to the real-time image to be searched are smoothed respectively to obtain the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map corresponding to the real-time image to be searched.

[0069] Since the methods for determining the smoothed horizontal element map, smoothed vertical element map, and smoothed combined element map of the real-time image to be searched are similar to the corresponding process for the template image, and the calculation methods of the two follow the same principles and steps, they will not be repeated here.

[0070] By traversing each pixel in the smoothed horizontal element graph, the smoothed vertical element graph, and the smoothed combined element graph corresponding to the real-time image to be searched, a structure tensor for each pixel in the real-time image to be searched is dynamically constructed.

[0071] Since the method for determining the structure tensor of each pixel in the real-time image to be searched is similar to the corresponding process for the template image, and the calculation methods of the two follow the same principles and steps, they will not be repeated here.

[0072] The structure tensor of each pixel in the real-time image to be searched is decomposed into eigenvalues ​​to obtain the first eigenvalue and the second eigenvalue corresponding to each pixel in the real-time image to be searched.

[0073] Since the determination of the first and second feature values ​​corresponding to each pixel in the real-time image to be searched is similar to the corresponding process in the template image, and the calculation methods of the two follow the same principles and steps, they will not be repeated here.

[0074] Based on the first feature value and the second feature value corresponding to each pixel in the real-time image to be searched, a weight map corresponding to the real-time image to be searched is obtained.

[0075] Since the method for determining the weight map corresponding to the real-time image to be searched is similar to that for the template image, and the calculation methods for both follow the same principles and steps, they will not be elaborated here.

[0076] Regarding the method for constructing the response graph, this application provides an optional embodiment: A response graph is constructed based on the real-time image to be searched and the template image.

[0077] This application does not limit the size of the response image. The width value of the response image is configured as the sum of the width difference and a first preset value (such as 1); the width difference is configured as the difference between the width value of the real-time image to be searched and the width value of the template image; the length value of the response image is configured as the sum of the length difference and a second preset value (such as 1); the length difference is configured as the difference between the length value of the real-time image to be searched and the width of the template image.

[0078] The response graph is initialized.

[0079] A response map related to the size of the real-time image is initialized. Each element of the response map is initialized to 0 or a specific initial value, which will be used as the basis for storing the score obtained during the matching process. This ensures that the template matching algorithm can be effectively applied even between images of different sizes, while taking into account the impact of edge effects.

[0080] Regarding S104, which utilizes an improved normalized cross-correlation formula to store the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched, to obtain the stored response map, this application provides an optional embodiment: Using an improved normalized cross-correlation formula, based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched, the matching value of each candidate sub-region in the real-time image to be searched is obtained.

[0081] For example, the weighted average gray value of the template image is obtained based on the weight map corresponding to the template image and the gray value of each pixel in the template image.

[0082] This application does not limit the method for determining the weighted average gray value of the template image, such as calculating it using the following formula: (3); in, This represents the weighted average gray value of the template image; This represents the weight value of the pixel at coordinates (x, y) in the template image; This represents the grayscale value of the pixel at coordinates (x, y) in the template image.

[0083] Based on the weight map corresponding to the real-time image to be searched and the gray value of each pixel in the real-time image to be searched, the weighted average gray value of each candidate sub-region in the real-time image to be searched is obtained.

[0084] This application does not limit the method for determining the weighted average gray value of each candidate sub-region in the real-time image to be searched. For example, it can be calculated using the following formula: (4); in, This represents the weighted average gray value of the candidate sub-region with the top-left corner coordinates (u, v) in the real-time image to be searched; This represents the weight value of the pixel at coordinates (x, y) in the candidate sub-region with coordinates (u, v) in the top-left corner of the real-time image to be searched. This represents the grayscale value of the pixel at coordinates (x, y) in the candidate sub-region with coordinates (u, v) at the top left corner of the real-time image to be searched.

[0085] Multiple candidate sub-regions in the real-time image to be searched are extracted by traversing through a sliding window.

[0086] In this embodiment, each candidate sub-region is the same size as the template image.

[0087] For any candidate sub-region: Based on the weight value of each pixel in the candidate sub-region and the weight value of each pixel in the template image, a combined weight value for each pixel in the candidate sub-region is obtained.

[0088] This application does not limit the calculation method of the combined weight value. For example, the combined weight calculation formula can be used to obtain the combined weight value of each pixel in the candidate sub-region.

[0089] For example, the formula for calculating the combined weights is as follows: w(x, y) = W T (x, y) × W S (x, y) (5); Where w(x, y) represents the combined weight value of the pixel at coordinates (x, y) in the candidate sub-region; W T (x, y) represents the weight value of the pixel at coordinates (x, y) in the template image; W S (x, y) represents the weight value of the pixel at coordinate (x, y) in the weight map corresponding to the candidate sub-region.

[0090] The matching value of the candidate sub-region is obtained based on the combined weight value of each pixel in the candidate sub-region, the gray value of each pixel in the template image, the weighted average gray value of the template image, the weighted average gray value of the candidate sub-region, and the gray value of each pixel in the candidate sub-region.

[0091] For example, the matching value of the candidate sub-region can be obtained using the matching value calculation formula: (6); in, This represents the matching value of the candidate sub-region with the top-left corner coordinates (u, v) in the real-time image to be searched.

[0092] The matching values ​​of each candidate sub-region in the real-time image to be searched are stored in the response map to obtain the stored response map.

[0093] To more clearly illustrate the difference between the improved normalized cross-correlation formula of this application and the traditional normalized cross-correlation formula, this application also provides the traditional normalized cross-correlation formula, as follows: (7); (8); (9); in, This represents the weighted average gray value of the template image calculated using the traditional normalized cross-correlation formula. This represents the weighted average gray value of the candidate sub-region with coordinates (u, v) at the top left corner of the real-time image to be searched, calculated using the traditional normalized cross-correlation formula. This represents the matching value of the candidate sub-region with coordinates (u, v) at the top left corner of the real-time image to be searched, calculated using the traditional normalized cross-correlation formula.

[0094] Comparing formulas (3) and (7), it is clear that the traditional Normalized Cross-Correlation (NCC) only performs simple summation or mean processing on the gray values ​​of each pixel when calculating the weighted average gray value of the template image, without introducing an independent weight configuration mechanism for each pixel. Similarly, by comparing formulas (4) and (8), it can be clearly seen that when calculating the weighted average gray value of the corresponding region of the real-time image to be searched, it also only performs undifferentiated weighting (usually equal weighting), without combining the importance or reliability of pixels to give differentiated weights.

[0095] Comparing formulas (6) and (9), it is clear that traditional NCC does not embed any pixel-level weight information based on image content throughout the matching process, resulting in it treating all pixels "equally." When faced with highly repetitive and strongly periodic structures (such as wafer images), a large number of geometrically similar candidate regions will produce almost identical matching responses, resulting in multiple local maxima (pseudo-peaks) with similar amplitudes in the response image. In this case, relying solely on the matching values ​​obtained from traditional NCC to locate the main peak is prone to misselecting the wrong position due to a lack of discriminative power, causing alignment jitter or failure.

[0096] In contrast, this application generates content-aware weight maps for the template image and the image to be searched, and explicitly introduces a dual-weight modulation mechanism in the improved normalized cross-correlation formula, which effectively enhances the response intensity of the true matching position and suppresses sidelobe interference caused by repetitive structures, thereby significantly improving the uniqueness and robustness of the main peak.

[0097] Regarding S105, which performs image alignment between the real-time image to be searched and the template image based on the stored response map, this application provides an optional embodiment: A global search is performed on the stored response graph, and the maximum value point in the stored response graph is taken as the main peak.

[0098] For example, the entire response graph is traversed, the numerical values ​​of each point are compared, and the point with the highest response value is identified as the final matching location (i.e., the main peak). This process must ensure comprehensive coverage (the search operation should fully scan every element in the response graph to ensure that no potential peak points are missed) and accurate identification (using strict criteria to define the "maximum value," requiring not only that the point has the highest response value in its neighborhood, but also possibly further verification of whether it meets other conditions (e.g., the degree of difference from surrounding points, local continuity, etc.) to eliminate the influence of noise or spurious peaks).

[0099] Considering that the response map may be very large, optimization algorithms can be applied in practical implementations to accelerate the search process. For example, divide-and-conquer strategies, multi-resolution analysis, or gradient-based methods can be used to quickly locate candidate regions, and then a detailed search can be performed within that region.

[0100] This embodiment can accurately and efficiently extract the most significant matching positions from the response graph, thereby improving the reliability and practicality of the overall matching algorithm.

[0101] Based on the coordinates of the pixels corresponding to the main peak, the optimal matching position of the real-time image to be searched is obtained.

[0102] This application does not limit the method for determining the optimal matching position, but as one optional embodiment: The target region is obtained based on the coordinates of the pixels corresponding to the main peak.

[0103] This application does not limit the method for determining the target region. For example, the local neighborhood of the pixel position corresponding to the main peak in the response image (such as a 3×3 or 5×5 area centered on that pixel) can be defined as the target region. The range of this neighborhood can be flexibly adjusted according to the actual matching accuracy requirements, image resolution, or noise level, aiming to capture a robust and representative local region around the optimal matching position for subsequent fine registration, sub-pixel optimization, or defect analysis. This method is only an example and does not constitute a limitation on the scope of protection of this application.

[0104] A fitting algorithm is used to obtain the coordinates of the sub-pixels corresponding to the main peak based on the target region.

[0105] For example, a quadratic surface fitting method can be used to construct a local response surface model centered on the main peak within the target region, and sub-pixel-level positions can be obtained by solving for the extreme points of this surface; alternatively, a centroid calculation method can be used, treating the response values ​​within the target region as weights and calculating their weighted centroids as sub-pixel coordinates. These fitting strategies can effectively overcome the integer pixel resolution limitation of the original response map, significantly improving the accuracy of matching and positioning, and meeting the requirements of high-precision image alignment. This application does not limit the specific form of the fitting algorithm, as long as it can achieve sub-pixel-level coordinate estimation based on the response distribution of the target region.

[0106] The coordinates of the sub-pixel corresponding to the main peak are used as the best matching position of the real-time image to be searched.

[0107] Image alignment is performed between the real-time image to be searched and the template image based on the optimal matching position.

[0108] Because the weighting mechanism employed in this application effectively suppresses spurious peaks, the response graph should exhibit a single, very sharp main peak with an extremely high signal-to-noise ratio. This characteristic enables the matching process to accurately pinpoint the optimal alignment position, avoiding mismatches caused by repetitive structures or other interfering factors.

[0109] Before performing template matching, this application introduces a novel weight calculation method based on local structure analysis to generate a structural feature weight map for the image. This weight map effectively highlights discriminative structural features while suppressing interference from highly repetitive regions. Based on this, the weight map is integrated into the subsequent matching process, significantly improving the signal-to-noise ratio of the matching response and fundamentally alleviating matching ambiguity caused by periodic patterns. Specifically, the weight generation mechanism automatically weakens the contribution of flat or highly similar regions within repeating units, while enhancing the weight of key locations such as macroscopic edges, corners, or structural boundaries with strong uniqueness, concentrating matching energy on truly meaningful regions. Thus, the matching process is intelligently guided to the most stable and discriminative image region, significantly improving alignment accuracy and result stability, avoiding "jumps" or "jitter" between adjacent repeating units, and significantly enhancing the algorithm's robustness to real-world interference factors such as noise, illumination fluctuations, and minor deformations.

[0110] Figure 2 This is a structural diagram of an image alignment device provided in an embodiment of this application, as shown below. Figure 2 As shown, based on the image alignment method provided in the preceding embodiments, this application also provides an image alignment apparatus comprising: The acquisition module is used to acquire template images and real-time images to be searched.

[0111] The weight map determination module is used to process the template image using a weight generation algorithm to obtain the weight map corresponding to the template image.

[0112] The weight map determination module is further used to process the real-time image to be searched using the weight generation algorithm to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is used to configure a corresponding weight value for each pixel in the template image or the real-time image to be searched.

[0113] The normalization module is used to store the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula, so as to obtain the stored response map.

[0114] The image alignment module is used to perform image alignment between the real-time image to be searched and the template image based on the stored response map.

[0115] As an optional embodiment, the weight graph determination module specifically includes: The gradient map generation unit is used to recursively calculate the template image to obtain the horizontal gradient map and the vertical gradient map corresponding to the template image.

[0116] The feature unit is used to obtain a first feature value and a second feature value corresponding to each pixel in the template image based on the horizontal gradient map and the vertical gradient map.

[0117] The weight map determination unit is used to obtain the weight map corresponding to the template image based on the first feature value and the second feature value corresponding to each pixel in the template image.

[0118] As an optional embodiment, the feature unit specifically includes: An element graph generation subunit is used to obtain a horizontal element graph, a vertical element graph, and a combined element graph based on the horizontal gradient graph and the vertical gradient graph; the horizontal element graph is obtained by squaring the horizontal gradient graph element by element; the vertical element graph is obtained by squaring the vertical gradient graph element by element; and the combined element graph is obtained by multiplying the horizontal gradient graph and the vertical gradient graph element by element.

[0119] The smoothing subunit is used to smooth the horizontal element map, the vertical element map, and the combined element map respectively to obtain the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map.

[0120] Tensor construction subunit is used to traverse each pixel in the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map, and dynamically construct the structure tensor of each pixel in the template image.

[0121] The decomposition subunit is used to perform eigenvalue decomposition on the structural tensor of each pixel in the template image to obtain the first eigenvalue and the second eigenvalue corresponding to each pixel in the template image.

[0122] As an optional embodiment, the smoothing processing unit is specifically used for: The horizontal element map, the vertical element map, and the combined element map are smoothed using a Gaussian smoothing filter to obtain the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map.

[0123] As an optional embodiment, the weight map determination unit is specifically used for: The weight value corresponding to each pixel in the template image is obtained by using the weight formula based on the first feature value and the second feature value corresponding to each pixel in the template image; Based on the weight value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

[0124] As an optional embodiment, the weighting formula is as follows: W(x,y)=(1-exp(-(λ(x,y)1-λ(x,y)2) 2 / σ 2 ))×(λ(x, y)1+λ(x, y)2); Where W(x, y) represents the weight value corresponding to the pixel at coordinate (x, y) in the template image; λ(x, y)1 represents the first feature value corresponding to the pixel at coordinate (x, y) in the template image; λ(x, y)2 represents the second feature value corresponding to the pixel at coordinate (x, y) in the template image; σ represents the scale parameter; and exp() represents the exponential function.

[0125] As an optional embodiment, the apparatus further includes: A response graph construction module is used to construct a response graph based on the real-time image to be searched and the template image; the width value of the response graph is configured as the sum of the width difference and a first preset value; the width difference is configured as the difference between the width value of the real-time image to be searched and the width value of the template image; the length value of the response graph is configured as the sum of the length difference and a second preset value; the length difference is configured as the difference between the length value of the real-time image to be searched and the width length value of the template image; and the response graph is initialized.

[0126] As an optional embodiment, the normalization module specifically includes: The normalization unit is used to obtain the matching value of each candidate sub-region in the real-time image to be searched based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula; the improved normalized cross-correlation formula is configured to introduce the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched for modulation when calculating the weight value.

[0127] A storage unit is used to store the matching value of each candidate sub-region in the real-time image to be searched into the response map to obtain the stored response map; the stored response map is used to store the matching values ​​of multiple candidate sub-regions in the real-time image to be searched.

[0128] As an optional embodiment, the normalization unit is specifically used for: Based on the weight map corresponding to the template image and the gray value of each pixel in the template image, the weighted average gray value of the template image is obtained; Based on the weight map corresponding to the real-time image to be searched and the gray value of each pixel in the real-time image to be searched, the weighted average gray value of each candidate sub-region in the real-time image to be searched is obtained. Multiple candidate sub-regions in the real-time image to be searched are extracted by traversing through a sliding window. For any candidate sub-region: Based on the weight value of each pixel in the candidate sub-region and the weight value of each pixel in the template image, a combined weight value for each pixel in the candidate sub-region is obtained; The matching value of the candidate sub-region is obtained based on the combined weight value of each pixel in the candidate sub-region, the gray value of each pixel in the template image, the weighted average gray value of the template image, the weighted average gray value of the candidate sub-region, and the gray value of each pixel in the candidate sub-region.

[0129] As an optional embodiment, the image alignment module specifically includes: The main peak determination unit is used to perform a global search on the stored response graph and take the maximum value point in the stored response graph as the main peak.

[0130] The optimal matching position determination unit is used to obtain the optimal matching position of the real-time image to be searched based on the coordinates of the pixel corresponding to the main peak.

[0131] The alignment unit is used to align the real-time image to be searched and the template image based on the optimal matching position.

[0132] As an optional embodiment, the optimal matching position determination unit is specifically used for: The target region is obtained based on the coordinates of the pixels corresponding to the main peak; A fitting algorithm is used to obtain the coordinates of the sub-pixels corresponding to the main peak based on the target region; the fitting algorithm includes quadratic surface fitting or centroid calculation. The coordinates of the sub-pixel corresponding to the main peak are used as the best matching position of the real-time image to be searched.

[0133] This application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an image alignment method.

[0134] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an image alignment method.

[0135] This application provides a computer program product, including a computer program that, when executed by a processor, implements an image alignment method.

[0136] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and equipment embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0137] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image alignment method, characterized in that, The method includes: Get the template image and the real-time image to be searched; The template image is processed using a weight generation algorithm to obtain a weight map corresponding to the template image; The weight generation algorithm is used to process the real-time image to be searched to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is used to configure a corresponding weight value for each pixel in the template image or the real-time image to be searched. The response map is stored based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using the improved normalized cross-correlation formula, and the stored response map is obtained. Based on the stored response graph, image alignment is performed between the real-time image to be searched and the template image; The method further includes: A response graph is constructed based on the real-time image to be searched and the template image; the width value of the response graph is configured as the sum of the width difference and a first preset value; the width difference is configured as the difference between the width value of the real-time image to be searched and the width value of the template image; the length value of the response graph is configured as the sum of the length difference and a second preset value; the length difference is configured as the difference between the length value of the real-time image to be searched and the width length value of the template image; The response graph is initialized. The process of storing the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula to obtain the stored response map specifically includes: The improved normalized cross-correlation formula is used to obtain the matching value of each candidate sub-region in the real-time image to be searched based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched; the improved normalized cross-correlation formula is configured to introduce the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched for modulation when calculating the weight value. The matching value of each candidate sub-region in the real-time image to be searched is stored in the response map to obtain the stored response map; the stored response map is used to store the matching values ​​of multiple candidate sub-regions in the real-time image to be searched.

2. The image alignment method according to claim 1, characterized in that, The step of processing the template image using a weight generation algorithm to obtain a weight map corresponding to the template image specifically includes: The template image is recursively calculated to obtain the horizontal gradient map and the vertical gradient map corresponding to the template image. Based on the horizontal gradient map and the vertical gradient map, the first feature value and the second feature value corresponding to each pixel in the template image are obtained; Based on the first feature value and the second feature value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

3. The image alignment method according to claim 2, characterized in that, The step of obtaining the first feature value and the second feature value corresponding to each pixel in the template image based on the horizontal gradient map and the vertical gradient map specifically includes: Based on the horizontal gradient map and the vertical gradient map, a horizontal element map, a vertical element map, and a combined element map are obtained; the horizontal element map is obtained by squaring the horizontal gradient map element by element; the vertical element map is obtained by squaring the vertical gradient map element by element; and the combined element map is obtained by multiplying the horizontal gradient map and the vertical gradient map element by element. The horizontal element map, the vertical element map, and the combined element map are smoothed respectively to obtain smoothed horizontal element map, smoothed vertical element map, and smoothed combined element map; By traversing each pixel in the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map, the structure tensor of each pixel in the template image is dynamically constructed. The eigenvalue decomposition is performed on the structural tensor of each pixel in the template image to obtain the first eigenvalue and the second eigenvalue corresponding to each pixel in the template image.

4. The image alignment method according to claim 3, characterized in that, The process of smoothing the horizontal element map, the vertical element map, and the combined element map to obtain smoothed horizontal element maps, smoothed vertical element maps, and smoothed combined element maps specifically includes: The horizontal element map, the vertical element map, and the combined element map are smoothed using a Gaussian smoothing filter to obtain the smoothed horizontal element map, the smoothed vertical element map, and the smoothed combined element map.

5. The image alignment method according to claim 2, characterized in that, The step of obtaining the weight map corresponding to the template image based on the first feature value and the second feature value corresponding to each pixel in the template image specifically includes: The weight value corresponding to each pixel in the template image is obtained by using the weight formula based on the first feature value and the second feature value corresponding to each pixel in the template image; Based on the weight value corresponding to each pixel in the template image, a weight map corresponding to the template image is obtained.

6. The image alignment method according to claim 5, characterized in that, The weighting formula is as follows: W(x,y)=(1-exp(-(λ(x,y)1-λ(x,y)2) 2 / σ 2 ))×(λ(x,y)1+λ(x,y)2); Where W(x, y) represents the weight value corresponding to the pixel at coordinate (x, y) in the template image; λ(x, y)1 represents the first feature value corresponding to the pixel at coordinate (x, y) in the template image; λ(x, y)2 represents the second feature value corresponding to the pixel at coordinate (x, y) in the template image; σ represents the scale parameter; and exp() represents the exponential function.

7. The image alignment method according to claim 1, characterized in that, The step of using an improved normalized cross-correlation formula to obtain the matching value of each candidate sub-region in the real-time image to be searched, based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched, specifically includes: Based on the weight map corresponding to the template image and the gray value of each pixel in the template image, the weighted average gray value of the template image is obtained; Based on the weight map corresponding to the real-time image to be searched and the gray value of each pixel in the real-time image to be searched, the weighted average gray value of each candidate sub-region in the real-time image to be searched is obtained. Multiple candidate sub-regions in the real-time image to be searched are extracted by traversing through a sliding window. For any candidate sub-region: Based on the weight value of each pixel in the candidate sub-region and the weight value of each pixel in the template image, a combined weight value for each pixel in the candidate sub-region is obtained; The matching value of the candidate sub-region is obtained based on the combined weight value of each pixel in the candidate sub-region, the gray value of each pixel in the template image, the weighted average gray value of the template image, the weighted average gray value of the candidate sub-region, and the gray value of each pixel in the candidate sub-region.

8. The image alignment method according to claim 1, characterized in that, The step of aligning the real-time image to be searched and the template image based on the stored response graph specifically includes: A global search is performed on the stored response graph, and the maximum value point in the stored response graph is taken as the main peak. Based on the coordinates of the pixel corresponding to the main peak, the optimal matching position of the real-time image to be searched is obtained; Image alignment is performed between the real-time image to be searched and the template image based on the optimal matching position.

9. The image alignment method according to claim 8, characterized in that, The step of obtaining the optimal matching position of the real-time image to be searched based on the coordinates of the pixel corresponding to the main peak specifically includes: The target region is obtained based on the coordinates of the pixels corresponding to the main peak; A fitting algorithm is used to obtain the coordinates of the sub-pixels corresponding to the main peak based on the target region; the fitting algorithm includes quadratic surface fitting or centroid calculation. The coordinates of the sub-pixel corresponding to the main peak are used as the best matching position of the real-time image to be searched.

10. An image alignment device, characterized in that, The image alignment device includes: The acquisition module is used to acquire template images and real-time images to be searched; The weight map determination module is used to process the template image using a weight generation algorithm to obtain the weight map corresponding to the template image; The weight map determination module is further configured to process the real-time image to be searched using the weight generation algorithm to obtain a weight map corresponding to the real-time image to be searched; the weight generation algorithm is configured to assign a corresponding weight value to each pixel in the template image or the real-time image to be searched. The normalization module is used to store the response map based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula, so as to obtain the stored response map. The image alignment module is used to perform image alignment between the real-time image to be searched and the template image based on the stored response map; The device also includes: A response graph construction module is used to construct a response graph based on the real-time image to be searched and the template image; the width value of the response graph is configured as the sum of a width difference and a first preset value; the width difference is configured as the difference between the width value of the real-time image to be searched and the width value of the template image; the length value of the response graph is configured as the sum of a length difference and a second preset value; the length difference is configured as the difference between the length value of the real-time image to be searched and the width length value of the template image; and the response graph is initialized. As an optional embodiment, the normalization module specifically includes: The normalization unit is used to obtain the matching value of each candidate sub-region in the real-time image to be searched based on the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched using an improved normalized cross-correlation formula; the improved normalized cross-correlation formula is configured to introduce the weight map corresponding to the template image and the weight map corresponding to the real-time image to be searched for modulation when calculating the weight value. A storage unit is used to store the matching value of each candidate sub-region in the real-time image to be searched into the response map to obtain the stored response map; the stored response map is used to store the matching values ​​of multiple candidate sub-regions in the real-time image to be searched.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image alignment method according to any one of claims 1-9.

Images