An adaptive template registration method based on image gray scale
By using an adaptive template registration method and leveraging FAST corner detection and the NCC algorithm, the problems of low computational efficiency and low accuracy in images with weak texture or indistinct features are solved, achieving efficient and accurate image registration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING AEROSPACE FEITENG EQUIPMENT TECHNOLOGY CO LTD
- Filing Date
- 2026-03-23
- Publication Date
- 2026-06-26
AI Technical Summary
Existing image registration methods are computationally inefficient and have low matching accuracy in images with weak texture or indistinct features. In particular, traditional template selection leads to computationally time-consuming and insufficient matching accuracy in the background alignment process between images in a video sequence.
An adaptive template registration method based on image grayscale is adopted. By detecting FAST corner points, non-maximum suppression, Euclidean distance clustering and NCC algorithm, the minimum bounding rectangle template is extracted, and the best matching position is calculated by affine transformation to complete image registration.
It improves the computational efficiency and matching accuracy of image registration, expands the scope of application, performs particularly well in smooth or low-texture images, and reduces the dependence on feature points.
Smart Images
Figure CN122289331A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing and machine vision, and relates to an image registration method, particularly an adaptive template registration method based on image grayscale. Background Technology
[0002] Commonly used traditional image registration methods include feature-based image registration and region-based image registration. Feature-based image registration relies on the quality and stability of feature extraction and is not suitable for images with weak texture or indistinct features. For image registration between video sequences, since the changes in illumination, translation, rotation, and scale between adjacent images are very small, region-based image registration methods have a wider range of applications. Currently, in template matching algorithms aimed at achieving background alignment, the template is usually selected as a central region slightly smaller than the image size. This template selection method has two problems: first, the area is too large, resulting in low computational efficiency and being very time-consuming; second, the image texture may not be rich, leading to low matching accuracy. Summary of the Invention
[0003] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art and provide an adaptive template registration method based on image grayscale to improve the accuracy and computational efficiency of the template matching algorithm.
[0004] The technical solution of this invention is: an adaptive template registration method based on image grayscale, comprising: FAST corner points are detected from the baseline image in the sequence diagram, and non-maximum suppression is applied to the detected corner points to obtain the final corner points; The final corner points are clustered based on Euclidean distance, and the minimum bounding rectangle of each corner point set is extracted as a template based on the clustering results. On the current image in the sequence image, the NCC algorithm is used to find the best matching position for the obtained template, and the center coordinates of the template on the reference image and the coordinates of the best matching position on the real-time image are saved. Based on the center coordinates of the multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, the affine transformation coefficients between the reference image and the current image are calculated according to the affine transformation formula to complete the registration work.
[0005] The step of detecting FAST corner points from the baseline image in the sequence image and performing non-maximum suppression on the detected corner points to obtain the final corner points includes: The FAST corner detection algorithm is used to select candidate points from the baseline image in the sequence image; For each candidate point, traverse its 16 pixels in its circular neighborhood. If there are n consecutive points that satisfy the condition that the gray values of the pixels are significantly greater than or significantly less than the corresponding candidate point's pixel values, and the gray value difference between the candidate point and the corresponding candidate point exceeds the threshold T, then the candidate point is marked as a FAST corner point. Calculate the response value at each FAST corner point; The sliding size on the baseline is... The window is selected, and the FAST corner with the largest response value is retained as the final corner, while other corners within the window are discarded.
[0006] The FAST corner detection algorithm is used to filter candidates from the baseline image in the sequence image, including: The first frame of each sequence image is used as the reference image. The FAST corner detection algorithm is used to traverse each pixel p on the reference image and find the pixel values at positions 1, 5, 9, and 13 in the circular neighborhood of p. If at least 3 of these points satisfy the condition that the gray value of the pixel is much greater than or much less than that of the center pixel p, and the gray value difference between the pixel p and the center pixel p exceeds the threshold T, then the pixel p is defined as a candidate point; otherwise, it is directly determined as a non-corner point.
[0007] The calculation of the response value for each FAST corner point includes: For each FAST corner point, calculate the absolute differences between the corner point and 16 points in its circular neighborhood. Then, subtract a threshold T from each of the 16 absolute differences. Sum the 16 differences that are greater than 0, adjust the offset, and take the minimum of this sum as the response value. The offset range is... .
[0008] The step of clustering the final corner points based on Euclidean distance and extracting the minimum bounding rectangle of each corner point set as a template based on the clustering results includes: The resulting corner points are sorted from largest to smallest according to their response values, and then clustered according to the corner point arrangement order based on Euclidean distance; in each clustering, the corner point with the largest response value among the unclassified corner points is taken as the cluster center. Determine whether all corner points in each corner point set obtained through clustering are collinear; if they are collinear, delete the corner point set; if the total number of sets after deletion is greater than or equal to 3, save the template center coordinates and size information. The minimum bounding rectangle of each set of corner points is calculated sequentially. The area enclosed by this bounding rectangle on the reference map is used as a template, and the center coordinates and size information of the template are saved.
[0009] The step involves using the NCC algorithm to find the optimal matching position for the obtained template on the current image in the sequence image, and saving the center coordinates of the template on the reference image and the coordinates of the optimal matching position on the real-time image, including: 11) Take any template and denote it as A. The size of template A is... Let B be the area covered by template A when it slides across the current image in the sequence. Calculate the average grayscale values of template A and template B:
[0010]
[0011] Where i is the x-coordinate of a pixel in the image, and j is the y-coordinate of a pixel in the image. and These are the grayscale mean values of A and B, respectively. 12) Subtract the corresponding grayscale mean from each pixel on templates A and B to obtain the normalized image. , :
[0012]
[0013] 13) Based on the obtained and Calculate the NCC value between templates A and B: ; 14) Slide template A on the current image of the sequence image in order from left to right and from top to bottom. At each position, calculate the NCC value of the region B covered by template A when it slides on the current image of the sequence image. When template A has finished sliding on the entire current image, the position with the maximum NCC value is recorded as the best matching position of template A on the current image. Save the coordinates of the center of the region of template A on the reference image and the coordinates of the obtained best matching position. 15) Select the next template and repeat steps 11)-14) according to the calculation method of template A to obtain the best matching position of each template on the current image, and save the area center coordinates of each template on the reference image and the obtained best matching position coordinates respectively.
[0014] The registration process involves calculating the affine transformation parameters of the reference image and the current image based on the center coordinates of the multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, according to the affine transformation formula, to complete the registration work, including: Based on the center coordinates of the multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, the affine transformation parameters of the reference image and the current image are calculated according to the affine transformation formula. After obtaining the affine transformation parameters, perform a process on each pixel of the registered image. Using the inverse transformation matrix Calculate its corresponding position in the reference image. Represented as: ,in It is the affine transformation parameter matrix The inverse transformation matrix; pass Pixels at integer coordinates around the reference map are calculated at non-integer coordinates. The gray value at the specified location is then assigned to the registered image. The registration process is completed by examining the pixels at each location.
[0015] The step of calculating the affine transformation parameters of the reference image and the current image based on the center coordinates of the obtained multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, according to the affine transformation formula, includes: Affine transformation is a combination of identity transformations, scaling transformations, rotation transformations, translation transformations, shearing transformations, and reflection transformations, and is represented as: ;in, These are the pixel coordinates in the reference image. These are the pixel coordinates in the transformed, registered image. Affine transformation matrix; the form of an affine transformation is:
[0016] Transformed into equation form:
[0017]
[0018] in The six parameters are the affine transformation parameters, which require at least three pairs of coordinate points to form six equations for solving. When there are more than three pairs of center coordinates, the NCC values are arranged from largest to smallest, and the first three sets are substituted into the affine transformation equations to calculate the parameters.
[0019] Using a bilinear interpolation algorithm, through Pixels at integer coordinates around the reference map are calculated at non-integer coordinates. The grayscale values at the location include: On the known reference map , , , Calculate the non-integer coordinates on the reference image by taking the grayscale values of the pixels at four integer coordinates. The grayscale value at that location is determined through the following process: exist The direction is correct , Linear interpolation of two points yields...
[0020]
[0021] according to , right Point interpolation:
[0022] Finally, after simplification, we get: .
[0023] The advantages of this invention compared to the prior art are: (1) When an image is too smooth and has little texture (such as a desert background), the number of feature points that can be extracted from the image is small and of low quality. At this time, the image registration accuracy based on feature points is very poor. This invention uses templates for registration, which reduces the dependence on image feature points and has a wider range of applications.
[0024] (2) In current template matching algorithms aimed at achieving background alignment, the template is usually selected as a central region slightly smaller than the image size. Compared with the template extracted in this invention, there are two disadvantages: firstly, the area is too large, resulting in low computational efficiency and being very time-consuming; secondly, the image texture may not be rich, leading to low matching accuracy. This invention selects a template region based on strong corner point clustering. The template region is small and has rich template information, which greatly improves the registration accuracy and computational efficiency. Attached Figure Description
[0025] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention; Figure 2 This is a circular template image for the FAST algorithm. Figure 3 This is a bilinear interpolation plot. Detailed Implementation
[0026] like Figure 1 As shown, the specific implementation steps of this invention are as follows: Step 1: Detect FAST corner points from the baseline image and perform non-maximum suppression on the detected corner points to obtain the final corner points. (11) Rapid screening of candidate points The FAST (Features from Accelerated Segment Test) corner detection algorithm is a highly efficient feature point extraction algorithm in the field of computer vision. Its core advantages are real-time performance and high computational efficiency. This algorithm is based on the distribution characteristics of pixel grayscale value differences in a circular neighborhood. The principle is as follows: given a pixel p, draw a circle with a radius of 3 centered on p, and compare 16 pixels on the circle with p for calculation, as shown in the attached manual. Figure 2 As shown. If there are n consecutive pixels among the 16 pixels that satisfy the condition that the gray values of all pixels are significantly greater than or significantly less than the center pixel p, and the gray value difference between the center pixel p and the center pixel p exceeds the threshold T (T=10), then p is determined to be a corner point.
[0027] In the fast candidate point selection, the first frame of each sequence image is used as the reference image. For each pixel p in the reference image, the pixel values at positions 1, 5, 9, and 13 in the circular neighborhood of p are found, as shown in the attached diagram. Figure 2 As shown in the diagram. If at least 3 of these points satisfy the condition that the grayscale value of each pixel is significantly greater than or significantly less than that of the center pixel p, and the grayscale difference between each pixel and the center pixel p exceeds the threshold T (T=10), then pixel p will proceed to the next step of precise corner detection; otherwise, it will be directly determined as a non-corner point.
[0028] (12) Precise detection of candidate points For each candidate point that passes the initial screening in (11), traverse the 16 pixels in its circular neighborhood. If there are n (n=12) consecutive points that satisfy that the gray value of each pixel is significantly greater than or significantly less than the corresponding candidate point pixel value, and the gray value difference with the corresponding candidate point exceeds the threshold T (T=10), then the candidate point is marked as a FAST corner point.
[0029] (13) Calculation of corner response values For each FAST corner point obtained from (12), calculate the absolute difference between the corner point and 16 points in its circular neighborhood. Then, subtract the threshold T from each of the 16 absolute differences. Sum the results of the 16 differences that are greater than 0. Adjust the offset and take the minimum value of this sum as the response value. The range of the offset is: (T=10). The corner response value reflects the salience of the corner: when the response value is large, it indicates that even if the center point intensity is adjusted, there is still a significant difference between the center point and the surrounding pixels, indicating that the current corner point is of good quality and is a strong corner point; conversely, when the response value is small, it indicates that a small adjustment of the center point intensity can eliminate the difference with the surrounding pixels, indicating that the current corner point is a weak corner point.
[0030] (14) Nonmaximum suppression Slide a size of 5 on the baseline graph. A window size of 5 is used, and the FAST corner point with the largest response value within the window is retained as the final corner point, while other corner points within the window are discarded. By reducing the number of corner points, the algorithm's running efficiency is improved.
[0031] Step 2: Cluster the final corner points obtained in step (1) based on Euclidean distance, and extract the minimum bounding rectangle of each corner point set as a template based on the clustering results. (21) Corner clustering The final corner points obtained in step (1) are sorted from largest to smallest according to their response values, and then clustered based on Euclidean distance according to the corner point arrangement. In each clustering, the corner point with the largest response value among the unclassified corner points is taken as the cluster center.
[0032] (22) Determine whether all corner points in each corner point set obtained by clustering in step (21) are collinear. If they are collinear, delete the corner point set. If the total number of sets after deletion is greater than or equal to 3, save the template center coordinates and size information; (23) Calculate the minimum bounding rectangle of each set of corner points obtained in step (22) in sequence, use the area enclosed by the bounding rectangle on the reference map as a template, and save the center coordinates and size information of the template.
[0033] Step 3: Use the NCC algorithm on the current image of the sequence image to find the best matching position for the template obtained in step (2), and save the center coordinates of the template on the reference image and the coordinates of the best matching position on the real-time image (31). The NCC algorithm finds the best matching position of the template by calculating the similarity between the template and each local region on the current image, which is the basis of template matching in image processing. Its implementation steps are as follows: Take out one of the templates obtained in step (23) and denote it as A. The size of template A is Let B be the area covered by the template as it slides across the current image in the sequence. Calculate the grayscale mean of A and B:
[0034]
[0035] Where i is the x-coordinate of a pixel in the image, and j is the y-coordinate of a pixel in the image. and These are the grayscale mean values of A and B, respectively.
[0036] (32) In order to remove the influence of brightness changes on the image, subtract the corresponding gray mean value obtained in step (31) from each pixel on A and B in step (31) to obtain the normalized image. , The formula is shown below:
[0037]
[0038] (33) According to step (32) and Calculate the NCC value between A and B using the following formula:
[0039] Where i is the x-coordinate of a pixel in the image, j is the y-coordinate of a pixel in the image, m is the width of template A, and n is the height of template A.
[0040] (34) Slide A sequentially on the current image of the sequence image from left to right and from top to bottom. At each position, calculate the NCC value of A and the area B covered by A when sliding on the current image of the sequence image. When A has finished sliding on the entire current image, the position with the maximum NCC value is recorded as the best matching position of template A on the current image. Save the coordinates of the center of the region of A on the reference image and the coordinates of the obtained best matching position. Similarly, the other templates obtained in step (23) are also calculated in the same way as template A to obtain their best matching positions on the current image. Save the coordinates of the center of the region of each template on the reference image and the coordinates of the obtained best matching position respectively.
[0041] Step 4: Based on the center coordinates of the multiple sets of templates obtained in step (3) on the reference image and the best matching position coordinates on the real-time image, calculate the affine transformation parameters of the reference image and the current image according to the affine transformation formula, and complete the registration work between the two images. (41) Based on the center coordinates of the multiple sets of templates obtained in step (3) on the reference image and the best matching position coordinates on the real-time image, calculate the affine transformation parameters of the reference image and the current image according to the affine transformation formula. The affine transformation is a combination of identity transformation, scaling transformation, rotation transformation, translation transformation, shearing transformation, and reflection transformation, expressed by the formula: ;in, These are the pixel coordinates in the reference image. These are the pixel coordinates in the transformed, registered image. Affine transformation matrix. The affine transformation is expressed as:
[0042] Transformed into equation form:
[0043]
[0044] in The six parameters are the affine transformation parameters, which require at least three pairs of coordinate points to form six equations for solving. Step three yields at least three pairs of center coordinates. If there are more than three pairs of center coordinates, arrange the NCC values from largest to smallest and take the first three pairs to substitute into the affine transformation equations to calculate the parameters.
[0045] (42) Image registration is the process of transforming two or more images obtained from different times, different sensors, or different shooting angles into a unified coordinate system by obtaining coordinate transformation parameters between images based on a certain similarity metric. An affine transformation is performed on the reference image, and bilinear interpolation is used to refine the transformed image, thus completing image registration. After obtaining the affine transformation parameters according to step (41), each pixel in the registered image is... Using the inverse transformation matrix Calculate its corresponding position in the reference image. This can be expressed as a formula: ,in It is the affine transformation parameter matrix The inverse transformation matrix; (43) The reference map obtained according to step (42) These coordinates are often not integers, while pixels on the baseline image are all located at integer coordinates. Since there are no pixels on it, it is necessary to calculate it using the pixels at integer coordinates around it. The grayscale value is then assigned to the grayscale value on the registered image. The pixel at that location. Bilinear interpolation can efficiently calculate this grayscale value. The calculation formula is shown below: On the known reference map , , , Find the grayscale values of pixels at four integer coordinates, and calculate the non-integer coordinates on the reference image. grayscale value at, such as Figure 3 As shown: First in The direction is correct , Linear interpolation of two points yields...
[0046]
[0047] Then according to , right Point interpolation:
[0048] Simplifying, we get: .
[0049] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention based on the above-disclosed technical content without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.
Claims
1. An image gray level based adaptive template registration method, characterized by, include: FAST corner points are detected from the baseline image in the sequence diagram, and non-maximum suppression is applied to the detected corner points to obtain the final corner points; The final corner points are clustered based on Euclidean distance, and the minimum bounding rectangle of each corner point set is extracted as a template based on the clustering results. On the current image in the sequence image, the NCC algorithm is used to find the best matching position for the obtained template, and the center coordinates of the template on the reference image and the coordinates of the best matching position on the real-time image are saved. Based on the center coordinates of the multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, the affine transformation coefficients between the reference image and the current image are calculated according to the affine transformation formula to complete the registration work.
2. The gray scale based adaptive template image registration method of claim 1, wherein: The step of detecting FAST corner points from the baseline image in the sequence image and performing non-maximum suppression on the detected corner points to obtain the final corner points includes: The FAST corner detection algorithm is used to select candidate points from the baseline image in the sequence image; For each candidate point, traverse its 16 pixels in its circular neighborhood. If there are n consecutive points that satisfy the condition that the gray values of the pixels are significantly greater than or significantly less than the corresponding candidate point's pixel values, and the gray value difference between the candidate point and the corresponding candidate point exceeds the threshold T, then the candidate point is marked as a FAST corner point. Calculate the response value at each FAST corner point; Sliding a window of size on the reference image, the FAST corner with the largest response value within the window is reserved as the final corner, and other corners within the window are discarded.
3. The gray scale based adaptive template image registration method of claim 2, wherein: The FAST corner detection algorithm is used to filter candidates from the baseline image in the sequence image, including: The first frame of each sequence image is used as the reference image. The FAST corner detection algorithm is used to traverse each pixel p on the reference image and find the pixel values at positions 1, 5, 9, and 13 in the circular neighborhood of p. If at least 3 of these points satisfy the condition that the gray value of the pixel is much greater than or much less than that of the center pixel p, and the gray value difference between the pixel p and the center pixel p exceeds the threshold T, then the pixel p is defined as a candidate point; otherwise, it is directly determined as a non-corner point.
4. The gray scale based adaptive template image registration method of claim 3, wherein: The calculation of the response value for each FAST corner point includes: For each FAST corner, the absolute difference between the corner and 16 points in the circular neighborhood of the corner is calculated, then the 16 absolute differences are subtracted by the threshold T respectively, and the sum of the results greater than 0 in the 16 differences is taken, and the minimum value of the sum is taken as the response value by adjusting the offset, wherein the range of the offset is .
5. The grayscale-based adaptive template image registration method according to claim 2, characterized in that: The step of clustering the final corner points based on Euclidean distance and extracting the minimum bounding rectangle of each corner point set as a template based on the clustering results includes: The resulting corner points are sorted from largest to smallest according to their response values, and then clustered according to the corner point arrangement order based on Euclidean distance; in each clustering, the corner point with the largest response value among the unclassified corner points is taken as the cluster center. Determine whether all corner points in each corner point set obtained through clustering are collinear; if they are collinear, delete the corner point set; if the total number of sets after deletion is greater than or equal to 3, save the template center coordinates and size information. The minimum bounding rectangle of each set of corner points is calculated sequentially. The area enclosed by this bounding rectangle on the reference map is used as a template, and the center coordinates and size information of the template are saved.
6. The grayscale-based adaptive template image registration method according to claim 5, characterized in that: The step involves using the NCC algorithm to find the optimal matching position for the obtained template on the current image in the sequence image, and saving the center coordinates of the template on the reference image and the coordinates of the optimal matching position on the real-time image, including: 11) Take any template and denote it as A. The size of template A is... Let B be the area covered by template A when it slides across the current image in the sequence. Calculate the average grayscale values of template A and template B: wherein i is the horizontal coordinate of the pixel point on the image, j is the vertical coordinate of the pixel point on the image, with respectively the mean value of the gray scale of A and B. 12) Subtract the obtained corresponding gray level mean value from each pixel point on the templates A and B to obtain a normalized image 、 : 13) Calculate the NCC value between template A and B: and , calculate the NCC value between template A and B: ； 14) Slide template A on the current image of the sequence image in order from left to right and from top to bottom. At each position, calculate the NCC value of the region B covered by template A when it slides on the current image of the sequence image. When template A has finished sliding on the entire current image, the position with the maximum NCC value is recorded as the best matching position of template A on the current image. Save the coordinates of the center of the region of template A on the reference image and the coordinates of the obtained best matching position. 15) Select the next template and repeat steps 11)-14) according to the calculation method of template A to obtain the best matching position of each template on the current image, and save the area center coordinates of each template on the reference image and the obtained best matching position coordinates respectively.
7. The grayscale-based adaptive template image registration method according to claim 6, characterized in that: The registration process involves calculating the affine transformation parameters of the reference image and the current image based on the center coordinates of the multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, according to the affine transformation formula, to complete the registration work, including: Based on the center coordinates of the multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, the affine transformation parameters of the reference image and the current image are calculated according to the affine transformation formula. After obtaining the affine transformation parameters, perform a process on each pixel of the registered image. Using the inverse transformation matrix Calculate its corresponding position in the reference image. Represented as: ,in It is the affine transformation parameter matrix The inverse transformation matrix; pass Pixels at integer coordinates around the reference map are calculated at non-integer coordinates. The gray value at the specified location is then assigned to the value on the registered image. The registration process is completed by examining the pixels at each location.
8. The grayscale-based adaptive template image registration method according to claim 7, characterized in that: The step of calculating the affine transformation parameters of the reference image and the current image based on the center coordinates of the obtained multiple sets of templates on the reference image and the best matching position coordinates on the real-time image, according to the affine transformation formula, includes: Affine transformation is a combination of identity transformations, scaling transformations, rotation transformations, translation transformations, shearing transformations, and reflection transformations, and is represented as: ;in, These are the pixel coordinates in the reference image. These are the pixel coordinates in the transformed, registered image. Affine transformation matrix; the affine transformation is expressed as: Transformed into equation form: in The six parameters are the affine transformation parameters, which require at least three pairs of coordinate points to form six equations for solving. When there are more than three pairs of center coordinates, the NCC values are arranged from largest to smallest, and the first three sets are substituted into the affine transformation equations to calculate the parameters.
9. The gray scale based adaptive template image registration method of claim 7, wherein: Using a bilinear interpolation algorithm, through Pixels at integer coordinates around the reference map are calculated at non-integer coordinates. The grayscale values at the location include: On the known reference map , , , Calculate the non-integer coordinates on the reference image by taking the grayscale values of the pixels at four integer coordinates. The grayscale value at that location is determined through the following process: exist The direction is correct , Linear interpolation of two points yields... according to , right Point interpolation: Finally, after simplification, we get: 。