Image fine registration method and system based on optimal correlation features

Abstract

This invention provides a method and system for fine image registration based on optimal correlation features, comprising: Step S1: Detecting key feature points using the SIFT method; Step S2: Matching key feature points and calculating the homography matrix; Step S3: Completing coarse image matching using the homography matrix; Step S4: Calculating matching points between the reference image and the coarsely registered image using the SIFT method to obtain the largest connected component of the matching point distribution; Step S5: Determining the bounding rectangle of the largest connected component; Step S6: Calculating the correlation coefficient for each pixel within the bounding rectangle; Step S7: Selecting the top 10% of correlated feature points based on their correlation coefficients, and estimating the homography matrix accordingly; Step S8: Transforming the coarsely registered image using the homography matrix to complete fine image registration.

Description

Technical Field

[0001] This invention relates to the field of image registration accuracy improvement, specifically to an image fine registration method and system based on optimal correlation features. Background Technology

[0002] Image registration technology aims to perform geometric calibration on multiple remote sensing images of the same scene acquired by detectors from different platforms, at different times, and from different shooting angles. By establishing spatial mapping relationships between pixels, ensuring the consistency of matched pixels, it is an indispensable key preprocessing step for subsequent processing such as change detection, navigation and positioning, and image fusion.

[0003] Currently, image registration algorithms are mainly divided into two categories: those based on grayscale information and those based on feature information. The former estimates similarity by statistically analyzing grayscale features, but it lacks robustness when dealing with large-angle rotations or drastic changes in illumination. The latter, such as the Scale Invariant Feature Transform (SIFT) algorithm, has significant advantages in scale scaling and rotation invariance, but there is still room for improvement in registration accuracy in complex remote sensing scenarios.

[0004] For specific application scenarios, existing technologies (such as Chinese patent CN202111467224.X) disclose an image registration method based on inertial navigation data assistance. This method constructs a transformation matrix by combining inertial navigation system parameters and uses a local normalized similarity metric algorithm to achieve fast convergence and precise registration. However, such methods often heavily rely on the real-time performance and accuracy of external inertial navigation data. In the absence of high-precision auxiliary sensors or when inertial navigation data drifts, their universality is limited.

[0005] Considering that the correlation between pixel pairs matching the image to be registered and the reference image is a core factor affecting accuracy, it is particularly important to further mine the optimal correlation features in the pure image dimension. Therefore, this application proposes an image fine registration method and system based on optimal correlation features. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide an image fine registration method and system based on optimal correlation features.

[0007] According to one aspect of the present invention, an image fine registration method based on optimal correlation features includes:

[0008] Step S1: Detect key feature points using the SIFT method; Step S2: Perform key feature point matching and calculate the homography matrix; Step S3: Perform coarse image matching using the homography matrix; Step S4: Use the SIFT method to calculate matching points on the reference image and the coarse registration image, and obtain the maximum connected component of the matching point distribution; Step S5: Determine the bounding rectangle of the largest connected component; Step S6: Calculate the correlation coefficient for each pixel within the outer rectangle; Step S7: Select the top 10% of related feature points by correlation coefficient and estimate the homography matrix accordingly; Step S8: Transform the coarsely registered image using the homography matrix to complete the fine registration of the image.

[0009] Preferably, step S1 specifically includes: For the reference image and the image to be registered, the SIFT operator is used to detect key feature points in both images. The detection steps are as follows: Sub-step S1.1: Set the original image input to The size space factor is σ, and the scale space is generated by convolving the original input image with a Gaussian kernel. :

[0010] in, Representing the Gaussian kernel:

[0011] Sub-step S1.2: Through a series of size space factors Gaussian filtering is applied to the input image to obtain multiple layers and groups of images, which are then used to construct a Gaussian pyramid structure. The size space factor of adjacent layers within the same group is also considered. The following relationship exists:

[0012] in Indicates the space factor of adjacent layer dimensions. For fixed coefficients, S is the number of images whose features are to be extracted; Within the same group, the image size of each layer is consistent, and the space factor of the later layer is the same as that of the earlier layer. times; In different groups, the first image of the latter group is a half-downsampled result of the third-to-last image of the former group, and the image size is half that of the former group; Sub-step S1.3: Based on the Gaussian pyramid structure, subtract adjacent layers from each other for different layers in the same set of images to obtain the Gaussian difference pyramid structure:

[0013] in, It is a Gaussian difference function. For size space, It is a fixed coefficient.

[0014] Sub-step S1.4: Within the same group, traverse each layer of the Gaussian difference image except for the first and last layers, compare each pixel with its 8 neighbors at the same scale and 18 pixels at the adjacent scales above and below—a total of 26 candidate points—and retain only the local extreme points. By performing a ternary second-order Taylor series expansion of the scale-space Gaussian difference function near the extreme points, the location information of the true extreme points near the discrete extreme points is determined. The expression for the ternary second-order Taylor series expansion of the scale-space Gaussian difference function is as follows:

[0015] in, It is a Gaussian difference function. ; Sub-step S1.5: Set the derivative of the above equation to zero and find the extreme value:

[0016] Will Substituting into the above expansion, we obtain the equation for the corresponding extreme point:

[0017] in, This represents the coordinate offset, used to obtain the new coordinates of the extreme point; Sub-step S1.6: Remove low-contrast extreme points and select the remaining extreme points as key feature points.

[0018] Preferably, step S2 specifically includes: Using the key feature points, feature descriptors are generated and nearest neighbor matching is performed by allocating them in the reference image and the image to be registered according to the direction information of the key points. The Euclidean distance of the matching points is filtered by the threshold method. The homography matrix is ​​calculated based on the features of the matching points.

[0019] Preferably, step S2 includes: Sub-step S2.1: Assign directional information to the key feature point, that is, calculate the gradient magnitude and direction of pixels in the neighborhood of the key feature point:

[0020]

[0021] in, For gradient magnitude, As direction, For size space, The scale space in which the key feature point is located is determined by the scale space factor of the key feature point. Sub-step S2.2: Based on the gradient magnitude and direction, construct a histogram of gradient direction, wherein the histogram includes counting 36 statistical units, each statistical unit representing 10°; the peak value in the histogram represents the main direction of the key feature point, and the auxiliary direction is 80% of the peak value of the main direction; Sub-step S2.3: Using the key feature point as the calculation center, divide the neighborhood into 16×16 pixel areas, and further divide each neighborhood into 4×4 sub-regions. In each sub-region, use an 8-direction gradient histogram to calculate the cumulative gradient value and generate... The feature vectors corresponding to the key feature points of the dimension; Sub-step S2.4: Normalize the feature vector to eliminate the influence of illumination changes and obtain a feature descriptor, wherein the feature descriptor represents the position, scale and orientation information of key feature points; Sub-step S2.5: Using the feature descriptors of key feature points in the reference image and the image to be registered, the key feature points are matched using the nearest neighbor matching method, and the Euclidean distance of the feature descriptors is calculated:

[0022] Where d represents the Euclidean distance between the description vectors of two feature points. and These represent the feature description vectors of the reference image and the image to be registered, respectively; If the ratio of the Euclidean distance between the nearest neighbor and the second nearest neighbor is less than 0.7, it confirms that the key feature point matching is complete. Sub-step S2.6: Using the matched key feature points, calculate the homography matrix using the RANSAC method:

[0023] in , Represents the coordinates of the matching feature points. It is a homography matrix.

[0024] Preferably, step S3 specifically includes: Using the homography matrix, perspective transformation is performed on the image to be registered to complete coarse registration of the image to be registered to the reference image.

[0025] Preferably, step S4 specifically includes: Sub-step S4.1: Using the coarse registration image and the reference image, crop the coarse registration image to a size range consistent with the reference image, based on the size of the reference image; Sub-step S4.2: Use the SIFT method to detect key feature points on the cropped reference image and coarse registration image, allocate them according to the direction information of the key points, generate feature descriptors, and perform nearest neighbor matching based on the feature descriptors; Sub-step S4.3: Filter the Euclidean distance of matching points using the threshold method, calculate the connected component distribution of the matching points in the image, and obtain the largest connected component of the matching point distribution by comparing the number of pixels in the connected component.

[0026] Preferably, step S5 specifically includes: Based on the largest connected component of the matching point distribution, the bounding rectangle of the connected component is determined. The local region where the bounding rectangle is located is the region where the matching pixels are most densely distributed after the coarse registration process of the reference image and the image to be registered, that is, the region where the correlation between the images is strongest.

[0027] Preferably, step S6 specifically includes: Based on the pixel positions within the bounding rectangle, the correlation coefficient between pixel pairs between the reference image and the coarse registration image is calculated one by one, as follows: Extract the corresponding pixel values ​​of the matching key feature points located in the bounding rectangular region from the reference image and the coarse registration image, and calculate the correlation coefficient of all pixel pairs within the bounding rectangular region:

[0028] in, and This represents the size of the computational window selected within the bounding rectangular region in both the reference image and the coarse registration image. The window size is set to 4×4 pixels, and the two windows are identical in size and position. This represents the local similarity parameter in the calculation window. and This represents the average pixel value of the reference image and the image to be registered within the calculation window. This represents the coordinates of the pixels within the calculation window. and This represents the pixel values ​​of the reference image and the image to be registered.

[0029] Preferably, step S7 specifically includes: Sub-step S7.1: Using the correlation coefficient between each pixel in the bounding rectangle, sort the correlation coefficient values ​​to obtain a pixel correlation coefficient distribution from largest to smallest; Sub-step S7.2: Select the pixels with the highest correlation coefficients (top 10%) and calculate the homography matrix using their correlation feature information:

[0030] in, , This represents the coordinates of the top 10% of matching feature points after filtering. This is the homography matrix for precise registration.

[0031] According to another aspect of the present invention, an image fine registration system based on optimal correlation features is characterized by comprising: Module M1: Detects key feature points using the SIFT method; Module M2: Performs key feature point matching and calculates the homography matrix; Module M3: Performs coarse image matching using the homography matrix; Module M4: Utilizes the SIFT method to calculate matching points between the reference image and the coarse registration image, and obtains the maximum connected component of the matching point distribution; Module M5: Determines the bounding rectangle of the largest connected component; Module M6: Calculates the correlation coefficient for each pixel within the circumscribed rectangle; Module M7: Filter the top 10% of related feature points by correlation coefficient and estimate the homography matrix accordingly; Module M8: Transforms the coarsely registered image using the homography matrix to complete the fine registration of the image.

[0032] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves fine registration of the image to be registered by rapidly locating local regions with optimal correlation features based on coarse registration of the image to be registered and the reference image. It then calculates the homography matrix using the most correlated matching feature points and performs perspective transformation on the coarsely registered image. Existing image registration methods generally estimate similarity by determining the statistical features of grayscale information in the image, but lack robustness to constraints such as angle and illumination. Alternatively, they extract stable key features from the image and construct descriptors to complete registration, but the registration accuracy needs improvement. This method selects key feature points unaffected by image rotation and size changes, first improves registration robustness through coarse registration, and then calculates the fine registration transformation matrix for regions with optimal correlation features, thereby improving image registration accuracy and overcoming the limitations and deficiencies of existing technologies.

[0033] 2. The invention can be applied to the research field of image fine registration and is a very important technical means to improve the registration accuracy of satellite remote sensing images.

[0034] 3. This invention considers that correlation is a crucial influencing factor in the matching pixel pairs between the image to be registered and the reference image. Based on coarse image registration, it improves image registration accuracy by extracting the feature pairs with the highest similarity, i.e., the strongest correlation. By quickly locating the region containing the optimal correlated features, the processing speed of image registration is significantly improved, providing a new solution for the field of remote sensing image registration. Attached Figure Description

[0035] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the working method of the present invention.

[0036] Figure 2 This is a schematic diagram of the reference image and the image to be registered in this invention.

[0037] Figure 3 This is a schematic diagram showing the distribution of key feature points in the coarse registration process of this invention.

[0038] Figure 4 This is a schematic diagram of the key feature point matching results in the coarse registration process of this invention.

[0039] Figure 5 This is a schematic diagram of the image fine registration result in this invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0041] This invention utilizes the SIFT method for key feature point detection; key point matching and homography matrix calculation; coarse image matching using the homography matrix; SIFT method for calculating matching points between the reference image and the coarsely registered image to obtain the largest connected component of the matching point distribution; determination of the circumscribed rectangle of the largest connected component; calculation of correlation coefficients for each pixel within the circumscribed rectangle; calculation of the homography matrix using the top 10% of associated feature points; and transformation of the coarsely registered image using the homography matrix to complete fine image registration.

[0042] According to the present invention, an image fine registration method based on optimal correlation features is provided, such as... Figure 1 As shown, it includes the following steps: Step S1: Detect key feature points using the SIFT method. For the reference image and the image to be registered, the SIFT operator is used to detect key feature points in both images. The detection steps are as follows: First, set the original image input as... The size space factor is σ. A scale space is generated by convolving the original input image with a Gaussian kernel. :

[0043] in This represents the Gaussian kernel.

[0044]

[0045] Through a series of size space factors Gaussian filtering is applied to the input image to obtain multiple layers and sets of images, thus establishing a Gaussian pyramid structure. The size space factors of adjacent layers in the same group have the following relationship:

[0046] in Indicates the space factor of adjacent layer dimensions. For fixed coefficients, S is the number of images from which image features are to be extracted.

[0047] Within the same group, the image size of each layer is consistent, and the space factor of the later layer is the same as that of the earlier layer. The image size is 1 / 2 times that of the previous group; in different groups, the first image of the latter group is a half-downsampled result of the third-to-last image of the former group, and the image size is half that of the former group.

[0048] After obtaining the Gaussian pyramid structure, the difference of Gaussian (DOG) pyramid structure is obtained by subtracting adjacent layers from each layer in the same set of images.

[0049] in, It is a Gaussian difference function. For size space, It is a fixed coefficient.

[0050] For each Gaussian difference image in the same group, excluding the first and last layers, each pixel is compared with its 8 neighborhoods at the same scale and the 18 pixels at the adjacent scales above and below, for a total of 26 pixels, and extreme value pixels are retained.

[0051] To locate the true extremum points near discrete extremum points, a ternary second-order Taylor series expansion is performed on the scale-space Gaussian difference function near the extremum points:

[0052] in, It is a Gaussian difference function. .

[0053] To find the extreme value, let the derivative of the above equation be zero:

[0054] Will Substituting into the above expansion, we can obtain the corresponding extreme point equation as follows:

[0055] in, This represents the coordinate offset, which can be used to obtain the new coordinates of the extreme points. Low-contrast extreme points are removed, and the remaining extreme points are selected as key feature points.

[0056] Step S2: Keypoint matching and homography matrix calculation. Using the key feature points, feature descriptors are generated and nearest neighbor matching is performed in the reference image and the image to be registered, based on the key point orientation information. The Euclidean distance of the matching points is used for filtering. The homography matrix is ​​calculated based on the matching point features. First, the key feature points are assigned orientation information, i.e., the gradient magnitude and direction of pixels in the neighboring region of the key feature point are calculated:

[0057]

[0058] in, For gradient magnitude, As direction, The scale space in which it is located is determined by the scale space factor of the key feature point.

[0059] A histogram of gradient directions is constructed, counting a total of 36 statistical units, with each unit representing 10°. The peak value in the histogram represents the main direction of the key feature point, and the secondary direction is set to be 80% of the peak value of the main direction.

[0060] Using key feature points as the computation center, a 16×16 pixel neighborhood is divided, further subdivided into 4×4 sub-regions. Within each sub-region, an 8-directional gradient histogram is used to calculate the accumulated gradient value, thus generating... The key feature points in a given dimension are represented by feature vectors. These feature vectors are then normalized to eliminate the influence of illumination variations, ultimately yielding feature descriptors that describe the location, scale, and orientation information of the key feature points.

[0061] Using the feature descriptors of key feature points in the reference image and the image to be registered, the key feature points are matched using a nearest neighbor matching method. The Euclidean distance between the feature descriptors is calculated:

[0062] Where d represents the Euclidean distance between the description vectors of two feature points. and These represent the feature description vectors of the reference image and the image to be registered, respectively. If the ratio of the Euclidean distance between the nearest neighbor and the second nearest neighbor is less than 0.7, it confirms that the key feature point matching is complete.

[0063] Using the key feature points of the matching, the homography matrix is ​​calculated using the RANSAC (Random Sample Consensus) method:

[0064] in , Represents the coordinates of the matching feature points. It is a homography matrix.

[0065] Step S3: Perform coarse image matching using the homography matrix. Using the homography matrix, perform perspective transformation on the image to be registered to complete coarse registration from the image to the reference image.

[0066] Step S4: Calculate matching points for the reference image and the coarse registration image using the SIFT method to obtain the largest connected component of the matching point distribution. Using the coarse registration image and the reference image, crop the coarse registration image to a size range consistent with the reference image, based on the reference image's dimensions. Detect key feature points on the cropped reference image and the coarse registration image using the SIFT method, and generate feature descriptors based on the key point orientation information, performing nearest neighbor matching. Filter the matching points using the Euclidean distance using a thresholding method (same method as steps S1 and S2). Calculate the connected component distribution of the matching points in the image, and obtain the largest connected component of the matching point distribution by comparing the number of pixels in the connected components.

[0067] Step S5: Determine the bounding rectangle of the largest connected component. Based on the largest connected component of the matching point distribution, determine the bounding rectangle of this connected component. The local region where this bounding rectangle is located is the region where the matching pixels are most densely distributed after the coarse registration process between the reference image and the image to be registered, i.e., the region with the strongest correlation between the images.

[0068] Step S6: Calculate the correlation coefficient for each pixel within the circumscribed rectangle. Based on the pixel positions within the circumscribed rectangle, calculate the correlation coefficient between pixel pairs between the reference image and the coarse registration image. First, extract the pixel values ​​corresponding to the matching key feature points located in the region from both the reference image and the coarse registration image. Calculate the correlation coefficient for all pixel pairs within the region:

[0069] in, and This indicates the size of the computational window (set to 4×4 pixels) selected within the bounding rectangular region in both the reference image and the coarse registration image. Both windows have the same size and corresponding position. This represents the local similarity parameter in the calculation window. and This represents the average pixel value of the reference image and the image to be registered within the calculation window. This represents the coordinates of the pixels within the calculation window. and This represents the pixel values ​​of the reference image and the image to be registered.

[0070] Step S7: Select the top 10% of associated feature points to calculate the homography matrix. Using the correlation coefficients between pixels within the circumscribed rectangle, sort the correlation coefficients numerically to obtain a pixel correlation coefficient distribution from largest to smallest. Select the pixels with the highest correlation coefficients (top 10%) and calculate the homography matrix using their associated feature information:

[0071] in , This represents the coordinates of the top 10% of matching feature points after filtering. This is the homography matrix for precise registration.

[0072] Step S8: Transform the coarsely registered image using the homography matrix to complete the fine image registration. Using the homography matrix, perform perspective transformation on the coarsely registered image to finally complete the fine image registration.

[0073] Furthermore, in conjunction with the appendix Figures 1 to 5 The image fine registration method based on optimal correlation features of the present invention is described in detail below: Key feature point detection is performed using the SIFT method. For the reference image and the image to be registered, the SIFT operator is used to detect key feature points in both images. The detection steps are as follows: First, set the original image input as... ,like Figure 2As shown, the size space factor is σ. A scale space is generated by convolving a Gaussian kernel with the original input image. :

[0074] in This represents the Gaussian kernel.

[0075]

[0076] Through a series of size space factors Gaussian filtering is applied to the input image to obtain multiple layers and sets of images, thus establishing a Gaussian pyramid structure. The size space factors of adjacent layers in the same group have the following relationship:

[0077] in Indicates the space factor of adjacent layer dimensions. For fixed coefficients, S is the number of images from which image features are to be extracted.

[0078] Within the same group, the image size of each layer is consistent, and the space factor of the later layer is the same as that of the earlier layer. The image size is 1 / 2 times that of the previous group; in different groups, the first image of the latter group is a half-downsampled result of the third-to-last image of the former group, and the image size is half that of the former group.

[0079] After obtaining the Gaussian pyramid structure, the difference of Gaussian (DOG) pyramid structure is obtained by subtracting adjacent layers from each layer in the same set of images.

[0080] in, It is a Gaussian difference function. For size space, It is a fixed coefficient.

[0081] For each Gaussian difference image in the same group, excluding the first and last layers, each pixel is compared with its 8 neighborhoods at the same scale and the 18 pixels at the adjacent scales above and below, for a total of 26 pixels, and extreme value pixels are retained.

[0082] To locate the true extremum points near discrete extremum points, a ternary second-order Taylor series expansion is performed on the scale-space Gaussian difference function near the extremum points:

[0083] in, It is a Gaussian difference function. .

[0084] To find the extreme value, let the derivative of the above equation be zero:

[0085] Will Substituting into the above expansion, we can obtain the corresponding extreme point equation as follows:

[0086] in, This represents the coordinate offset, from which the new coordinates of the extreme points can be obtained. Low-contrast extreme points are removed, and the remaining extreme points are selected as key feature points, as shown in the following figure. Figure 3 As shown.

[0087] Keypoint matching and homography matrix calculation. Using the key feature points, feature descriptors are generated and nearest neighbor matching is performed in the reference image and the image to be registered, based on the orientation information of the key points. A thresholding method is used to filter the Euclidean distance of the matching points. The homography matrix is ​​calculated based on the features of the matching points. First, the key feature points are assigned orientation information, i.e., the gradient magnitude and direction of pixels in the neighboring region of the key feature point are calculated:

[0088]

[0089] in, For gradient magnitude, As direction, The scale space in which it is located is determined by the scale space factor of the key feature point.

[0090] A histogram of gradient directions is constructed, counting a total of 36 statistical units, with each unit representing 10°. The peak value in the histogram represents the main direction of the key feature point, and the secondary direction is set to be 80% of the peak value of the main direction.

[0091] Using key feature points as the computation center, a 16×16 pixel neighborhood is divided, further subdivided into 4×4 sub-regions. Within each sub-region, an 8-directional gradient histogram is used to calculate the accumulated gradient value, thus generating... The key feature points in a given dimension are represented by feature vectors. These feature vectors are then normalized to eliminate the influence of illumination variations, ultimately yielding feature descriptors that describe the location, scale, and orientation information of the key feature points.

[0092] Using the feature descriptors of key feature points in the reference image and the image to be registered, the key feature points are matched using a nearest neighbor matching method. The Euclidean distance between the feature descriptors is calculated:

[0093] Where d represents the Euclidean distance between the description vectors of two feature points. and These represent the feature description vectors of the reference image and the image to be registered, respectively. If the ratio of the Euclidean distance between the nearest neighbor and the second nearest neighbor is less than 0.7, it confirms that the key feature point matching is complete, and the matching result is as follows: Figure 4 As shown.

[0094] Using the key feature points of the matching, the homography matrix is ​​calculated using the RANSAC (Random Sample Consensus) method:

[0095] in , Represents the coordinates of the matching feature points. It is a homography matrix.

[0096] Coarse image matching is performed using a homography matrix. The homography matrix is ​​then used to perform perspective transformation on the image to be registered, completing the coarse registration from the image to the reference image.

[0097] The SIFT method is used to calculate matching points in the reference image and the coarse registration image to obtain the largest connected component of the matching point distribution. Using the coarse registration image and the reference image, the coarse registration image is cropped to the same size range as the reference image. The SIFT method is then used to detect key feature points in the cropped reference image and the coarse registration image, and feature descriptors are generated and nearest neighbor matching is performed based on the orientation information of the key points. A threshold method is used to filter the Euclidean distance of the matching points (the method is the same as steps S1 and S2). Using the matching points in the image, the connected component distribution is calculated, and the largest connected component of the matching point distribution is obtained by comparing the number of pixels in the connected components.

[0098] Determine the bounding rectangle of the largest connected component. Based on the largest connected component of the matching point distribution, determine the bounding rectangle of this connected component. The local region where this bounding rectangle is located is the region where the matching pixels are most densely distributed after the coarse registration process between the reference image and the image to be registered, i.e., the region with the strongest correlation between the images.

[0099] Calculate the correlation coefficient for each pixel within the circumscribed rectangle. Based on the pixel positions within the circumscribed rectangle, calculate the correlation coefficient between pixel pairs between the reference image and the coarse registration image. First, extract the pixel values ​​corresponding to the matching key feature points located in the region from both the reference image and the coarse registration image. Calculate the correlation coefficient for all pixel pairs within the region:

[0100] in, and This indicates the size of the computational window (set to 4×4 pixels) selected within the bounding rectangular region in both the reference image and the coarse registration image. Both windows have the same size and corresponding position. This represents the local similarity parameter in the calculation window. and This represents the average pixel value of the reference image and the image to be registered within the calculation window. This represents the coordinates of the pixels within the calculation window. and This represents the pixel values ​​of the reference image and the image to be registered.

[0101] The homography matrix is ​​calculated using the top 10% of associated feature points. The correlation coefficients between pixels within the bounding rectangle are then sorted numerically to obtain a distribution of pixel correlation coefficients from largest to smallest. The pixels with the highest correlation coefficients (top 10%) are selected, and their associated feature information is used to calculate the homography matrix.

[0102] in , This represents the coordinates of the top 10% of matching feature points after filtering. This is the homography matrix for precise registration.

[0103] The homography matrix is ​​used to transform the coarsely registered image to complete the fine image registration. The homography matrix is ​​then used to perform perspective transformation on the coarsely registered image to finally complete the fine image registration. The registration result is as follows: Figure 5 As shown.

[0104] The present invention also provides an image fine registration system based on optimal correlation features. The image fine registration system based on optimal correlation features can be implemented by executing the process steps of the image fine registration method based on optimal correlation features. That is, those skilled in the art can understand the image fine registration method based on optimal correlation features as a preferred embodiment of the image fine registration system based on optimal correlation features.

[0105] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0106] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for fine image registration based on optimal correlation features, characterized in that, include: Step S1: Detect key feature points using the SIFT method; Step S2: Perform key feature point matching and calculate the homography matrix; Step S3: Perform coarse image matching using the homography matrix; Step S4: Use the SIFT method to calculate matching points on the reference image and the coarse registration image, and obtain the maximum connected component of the matching point distribution; Step S5: Determine the bounding rectangle of the largest connected component; Step S6: Calculate the correlation coefficient for each pixel within the outer rectangle; Step S7: Select the top 10% of related feature points by correlation coefficient and estimate the homography matrix accordingly; Step S8: Transform the coarsely registered image using the homography matrix to complete the fine registration of the image.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: For the reference image and the image to be registered, the SIFT operator is used to detect key feature points in both images. The detection steps are as follows: Sub-step S1.1: Set the original image input to The size space factor is σ, and the scale space is generated by convolving the original input image with a Gaussian kernel. : in, Representing the Gaussian kernel: Sub-step S1.2: Through a series of size space factors Gaussian filtering is applied to the input image to obtain multiple layers and groups of images, which are then used to construct a Gaussian pyramid structure. The size space factor of adjacent layers within the same group is also considered. The following relationship exists: in Indicates the space factor of adjacent layer dimensions. For fixed coefficients, S is the number of images whose features are to be extracted; Within the same group, the image size of each layer is consistent, and the space factor of the later layer is the same as that of the earlier layer. times; In different groups, the first image of the latter group is a half-downsampled result of the third-to-last image of the former group, and the image size is half that of the former group; Sub-step S1.3: Based on the Gaussian pyramid structure, subtract adjacent layers from each other for different layers in the same set of images to obtain the Gaussian difference pyramid structure: in, It is a Gaussian difference function. For size space, It is a fixed coefficient. Sub-step S1.4: Within the same group, traverse each layer of the Gaussian difference image except for the first and last layers, compare each pixel with its 8 neighbors at the same scale and 18 pixels at the adjacent scales above and below—a total of 26 candidate points—and retain only the local extreme points. By performing a ternary second-order Taylor series expansion of the scale-space Gaussian difference function near the extreme points, the location information of the true extreme points near the discrete extreme points is determined. The expression for the ternary second-order Taylor series expansion of the scale-space Gaussian difference function is as follows: in, It is a Gaussian difference function. ; Sub-step S1.5: Set the derivative of the above equation to zero and find the extreme value: Will Substituting into the above expansion, we obtain the equation for the corresponding extreme point: in, This represents the coordinate offset, used to obtain the new coordinates of the extreme point; D is the Gaussian difference function of D(X). Sub-step S1.6: Remove low-contrast extreme points and select the remaining extreme points as key feature points.

3. The method according to claim 1, characterized in that, Step S2 specifically includes: Using the key feature points, feature descriptors are generated and nearest neighbor matching is performed by allocating them in the reference image and the image to be registered according to the direction information of the key points. The Euclidean distance of the matching points is filtered by the threshold method. The homography matrix is ​​calculated based on the features of the matching points.

4. The method according to claim 1, characterized in that, Step S2 includes: Sub-step S2.1: Assign directional information to the key feature point, that is, calculate the gradient magnitude and direction of pixels in the neighborhood of the key feature point: in, For gradient magnitude, As direction, For size space, The scale space in which the key feature point is located is determined by the scale space factor of the key feature point. Sub-step S2.2: Based on the gradient magnitude and direction, construct a histogram of gradient direction, wherein the histogram includes counting 36 statistical units, each statistical unit representing 10°; the peak value in the histogram represents the main direction of the key feature point, and the auxiliary direction is 80% of the peak value of the main direction; Sub-step S2.3: Using the key feature point as the calculation center, divide the neighborhood into 16×16 pixel areas, and further divide each neighborhood into 4×4 sub-regions. In each sub-region, use an 8-direction gradient histogram to calculate the cumulative gradient value and generate... The feature vectors corresponding to the key feature points of the dimension; Sub-step S2.4: Normalize the feature vector to eliminate the influence of illumination changes and obtain a feature descriptor, wherein the feature descriptor represents the position, scale and orientation information of key feature points; Sub-step S2.5: Using the feature descriptors of key feature points in the reference image and the image to be registered, the key feature points are matched using the nearest neighbor matching method, and the Euclidean distance of the feature descriptors is calculated: Where d represents the Euclidean distance between the description vectors of two feature points. and These represent the feature description vectors of the reference image and the image to be registered, respectively; If the ratio of the Euclidean distance between the nearest neighbor and the second nearest neighbor is less than 0.7, it confirms that the key feature point matching is complete. Sub-step S2.6: Using the matched key feature points, calculate the homography matrix using the RANSAC method: in , Represents the coordinates of the matching feature points. It is a homography matrix.

5. The method according to claim 1, characterized in that, Step S3 specifically includes: Using the homography matrix, perspective transformation is performed on the image to be registered to complete coarse registration of the image to be registered to the reference image.

6. The method according to claim 1, characterized in that, Step S4 specifically includes: Sub-step S4.1: Using the coarse registration image and the reference image, crop the coarse registration image to a size range consistent with the reference image, based on the size of the reference image; Sub-step S4.2: Use the SIFT method to detect key feature points on the cropped reference image and coarse registration image, allocate them according to the direction information of the key points, generate feature descriptors, and perform nearest neighbor matching based on the feature descriptors; Sub-step S4.3: Filter the Euclidean distance of matching points using the threshold method, calculate the connected component distribution of the matching points in the image, and obtain the largest connected component of the matching point distribution by comparing the number of pixels in the connected component.

7. The method according to claim 1, characterized in that, Step S5 specifically includes: Based on the largest connected component of the matching point distribution, the bounding rectangle of the connected component is determined. The local region where the bounding rectangle is located is the region where the matching pixels are most densely distributed after the coarse registration process of the reference image and the image to be registered, that is, the region where the correlation between the images is strongest.

8. The method according to claim 1, characterized in that, Step S6 specifically includes: Based on the pixel positions within the bounding rectangle, the correlation coefficient between pixel pairs between the reference image and the coarse registration image is calculated one by one, as follows: Extract the corresponding pixel values ​​of the matching key feature points located in the bounding rectangular region from the reference image and the coarse registration image, and calculate the correlation coefficient of all pixel pairs within the bounding rectangular region: in, and This represents the size of the computational window selected within the bounding rectangular region in both the reference image and the coarse registration image. The window size is set to 4×4 pixels, and the two windows are identical in size and position. This represents the local similarity parameter in the calculation window. and This represents the average pixel value of the reference image and the image to be registered within the calculation window. This represents the coordinates of the pixels within the calculation window. and This represents the pixel values ​​of the reference image and the image to be registered.

9. The method according to claim 1, characterized in that, Step S7 specifically includes: Sub-step S7.1: Using the correlation coefficient between each pixel in the bounding rectangle, sort the correlation coefficient values ​​to obtain a pixel correlation coefficient distribution from largest to smallest; Sub-step S7.2: Select the pixels with the highest correlation coefficients (top 10%) and calculate the homography matrix using their correlation feature information: in, , This represents the coordinates of the top 10% of matching feature points after filtering. This is the homography matrix for precise registration.

10. An image registration system based on optimal correlation features, characterized in that, include: Module M1: Detects key feature points using the SIFT method; Module M2: Performs key feature point matching and calculates the homography matrix; Module M3: Performs coarse image matching using the homography matrix; Module M4: Utilizes the SIFT method to calculate matching points between the reference image and the coarse registration image, and obtains the maximum connected component of the matching point distribution; Module M5: Determines the bounding rectangle of the largest connected component; Module M6: Calculates the correlation coefficient for each pixel within the circumscribed rectangle; Module M7: Filter the top 10% of related feature points by correlation coefficient and estimate the homography matrix accordingly; Module M8: Transforms the coarsely registered image using the homography matrix to complete the fine registration of the image.

Images