Image matching method fusing iterative weighted regression and affine transformation to improve RANSAC (Random Sample Consensus)

Abstract

The invention belongs to the technical field of image matching, and particularly relates to an image matching method for improving RANSAC (Random Sample Consensus) by fusing iterative weighted regression and affine transformation, which comprises the following steps of: S1, inputting a reference image and a target image, and carrying out graying preprocessing on the two images; s2, key feature points are extracted from the grayed reference image and target image by using a feature extraction algorithm, then feature point matching is performed in combination with an FLANN algorithm, the FLANN finds the most similar feature point pair through approximate nearest neighbor search, and a preliminary matching point set is generated as a basis for subsequent mismatching elimination and optimization; through the affine matrix RANSAC method optimized by iterative weight regression, the problem that the matching precision is reduced due to mismatching caused by various transformation influences such as rotation, zooming and non-rigid deformation after preliminary matching of the image can be better solved, and the robustness and reliability of the matching method are enhanced.

Description

Technical Field

[0001] The invention relates to the technical field of image matching, and in particular to an image matching method integrating iterative weighted regression with affine transformation to improve RANSAC. Background Art

[0002] After the image is extracted and matched, many mismatches are inevitably produced due to the influence of non-rigid deformation, complex image changes and noise interference. When mismatches reduce the matching accuracy, it is necessary to use an outlier removal algorithm to remove the mismatches. In the prior art, the Random Sample Consensus (RANSAC) algorithm is an effective method to remove mismatches by continuously iterating the fitting model.

[0003] The RANSAC algorithm is an algorithm that searches for the optimal parameter model through random sampling and multiple iterations. Under multiple iterations, for all matching points in the space, points that meet the constraints are classified as inliers, and points that do not meet the constraints are classified as outliers. By fitting the model and parameter estimation for the inliers and removing the outliers that do not meet the conditions, a more robust estimation result is obtained, that is, solving an optimal homography matrix H of size 3×3 to eliminate false matches.

[0004] However, after the initial matching of images, mismatching may occur due to various transformations such as rotation, scaling, and non-rigid deformation, which reduces the matching accuracy. Therefore, we propose an image matching method that combines iterative weighted regression with affine transformation to improve RANSAC to solve the above problem. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies in the prior art, the present invention provides an image matching method that integrates iterative weighted regression and affine transformation to improve RANSAC, which solves the problems raised in the above background technology.

[0007] (II) Technical solution

[0008] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0009] An image matching method integrating iterative weighted regression and affine transformation to improve RANSAC includes the following steps:

[0010] S1: Input the reference image and the target image, and pre-process the two images into grayscale;

[0011] S2: Use the feature extraction algorithm to extract key feature points from the grayscale reference image and target image, and then combine the FLANN algorithm to match the feature points. FLANN finds the most similar feature point pairs through approximate nearest neighbor search and generates a preliminary matching point set as the basis for subsequent mismatch elimination and optimization;

[0012] S3: Introduces an affine RANSAC false match elimination method that integrates iterative weighted regression. It optimizes RANSAC using an affine transformation model and dynamically adjusts the affine parameter estimation accuracy in combination with the weighted least squares method. By assigning weights to each matching point and adjusting the weights in each iteration according to the size of the matching point residual, high-quality matching points are given larger weights, while low-quality matching points are gradually suppressed, thereby accurately eliminating false matches.

[0013] S4: A statistically based modified Z-score method is used to further filter out false matches to further improve matching accuracy.

[0014] Furthermore, the feature extraction algorithm in S2 adopts SIFT or SURF or ORB.

[0015] Furthermore, the specific steps of S3 are:

[0016] S3.1: Initialize the affine model, randomly select four pairs of matching pairs from the matching pairs, and use the selected matching pairs to estimate the parameters of the affine transformation, that is, the affine transformation matrix A and the translation vector t;

[0017] S3.2: Optimize the affine transformation model and use weighted least squares method for iterative weighted regression;

[0018] S3.3: Update the inlier set and remove false matches.

[0019] Furthermore, the specific steps of S3.1 are:

[0020] S3.1.1: Assume that the matching point set of the reference image is (x i ,y i ), the matching estimated point set mapped by affine transformation is (x i ',y i '), using the selected four sets of matching pairs to initialize the affine model, the mathematical representation of the affine transformation is:

[0021]

[0022] In the formula, the affine transformation matrix is Translation Vector

[0023] S3.1.2: Arrange the affine transformation results obtained from the selected four pairs of matching data into a matrix form to obtain:

[0024]

[0025] The above equation is expressed as M·θ=b, where M is the coefficient matrix, describing the mapping relationship between the reference point coordinates and the target point coordinates; θ is the affine transformation parameter, including the affine matrix and the displacement vector; b is the vector of the target point coordinates;

[0026] S3.1.3: According to the vector θ obtained by solving, take out the first four elements to form a matrix as the affine transformation parameter A, and take out the last two elements to form a vector form as the affine transformation parameter t.

[0027] Furthermore, the specific steps of S3.2 are:

[0028] S3.2.1: Calculate the residual d of each matching point by the following formula i , that is, the Euclidean distance between the true position of the matching point and the point estimated by the current affine transformation model:

[0029]

[0030] In the formula, (x r ,y r ) is the real matching point;

[0031] S3.2.2: Initialize the weight for each pair of matching points. Initially, the weight of each pair of matching points is set to 1, that is, w i =1(i=1,2,…,n); In each iteration, the weight is updated according to the residual value of the current matching point, and the affine transformation matrix A and the translation vector t are optimized by weighted least squares method to minimize the weighted residual sum of squares, thereby optimizing the fit of the affine transformation model. The goal of weighted least squares method is to minimize the objective function:

[0032]

[0033] In each iteration, the weight w of the matching point i It will be updated according to its residual, and points with smaller residuals will get larger weights, thereby optimizing the fitting effect of the model.

[0034] Furthermore, the specific steps of S3.3 are:

[0035] S3.3.1: Calculate the residual of each matching point and compare it with the set threshold. Matching points smaller than the threshold are considered as inliers and added to the inliers set P; points larger than the threshold are considered as outliers and removed.

[0036] S3.3.2: For each matching point in the inlier set, dynamically adjust the weight using the following formula:

[0037]

[0038] In the formula, w i is the weight of each pair of matching points, λ is a small positive number; for points with smaller residuals, the weight is larger; conversely, for points with larger residuals, the weight is smaller;

[0039] S3.3.3: Update the affine transformation model according to the current inlier set and the updated weights, and re-estimate the affine transformation parameters A and t by weighted least squares method. During the iteration process, if the number of inliers in the inlier set does not increase significantly, or when the maximum number of iterations is reached, the algorithm terminates and outputs the final affine transformation model.

[0040] Furthermore, the specific content of S4 is: the calculation of the modified Z score is as follows:

[0041]

[0042] Where x is the Euclidean distance between the matching point pairs; Median(X) is the median of the Euclidean distances of all matching point pairs; MAD(X) is the median absolute deviation, MAD(X) = Median(x i -Median(X)), used to measure the distribution range of Euclidean distance;

[0043] By calculating the corrected Z score of each matching point pair to measure its degree of deviation in the population, the absolute value of the corrected Z score is compared with the set threshold, as shown in the following formula:

[0044]

[0045] If the absolute value is greater than the threshold, the corresponding matching point is regarded as an outlier and removed; otherwise, the matching point is retained as a valid matching pair.

[0046] (III) Beneficial effects

[0047] Compared with the prior art, the present invention provides an image matching method that integrates iterative weighted regression and affine transformation to improve RANSAC, which has the following beneficial effects:

[0048] The present invention improves the RANSAC image matching method by adopting the fusion of iterative weighted regression and affine transformation. After the initial feature matching, the affine RANSAC mismatch elimination algorithm fused with iterative weighted regression is used to eliminate mismatches. The method estimates the model through the affine matrix, significantly improving the estimation accuracy of the transformation relationship. At the same time, the weighted least squares method is used to optimize the internal point set, dynamically adjust the weights of the matching points, and further accurately eliminate the mismatch points; the modified Z score is introduced as a post-processing step to perform fine filtering on the preliminary elimination results, which can effectively reduce the negative impact of the mismatch points on the matching accuracy, thereby making the final matching points more accurate and reliable. The method effectively solves the problem of mismatching and reducing the matching accuracy due to the influence of various transformations such as rotation, scaling and non-rigid deformation after the initial image matching, thereby significantly improving the matching correctness and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the overall method of the present invention;

[0050] Figure 2 It is a flow chart of the affine RANSAC mismatch elimination method introduced by the present invention and fused with iterative weighted regression. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example

[0053] like Figure 1-2 As shown, an image matching method that integrates iterative weighted regression and affine transformation to improve RANSAC is proposed in one embodiment of the present invention, and includes the following steps:

[0054] S1: Input the reference image and the target image, and perform grayscale preprocessing on the two images. The purpose of grayscale is to remove the color information in the image, reduce the computational complexity, and retain the main structural information of the image, providing a more concise input for subsequent feature extraction and matching;

[0055] S2: Use feature extraction algorithms such as SIFT, SURF or ORB to extract key feature points from the grayscale reference image and target image. Then, combine the FLANN algorithm to match feature points. FLANN finds the most similar feature point pairs through efficient approximate nearest neighbor search and generates a preliminary set of matching points as the basis for subsequent mismatch elimination and optimization;

[0056] S3: Introduce the affine RANSAC mismatch elimination method that integrates iterative weighted regression, optimize RANSAC using the affine transformation model, and dynamically adjust the affine parameter estimation accuracy in combination with the weighted least squares method. By assigning weights to each matching point, the weights are adjusted in each iteration according to the size of the matching point residual, so that high-quality matching points obtain larger weights, while low-quality matching points are gradually suppressed, thereby accurately eliminating mismatches; compared with the traditional RANSAC method, this method significantly improves the ability to eliminate mismatches, especially when the mismatch ratio is high, it can effectively improve the matching accuracy and robustness. This method continuously iteratively updates the weights of the matching points and the affine transformation model to improve the matching accuracy. The specific steps are as follows:

[0057] S3.1: Initialize the affine model, randomly select four pairs of matching pairs from the matching pairs, and use the selected matching pairs to estimate the parameters of the affine transformation, that is, the affine transformation matrix A and the translation vector t;

[0058] The specific steps are as follows: S3.1.1: Assume that the matching point set of the reference image is (x i ,y i ), the matching estimated point set mapped by affine transformation is (x i ',y i '), using the selected four sets of matching pairs to initialize the affine model, the mathematical representation of the affine transformation is:

[0059]

[0060] In the formula, the affine transformation matrix is Translation Vector

[0061] S3.1.2: Arrange the affine transformation results obtained from the selected four pairs of matching data into a matrix form to obtain:

[0062]

[0063] The above equation is expressed as M·θ=b, where M is the coefficient matrix, describing the mapping relationship between the reference point coordinates and the target point coordinates; θ is the affine transformation parameter, including the affine matrix and the displacement vector; b is the vector of the target point coordinates;

[0064] S3.1.3: According to the vector θ obtained by the solution, the first four elements are taken out to form a matrix as the affine transformation parameter A, and the last two elements are taken out to form a vector form as the affine transformation parameter t;

[0065] S3.2: Optimize the affine transformation model and use weighted least squares method for iterative weighted regression;

[0066] The specific steps are as follows: S3.2.1: Calculate the residual d of each matching point by the following formula: i , that is, the Euclidean distance between the true position of the matching point and the point estimated by the current affine transformation model:

[0067]

[0068] In the formula, (x r ,y r ) is the real matching point;

[0069] S3.2.2: Initialize the weight for each pair of matching points. Initially, the weight of each pair of matching points is set to 1, that is, w i =1(i=1,2,…,n); In each iteration, the weight is updated according to the residual value of the current matching point, and the affine transformation matrix A and the translation vector t are optimized by weighted least squares method to minimize the weighted residual sum of squares, thereby optimizing the fit of the affine transformation model. The goal of weighted least squares method is to minimize the objective function:

[0070]

[0071] In each iteration, the weight w of the matching point i It will be updated according to its residual, and points with smaller residuals will get larger weights, thereby optimizing the fitting effect of the model;

[0072] S3.3: Update the internal point set and remove false matches;

[0073] The specific steps are as follows: S3.3.1: Calculate the residual of each matching point and compare it with the set threshold. Matching points smaller than the threshold are considered as inliers and added to the inliers set P; points larger than the threshold are considered as outliers and removed;

[0074] S3.3.2: For each matching point in the inlier set, dynamically adjust the weight using the following formula:

[0075]

[0076] In the formula, w i is the weight of each pair of matching points, and λ is a small positive number. For points with smaller residuals, the weight is larger; conversely, for points with larger residuals, the weight is smaller. In this way, the algorithm can dynamically adjust the quality of matching points, making the suppression of mismatching more accurate.

[0077] S3.3.3: Update the affine transformation model according to the current inlier set and the updated weights, and re-estimate the affine transformation parameters A and t by weighted least squares method. During the iteration process, if the number of inliers in the inlier set does not increase significantly, or when the maximum number of iterations is reached, the algorithm terminates and outputs the final affine transformation model.

[0078] By iteratively optimizing the residuals and weights of the internal point set, recalculating the affine transformation parameters A and the translation vector t, accurate elimination of false matches can be achieved and the matching accuracy and robustness can be significantly improved.

[0079] S4: A statistically based modified Z-score method is used to further filter out false matches to further improve matching accuracy; the modified Z-score is calculated as follows:

[0080]

[0081] Where x is the Euclidean distance between the matching point pairs; Median(X) is the median of the Euclidean distances of all matching point pairs; MAD(X) is the median absolute deviation, MAD(X) = Median(x i -Median(X)), used to measure the distribution range of Euclidean distance;

[0082] By calculating the corrected Z score of each matching point pair to measure its degree of deviation in the population, the absolute value of the corrected Z score is compared with the set threshold, as shown in the following formula:

[0083]

[0084] If the absolute value is greater than the threshold, the corresponding matching point is regarded as an outlier and removed. Otherwise, the matching point is retained as a valid matching pair. By introducing the corrected Z score as a post-processing step and fine-tuning the preliminary elimination results, the negative impact of the mismatched points on the matching accuracy can be effectively reduced, making the final matching points more accurate and reliable.

[0085] The affine matrix RANSAC method optimized by iterative weight regression can better solve the problem of reduced matching accuracy caused by mismatching due to various transformations such as rotation, scaling and non-rigid deformation after the initial image matching, thereby enhancing the robustness and reliability of the matching method.

[0086] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image matching method integrating iterative weighted regression and affine transformation to improve RANSAC, characterized by: The steps include: S1: Input the reference image and the target image, and pre-process the two images into grayscale; S2: Use the feature extraction algorithm to extract key feature points from the grayscale reference image and target image, and then combine the FLANN algorithm to match the feature points. FLANN finds the most similar feature point pairs through approximate nearest neighbor search and generates a preliminary matching point set as the basis for subsequent mismatch elimination and optimization; S3: Introduces an affine RANSAC false match elimination method that integrates iterative weighted regression. It optimizes RANSAC using an affine transformation model and dynamically adjusts the affine parameter estimation accuracy in combination with the weighted least squares method. By assigning weights to each matching point and adjusting the weights in each iteration according to the size of the matching point residual, high-quality matching points are given larger weights, while low-quality matching points are gradually suppressed, thereby accurately eliminating false matches. S4: A statistically based modified Z-score method is used to further filter out false matches to further improve matching accuracy.

2. The image matching method according to claim 1, characterized in that: The feature extraction algorithm in S2 adopts SIFT or SURF or ORB.

3. The image matching method according to claim 1, characterized in that: The specific steps of S3 are: S3.1: Initialize the affine model, randomly select four pairs of matching pairs from the matching pairs, and use the selected matching pairs to estimate the parameters of the affine transformation, that is, the affine transformation matrix A and the translation vector t; S3.2: Optimize the affine transformation model and use weighted least squares method for iterative weighted regression; S3.3: Update the inlier set and remove false matches.

4. The image matching method according to claim 3, characterized in that: The specific steps of S3.1 are as follows: S3.1.1: Assume that the matching point set of the reference image is (x i ,y i ), the matching estimated point set mapped by affine transformation is (x i ',y i '), using the selected four sets of matching pairs to initialize the affine model, the mathematical representation of the affine transformation is: In the formula, the affine transformation matrix is Translation Vector S3.1.2: Arrange the affine transformation results obtained from the selected four pairs of matching data into a matrix form to obtain: The above equation is expressed as M·θ=b, where M is the coefficient matrix, describing the mapping relationship between the reference point coordinates and the target point coordinates; θ is the affine transformation parameter, including the affine matrix and the displacement vector; b is the vector of the target point coordinates; S3.1.3: According to the vector θ obtained by the solution, the first four elements are taken out to form a matrix as the affine transformation parameter A, and the last two elements are taken out to form a vector form as the affine transformation parameter t; The specific steps of S3.2 are: S3.2.1: Calculate the residual d of each matching point by the following formula i , that is, the Euclidean distance between the true position of the matching point and the point estimated by the current affine transformation model: In the formula, (x r ,y r ) is the real matching point; S3.2.2: Initialize the weight for each pair of matching points. Initially, the weight of each pair of matching points is set to 1, that is, w i =1(i=1,2,…,n); In each iteration, the weight is updated according to the residual value of the current matching point, and the affine transformation matrix A and the translation vector t are optimized by weighted least squares method to minimize the weighted residual sum of squares, thereby optimizing the fit of the affine transformation model. The goal of weighted least squares method is to minimize the objective function: In each iteration, the weight w of the matching point i It will be updated according to its residual, and points with smaller residuals will get larger weights, thereby optimizing the fitting effect of the model; The specific steps of S3.3 are as follows: S3.3.1: Calculate the residual of each matching point and compare it with the set threshold. Matching points smaller than the threshold are considered as inliers and added to the inliers set P; points larger than the threshold are considered as outliers and removed; S3.3.2: For each matching point in the inlier set, dynamically adjust the weight using the following formula: In the formula, w i is the weight of each pair of matching points, λ is a small positive number; for points with smaller residuals, the weight is larger; conversely, for points with larger residuals, the weight is smaller; S3.3.3: Update the affine transformation model according to the current inlier set and the updated weights, and re-estimate the affine transformation parameters A and t by weighted least squares method. During the iteration process, if the number of inliers in the inlier set does not increase significantly, or when the maximum number of iterations is reached, the algorithm terminates and outputs the final affine transformation model.

5. The image matching method according to claim 1, characterized in that: The specific content of S4 is: The calculation of the modified Z score is as follows: Where x is the Euclidean distance between the matching point pairs; Median(X) is the median of the Euclidean distances of all matching point pairs; MAD(X) is the median absolute deviation, MAD(X) = Median(x i -Median(X)), used to measure the distribution range of Euclidean distance; By calculating the corrected Z score of each matching point pair to measure its degree of deviation in the population, the absolute value of the corrected Z score is compared with the set threshold, as shown in the following formula: |Z mod |>T If the absolute value is greater than the threshold, the corresponding matching point is regarded as an outlier and removed; otherwise, the matching point is retained as a valid matching pair.

Images