Image feature point matching and registration point screening method

Abstract

The application discloses a kind of image feature point matching and registration point screening method, comprising the following steps: S10, using cosine similarity to match feature point;S20, using improved RANSAC algorithm to screen registration point.The average error and maximum error of the registration point are introduced, when using RANSAC for screening for the first time, the average error and maximum error between the screened registration points are calculated, when the maximum error is greater than 1.5 times of the average error, continue to screen the points screened this time using RANSAC, at this time, the average error of last time is used instead of the maximum allowable error given by user.Through this setting, the influence of the maximum allowable error given by user can be greatly excluded, and the stability of the algorithm is increased.

Description

Technical Field

[0001] This invention belongs to the field of image registration, and specifically relates to a method for image feature point matching and registration point selection. Background Technology

[0002] Infrared-to-visible image registration is a technique used to align and match infrared images with visible light images, commonly used in military, medical, meteorological, and other fields. The origins of this technique can be traced back to the mid-20th century.

[0003] In the prior art, the technical solution includes the regularized Gaussian field criterion for non-rigid registration of visible light and infrared face images. Its key idea is to represent the image through edge mapping and align the edge mapping with the non-rigid model through a robust criterion.

[0004] In addition, in infrared (IR) and visible light (VIS) image fusion, an enhanced affine transformation (EAT) method is proposed for non-rigid image registration, taking into account non-planar scenes and differences between IR and VIS cameras. The optimal EAT model is established through point set registration and global deformation to improve the accuracy and efficiency of image registration.

[0005] It also includes an infrared to visible light image registration method based on edge contour extreme points as feature points. This method has achieved good results in power equipment image registration, but there are still some problems in general scenarios.

[0006] In summary, multispectral image registration is a crucial task for aligning images captured in different bands or spectra. Researchers have made significant progress through various methods and techniques, overcoming challenges related to resolution, field of view, and geometric transformations to achieve pixel-level alignment. These techniques are of great importance for applications in multiple fields, including remote sensing, medical imaging, and security monitoring. Summary of the Invention

[0007] In view of this, the present invention proposes an image feature point matching and registration point selection method, comprising the following steps:

[0008] S10, cosine similarity is used for feature point matching;

[0009] S20 uses an improved RANSAC algorithm to select registration points.

[0010] Preferably, step S10 specifically includes the following steps:

[0011] S11, take the dot product of the infrared image feature descriptor vector and the visible light feature descriptor vector at each scale, calculate the cosine value between each vector, and form an array;

[0012] S12: Sort the array in ascending order, compare the ratio of the minimum cosine value to the second minimum cosine value. When the ratio is less than a certain threshold, the two feature points that yield the minimum cosine value are considered to be registered.

[0013] S13, When a feature point in an infrared image is registered with different feature points in visible light images at multiple scales, compare their cosine values ​​and find the two points with the smallest values ​​as the final registration points;

[0014] Through the above steps, several registration points are obtained, including erroneous registration points.

[0015] Preferably, step S20 specifically includes the following steps:

[0016] S21, Input coarse registration point set (P1, P2) and allowable error Error, where P1 is the infrared image point set and P2 is the visible light image point set;

[0017] S22, randomly select 3 pairs of registration points (a1,a2), (b1,b2), (c1,c2) ​​from the coarse registration point set (P1,P2);

[0018] S23, determine whether (a1,a2), (b1,b2), and (c1,c2) ​​are collinear or the distance is less than the preset value. If yes, return to S22 to re-extract 3 pairs of registration points; otherwise, proceed to the next step.

[0019] S24. Use the cv2.estimateAffinePartial2D function in OpenCV to find the affine transformation matrix for these three pairs of data.

[0020] S25, perform affine transformation on all points in P1 except for the three points (a1,b1,c1) to obtain the error between the transformed coordinates and the corresponding points in the reference point P2, which is the square of the distance. When the error of a certain point is within twice the allowable error, the confidence of the combination is increased by 1.

[0021] S26, repeat process S22-S25 N times. n is the total number of elements in P1. Several possible combinations are obtained, and the coordinates and confidence scores are recorded.

[0022] S27. Compare their confidence levels, find the three pairs of registration points with the highest confidence levels, and calculate their affine transformation matrix.

[0023] S28. Use this matrix to perform an affine transformation on other points in P1 to obtain the square of the distance between the transformed coordinates and the corresponding coordinates in P2, which is the error. If the error is less than 1.5 times the allowable error, record the original index coordinates of the point.

[0024] S29, calculate the mean error and maximum error of the registration points;

[0025] S30, determine whether meanError ≤ 1.5 * maxError;

[0026] If S31 is not correct, then replace the Error in S21 with meanError, filter the set of registration points and replace the set of registration points for this input point, and return to S22.

[0027] If S32 is true, then output the registration point index value.

[0028] Compared with the prior art, the image feature point matching and registration point selection method disclosed in this invention has at least the following beneficial effects:

[0029] Traditional RANSAC requires a manually specified maximum permissible error. A large value leads to numerous false matches, while a small value results in an insufficient number of registration points. To address this issue, the proposed improved RANSAC algorithm introduces an average error and a maximum error for registration points. After the initial RANSAC screening, the average and maximum errors between the selected registration points are calculated. If the maximum error exceeds 1.5 times the average error, RANSAC screening continues for the remaining points, using the previous average error instead of the user-specified maximum permissible error. This approach significantly reduces the influence of the user-specified maximum permissible error, increasing the algorithm's stability. Attached Figure Description

[0030] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0031] Figure 1 This is a schematic diagram of the registration points of infrared images and multi-scale visible light images in the image feature point matching and registration point screening method of this invention.

[0032] Figure 2 This is a schematic diagram of the coarse registration points of the infrared image and the visible light image in the image feature point matching and registration point screening method of the present invention.

[0033] Figure 3 This is a flowchart of step S20 of the image feature point matching and registration point filtering method according to an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of the registration points after screening by the RANSAC algorithm in the prior art;

[0035] Figure 5This is another schematic diagram of the registration points after screening by the RANSAC algorithm in the existing technology;

[0036] Figure 6 This is a filtered registration point map of an image feature point matching and registration point filtering method according to an embodiment of the present invention;

[0037] Figure 7 This is a filtered registration point map of an image feature point matching and registration point filtering method according to another embodiment of the present invention. Detailed Implementation

[0038] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] The image feature point matching and registration point selection method of the present invention includes the following steps:

[0040] S10, cosine similarity is used for feature point matching;

[0041] S20 uses an improved RANSAC algorithm to select registration points.

[0042] S10 specifically includes the following steps:

[0043] S11, take the dot product of the infrared image feature descriptor vector and the visible light feature descriptor vector at each scale, calculate the cosine value between each vector, and form an array;

[0044] S12: Sort the array in ascending order, compare the ratio of the minimum cosine value to the second minimum cosine value. When the ratio is less than a certain threshold, the two feature points that yield the minimum cosine value are considered to be registered.

[0045] S13, When a feature point in an infrared image is registered with different feature points in visible light images at multiple scales, compare their cosine values ​​and find the two points with the smallest values ​​as the final registration points;

[0046] Through the above steps, several registration points are obtained, including erroneous registration points.

[0047] See Figure 1 When point A in the infrared image is registered with point B in the visible light image at scale 1, and then with point C in scale 2, the cosine value between the descriptor vectors of point A and point B in scale 1 is compared with the cosine value between the descriptor vectors of point A and point C in scale 2. The set of points with the smaller cosine value is taken as the final registration points.

[0048] After filtering using the cosine similarity of the S10 points, many registration points can be obtained. Among these, a large number of incorrect registration points are identified. (See [link to relevant documentation]). Figure 2 The image is a rough matching map of the infrared and visible light images, so further screening is needed.

[0049] For coarse matching point selection, a commonly used method is the Random Adaptive Consensus Algorithm (RANSAC). Its core idea is to estimate the model by randomly selecting a subset of data points (i.e., random sampling), and then determine which data points are "inliers" (fitting the model) and which are "outliers" (outliers) by calculating the fitting error between the data points and the estimated model. This algorithm iterates multiple times to find the optimal model, ultimately outputting a robust model estimate.

[0050] In the feature point registration and selection process, the algorithm flow is based on the core idea of ​​RANSAC: From the coarse registration point set (P1, P2) (P1 is the infrared image point set, P2 is the light image point set), three pairs of non-collinear and sufficiently far-distance registration points (a1, a2), (b1, b2), and (c1, c2) are randomly selected. The affine transformation matrix of these three pairs of data is calculated (using the cv2.estimateAffinePartial2D function in OpenCV, assuming (a2, b2, c2) as the reference). Affine transformations are performed on all points in P1 except for (a1, b1, c1), obtaining the error (squared distance) between the transformed coordinates and the reference point (the corresponding point in P2). When the error of a point is within twice the maximum permissible error, the confidence level of the combination is incremented by 1. This process is then repeated N times. (n is the total number of elements in P1), several possible combinations are obtained, and their confidence levels are compared. The three pairs of registration points with the highest confidence levels are found. Based on the affine transformation matrix of these three pairs of points, radial transformation is performed on all points in P1, and the error between the transformed point and the reference point (P2) is calculated. When the error is less than the maximum allowable error, the point can be considered to be a point of accurate registration.

[0051] Traditional RANSAC requires manually specifying a maximum permissible error. If this value is too large, it will lead to many false matches in the screening process; if it is too small, it may result in an insufficient number of registration points. For this issue, see [link to relevant documentation]. Figure 3 S20 of the present invention specifically includes the following steps:

[0052] S21, Input coarse registration point set (P1, P2) and allowable error Error, where P1 is the infrared image point set and P2 is the visible light image point set;

[0053] S22, randomly select 3 pairs of registration points (a1,a2), (b1,b2), (c1,c2) ​​from the coarse registration point set (P1,P2);

[0054] S23, determine whether (a1,a2), (b1,b2), and (c1,c2) ​​are collinear or the distance is less than the preset value. If yes, return to S22 to re-extract 3 pairs of registration points; otherwise, proceed to the next step.

[0055] S24. Use the cv2.estimateAffinePartial2D function in OpenCV to find the affine transformation matrix for these three pairs of data.

[0056] S25, perform affine transformation on all points in P1 except for the three points (a1,b1,c1) to obtain the error between the transformed coordinates and the corresponding points in the reference point P2, which is the square of the distance. When the error of a certain point is within twice the allowable error, the confidence of the combination is increased by 1.

[0057] S26, repeat process S22-S25 N times. n is the total number of elements in P1. Several possible combinations are obtained, and the coordinates and confidence scores are recorded.

[0058] S27. Compare their confidence levels, find the three pairs of registration points with the highest confidence levels, and calculate their affine transformation matrix.

[0059] S28. Use this matrix to perform an affine transformation on other points in P1 to obtain the square of the distance between the transformed coordinates and the corresponding coordinates in P2, which is the error. If the error is less than 1.5 times the allowable error, record the original index coordinates of the point.

[0060] S29, calculate the mean error and maximum error of the registration points;

[0061] S30, determine whether meanError ≤ 1.5 * maxError;

[0062] If S31 is not correct, then replace the Error in S21 with meanError, filter the set of registration points and replace the set of registration points for this input point, and return to S22.

[0063] If S32 is true, then output the registration point index value.

[0064] This setting greatly reduces the influence of the user-defined maximum allowable error, increasing the stability of the algorithm.

[0065] Figure 4 and Figure 5 These are the registration points selected by the traditional RANSAC algorithm, with red indicating incorrect registration points. Figure 6 and Figure 7 The comparison of the four figures clearly shows that the improved RANSAC algorithm used in this invention achieves better screening results.

[0066] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation are within the scope of protection claimed by the present invention.

[0067] The present invention has been described in detail above, but its specific implementation is not limited thereto. Various modifications or alterations can be made by those skilled in the art without departing from the spirit and scope of the claims of this application.

Claims

1. A method for image feature point matching and registration point selection, characterized in that, Includes the following steps: S10, cosine similarity is used for feature point matching; S20 uses an improved RANSAC algorithm to select registration points; S20 specifically includes the following steps: S21, Input coarse registration point set (P1, P2) and allowable error Error, where P1 is the infrared image point set and P2 is the visible light image point set; S22, randomly select 3 pairs of registration point combinations (a1, a2), (b1, b2), (c1, c2) from the coarse registration point set (P1, P2); S23, determine whether (a1,a2), (b1, b2), and (c1, c2) are collinear or the distance is less than the preset value. If yes, return to S22 to re-extract 3 pairs of registration points; otherwise, proceed to the next step. S24. Use the cv2.estimateAffinePartial2D function in OpenCV to find the affine transformation matrix for these three pairs of data. S25, perform affine transformation on all points in P1 except for the three points (a1,b1,c1) to obtain the error between the transformed coordinates and the corresponding points in the reference point P2, which is the square of the distance. When the error of a certain point is within twice the allowable error, the confidence of the combination is increased by 1. S26, repeat process S22-S25 N times. n is the total number of elements in P1. Several possible combinations are obtained, and the coordinates and confidence levels are recorded. S27. Compare their confidence levels, find the three pairs of registration points with the highest confidence levels, and calculate their affine transformation matrix. S28. Use this matrix to perform an affine transformation on other points in P1 to obtain the square of the distance between the transformed coordinates and the corresponding coordinates in P2, which is the error. If the error is less than 1.5 times the allowable error, record the original index coordinates of the point. S29, calculate the mean error and maximum error of the registration points; S30, determine whether meanError ≤ 1.5 * maxError; If S31 is not correct, then replace the Error in S21 with meanError, filter the set of registration points and replace the set of registration points for this input point, and return to S22. If S32 is true, then output the registration point index value.

2. The image feature point matching and registration point selection method according to claim 1, characterized in that, S10 specifically includes the following steps: S11, take the dot product of the infrared image feature descriptor vector and the visible light feature descriptor vector at each scale, calculate the cosine value between each vector, and form an array; S12: Sort the array in ascending order, compare the ratio of the minimum cosine value to the second minimum cosine value. When the ratio is less than a certain threshold, the two feature points that yield the minimum cosine value are considered to be registered. S13, When a feature point in an infrared image is registered with different feature points in visible light images at multiple scales, compare their cosine values ​​and find the two points with the smallest values ​​as the final registration points; Through the above steps, several registration points are obtained, including erroneous registration points.

Images