Line feature matching method based on point-line geometric invariants

Abstract

The invention discloses a line feature matching method based on point-line geometric invariants, and belongs to the technical field of computer vision and image processing. The method comprises the following steps: firstly, acquiring point feature matching pairs by using a robust point feature matching algorithm; establishing a rectangular support region based on the line features, and distributing the rectangular support region into a full support region and a semi-support region according to the point features; and under a two-dimensional coordinate system, calculating a distance ratio from the point feature to the line feature to construct a distance invariant matched with the line feature, calculating a projection included angle of a direction vector of the line feature, constructing a direction invariant, and comprehensively establishing a line feature matching invariant. Based on this, a line feature matching score matrix is constructed, an initial line feature matching pair is obtained, and a mismatching pair is filtered according to spatial geometric attributes of the line feature matching pair; a large number of experiments show that the method has remarkable robustness while ensuring the accuracy of line feature matching.

Description

Technical Field

[0001] This application belongs to the technical field of computer vision and image processing, and particularly relates to a line feature matching method based on point-line geometric invariants. Background Art

[0002] Image feature matching is one of the core tasks in the field of computer vision and is widely applied in directions such as 3D reconstruction, object recognition, image stitching, and scene understanding. Traditional feature matching methods mainly rely on point features and can extract a large number of stable feature points and achieve high-precision matching in regions with rich textures. However, in low-texture or weak-texture scenes, the quality and quantity of point feature extraction decline, leading to matching failures or loss of precision. In addition, point features have limited robustness in the face of illumination changes, noise, and partial occlusion, and their application in scenarios is somewhat restricted. As another basic visual perception feature in machine vision, line features can effectively represent edge, contour, and structural information in a scene, especially providing supplementary information in regions lacking textures. Line feature matching can not only improve the stability and precision of matching but also demonstrate unique advantages in tasks such as 3D reconstruction, object tracking, and scene understanding.

[0003] Current line feature matching methods mainly rely on local feature descriptors or deep learning models. However, descriptor-based methods show low robustness when dealing with rotation and scale changes. At the same time, deep learning methods require high hardware computing costs, have strong dependence on training data, and have poor generalization ability.

[0004] Therefore, there is an urgent need for a line feature matching method to address the problem that the existing technology shows low robustness when dealing with rotation and scale changes. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a line feature matching method based on point-line geometric invariants to address the problem that the existing technology shows low robustness when dealing with rotation and scale changes.

[0006] The present invention adopts the following technical solutions:

[0007] The present invention provides a line feature matching method based on point-line geometric invariants, including:

[0008] Detect and extract a certain number of point features from image I1, match the point features of image I1 to image I2, and obtain the point features of image I2;

[0009] In the two-dimensional image plane, the distances between the line features in images I1 and I2 and each point feature within their corresponding support regions are calculated respectively through the two-dimensional coordinates of each point feature and the line feature equations; according to the said distances, the distance result sequences of the full support regions and semi-support regions of the line features in images I1 and I2 are obtained respectively; a point-line geometric invariant of the full support region is constructed based on the distance result sequence of the full support region of the line features in images I1 and I2, and according to the point-line geometric invariant of the full support region, the distances with incorrect matches in the distance result sequence are eliminated to obtain the line feature matching result sequence of the full support region; according to the line feature matching result sequence, the final point-line geometric invariant of the full support region of the line features is calculated; according to the ratios between the corresponding feature points in the distance result sequences of the semi-support regions of the line features in images I1 and I2, a ratio result sequence of the semi-support region is obtained, and the variance of the ratio result sequence is calculated;

[0010] According to the variance of the ratio result sequence of the semi-support region and the point-line geometric invariant of the full support region, the distance score between the line feature in image I1 and the line feature in image I2 is determined;

[0011] An image plane coordinate system is established, and the homogeneous coordinates of the two endpoints of the line feature in image I1 are used to calculate the direction vector of the line feature, and the direction vector of the line feature in image I1 is mapped to image I2; the included angle value between the mapped direction vector of the line feature and the corresponding direction vector of the line feature in image I2 is calculated, and according to the included angle value, the angle score between the line feature in image I1 and the line feature in image I2 is calculated;

[0012] According to the distance score and the angle score, a two-dimensional line feature matching score matrix is constructed; the element value in the two-dimensional line feature matching score matrix represents the score between the line feature in image I1 and the line feature in image I2;

[0013] Through the spatial geometric constraint information between the line features, the incorrectly matched line feature matching pairs in the two-dimensional line feature matching score matrix are eliminated to obtain the line feature matching result.

[0014] Preferably, the distances between the line features in images I1 and I2 and each point feature within their corresponding support regions are calculated through the two-dimensional coordinates of each point feature and the line feature equations in images I1 and I2, and the formula is:

[0015]

[0016] In the formula, is the distance from the point feature p i within the support region of the line feature on image I j to the line feature l i = 1, 2, 3,..., j = 1, 2, 3,..., It is the line feature equation (ax + by + c = 1) in a two-dimensional coordinate system, where the equation coefficients (a, b, c) are unit vectors, i = 1, 2, 3, …, n, j = 1, 2, 3, …, n.

[0017] Preferably,

[0018] The expression of the distance result sequence of the full support regions of the line features in images I1 and I2 is:

[0019]

[0020] In the formula, Q1, Q2, Q3, and Q4 are the distance result sequences calculated from the point features in the left and right support regions of the line features to the line features for the image pair to be matched. Q1 is the distance sequence from the point features in the left support region of the line feature l1 in image I1 to the line feature l1, Q2 is the result of the distance sequence from the point features in the right support region of the line feature l1 in image I1 to the line feature l1, Q3 is the distance sequence from the point features in the left support region of the line feature l2 in image I2 to the line feature l2, and Q4 is the distance sequence from the points in the right support region of the line feature l2 in image I2 to the line feature;

[0021] The expression of the distance result sequence of the full support regions of the line features in images I1 and I2 is:

[0022]

[0023] In the formula, and are the distance result sequences calculated from the line features on images I1 and I2 and the point features in their corresponding half support regions respectively.

[0024] Preferably, obtaining the line feature matching result sequence specifically includes:

[0025] Constructing the point-line geometric invariant of the full support region through the distance between the point features and the line features in the distance result sequence, and the expression is:

[0026]

[0027] In the formula, is the distance from the point feature p i in the left support region of the line feature l1 on image I i to l1, is the distance from the point feature p i in the right support region of the line feature l1 on image I i to l1, is the distance from the point feature p j in the left support region of the line feature l2 on image I j to l2, is for image Ij The point feature p within the right support region of the line feature l2 on the image j The distance to l2, where i = 1, 2, 3, …, n and j = 1, 2, 3, …, n;

[0028] Calculate the ratio of the distance between the point feature and the line feature in the distance result sequence Q1 to the distance between the point feature and the line feature in Q3, and the ratio of the distance between the point feature and the line feature in Q2 to the distance between the point feature and the line feature in Q4, to obtain the ratio result sequence of the full support region;

[0029] According to the point-line geometric invariant, eliminate the ratios in the ratio result sequence of the full support region that do not meet the point-line geometric invariant threshold, and retain the distances between the point feature and the line feature in the remaining ratio result sequence, to obtain the line feature matching result sequence {Q’1, Q’2, Q’3, Q’4}.

[0030] Preferably, according to the line feature matching result sequence, the calculation formula for the final point-line geometric invariant of the line feature full support region is:

[0031]

[0032] In the formula, and and respectively represent the sum of the distances from all point features in image I1 and image I2 to the corresponding line features.

[0033] Preferably, obtaining the ratio result sequence of the semi-support region specifically includes:

[0034] Based on the distance result sequence of the line feature semi-support region, calculate the point-line geometric invariant within the semi-support region, and the formula is:

[0035]

[0036] In the formula, and are the distances from the corresponding point feature p i to the line feature l in image I1 and image I2; λ1 to λ n represent the ratio of the distance of each point feature to its corresponding line feature;

[0037] According to the ratio between the corresponding feature points in the distance result sequence of the line feature semi-support region in image I1 and image I2, obtain the ratio result sequence of the semi-support region as q(λ1, λ2, …, λ n ).

[0038] Preferably, according to the variance of the line feature matching result sequence and the point-line geometric invariant of the full support region, determine the distance score between the line feature in image I1 and the line feature in image I2, specifically including:

[0039] According to the ratio result sequence q(λ1, λ2, …, λ n ), take the median of the sequence as the lower limit of the interval and three times the median as the upper limit of the interval, and remove the outliers in the distance sequence from point features to line features;

[0040] Calculate the variance of the ratio result sequence of the semi - support domain, and obtain the distance score between the line feature in image I1 and the line feature in image I2 according to the variance of the ratio result sequence of the semi - support domain and the final point - line geometric invariant of the full - support domain of the line feature. The formula is:

[0041]

[0042] In the formula, socore distance is the distance score between the line feature in image I1 and the line feature in image I2, and var(q) is the variance of the ratio result sequence of the semi - support domain.

[0043] Preferably, calculate the angle score between the line feature in image I1 and the line feature in image I2 according to the included - angle value, specifically including:

[0044] Based on the angle value α between the direction vector v1 of the line feature and the direction vector v2 of the mapped line feature, obtain the angle score between the line feature in image I1 and the line feature in image I2. The formula is:

[0045]

[0046] When the included - angle between the direction vector of the line feature and the direction vector of the line feature is in the interval of 175 degrees to 180 degrees, or in the interval of 0 degrees to 5 degrees, the line - feature matching score is socore angle = 1. When the included - angle value is between 5 degrees and 175 degrees, the line - feature matching score is

[0047] Preferably, construct a two - dimensional line - feature matching score matrix, specifically including:

[0048] Based on the distance score and the angle score between the line feature in image I1 and the line feature in image I2, construct a score matrix S; where the score score ij in the two - dimensional line - feature matching score matrix is calculated by the formula:

[0049] socore ij = w1 * socore angle + w2 * socore distance ;

[0050] w1 + w2 = 1;

[0051] Wherein, w1 is the weight of the angle constraint invariant, w2 is the weight of the distance constraint invariant, and socore angle is the angle score, and socore distance is the distance invariant score.

[0052] Preferably, the spatial geometric constraint information between line features includes: the constraints (d1, d2) of the distances from the endpoints of the line feature matching pair to the corresponding line feature and the constraint d of the distance between the midpoints of the line feature matching pair m , and the formula is:

[0053]

[0054] Wherein, (a, b, c) is the direction vector of the line feature, and (u1, v1, 1) and (u2, v2, 1) are the homogeneous coordinates of the two endpoints of the line feature. d1 and d2 are the distances from the endpoints of the line feature to the line, is the midpoint coordinate of the corresponding line feature, is the midpoint coordinate on the straight line l1, is the midpoint coordinate on the straight line l2.

[0055] The above at least one technical solution adopted by the present invention can achieve the following beneficial effects:

[0056] The present invention divides the support domain of the line feature into two regions, left and right, and calculates the ratio of the distances from the point features in the corresponding support regions to the line features where they are located. This ratio effectively constructs a geometric invariant that is not affected by image scaling, rotation, translation, or other transformations, thereby improving the matching accuracy in the image processing process when facing different geometric changes. By setting this ratio as a constant and based on the geometric relationship between the point and the line, it is ensured that during the matching process, even from different perspectives, the geometric constraints between the point feature and the line feature remain stable, thus avoiding matching errors caused by image transformation; the present invention also combines the direction information of the line feature, that is, the direction vector of the line feature. The direction vector of each line feature is calculated from its endpoint coordinates in the two-dimensional image, which can effectively describe the spatial direction of the line feature on the two-dimensional image. Through the constraint of the line feature direction vector, the accuracy and consistency of the line feature matching are ensured; after constructing the point-line geometric invariant, a matching matrix is calculated, and the preliminary line feature matching pairs are derived from this matching matrix in sequence, and each matching pair is screened according to the geometric constraints, thereby eliminating the incorrect matches that do not conform to the geometric constraints, effectively solving the problem that the prior art shows low robustness when dealing with rotation and scale changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0058] Figure 1 Flow schematic diagram of a line feature matching method based on point - line geometric invariants provided by the present invention;

[0059] Figure 2 Explanation schematic diagram of the full support domain, semi - support domain, direction vector, and scoring matrix of line features for line feature matching based on point - line geometric invariants provided by the present invention;

[0060] Figure 3 Example diagram of line feature matching of a line feature matching method based on point - line geometric invariants provided by the present invention;

[0061] Figure 4 Example diagram of verification experiment results under different parameters of a line feature matching method based on point - line geometric invariants provided by the present invention;

[0062] Figure 5 Visualization result of line feature matching of a line feature matching method based on point - line geometric invariants provided by the present invention. Detailed implementation manners

[0063] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0064] The technical solutions provided by each embodiment of the present application will be described in detail below with reference to the drawings.

[0065] Figure 1 Flow schematic diagram of a line feature matching method based on point - line geometric invariants in the present invention, specifically including the following steps:

[0066] S101: Detect and extract a certain number of point features from image I1, match the point features of image I1 to image I2, and obtain the point features of image I2.

[0067] The point feature matching algorithm can find robust point feature matching pairs in scenes under different perspectives and conditions, forming a preliminary point feature matching result.

[0068] Specifically, the point feature matching algorithm includes, but is not limited to, the method based on GMS point feature matching. The GMS (Grid-based Motion Statistics) algorithm divides the image into grids and combines motion statistical information for feature matching. Through this algorithm, point feature matching under perspective changes, noise, or other interference conditions can be effectively processed, effectively meeting the accuracy and robustness requirements of point feature matching.

[0069] S102: In the two-dimensional plane of the image, calculate the distances between the line features in images I1 and I2 and each point feature within its corresponding support region respectively through the two-dimensional coordinates of each point feature and the line feature equation; according to the distances, obtain the distance result sequences of the full support region and the half support region of the line features in images I1 and I2 respectively; construct the point-line geometric invariant of the full support region based on the distance result sequence of the full support region of the line features in images I1 and I2, and according to the point-line geometric invariant of the full support region, eliminate the distances with incorrect matches in the distance result sequence to obtain the line feature matching result sequence of the full support region; calculate the final point-line geometric invariant of the full support region of the line features according to the line feature matching result sequence; obtain the ratio result sequence of the half support region according to the ratios between the corresponding feature points in the distance result sequences of the half support regions of the line features in images I1 and I2, and calculate the variance of the ratio result sequence.

[0070] Optionally, the point features are all located on the image I1 to be matched and distributed on both sides of the support region of the line feature l1, and the corresponding point features in image I2 are and the line features l2 are both distributed on image I2, then the point features in the adjacent regions of the line feature pair l1 and l2 satisfy the transformation relation:

[0071]

[0072] where s is the scaling factor and H is the homography matrix, is the point feature on the image I1 to be matched, is the point feature on image I2, and i = 1, 2, 3, ….

[0073] Optionally, calculate the distances between the line features in images I1 and I2 and the matching point pairs of the point features within their support regions through the two-dimensional coordinates of each point feature matching pair and the line feature equation. The formula is:

[0074]

[0075] where is the distance from the point feature p i within the support region of the line feature on image I j to the line feature l, i = 1, 2, 3, …, j = 1, 2, 3, … is the transposed form of the homogeneous coordinates of the point feature, is the line feature equation (ax + by + c = 1) in the two-dimensional coordinate system, where the equation coefficients (a, b, c) are unit vectors.

[0076] Optionally, the expressions for the distance result sequences of the full support regions of the line features in images I1 and I2 are respectively:

[0077]

[0078] In the formula, Q1, Q2, Q3, and Q4 are the distance result sequences calculated from the point features in the left and right support regions of the line features in the image pair to be matched. Q1 is the distance sequence from the point features in the left support region of the line feature l1 in image I1 to the line feature l1, Q2 is the distance sequence result from the point features in the right support region of the line feature l1 in image I1 to the line feature l1, Q3 is the distance sequence from the point features in the left support region of the line feature l2 in image I2 to the line feature l2, and Q4 is the distance sequence from the points in the right support region of the line feature l2 in image I2 to the line feature;

[0079] Optionally, the expressions for the distance result sequences of the half support regions of the line features in images I1 and I2 are respectively:

[0080]

[0081] In the formula, and are respectively the distance result sequences calculated from the line feature l1 and the point features in its corresponding half support region in images I1 and I2.

[0082] Specifically, the support regions where the point features are located include: the full support region (normal supporting region) and the half support region (sem-supporting region); among them, the support region is the rectangular range formed by the line features ( Figure 2 (a)), and the position relationship of the line features in the image matching pair can be indirectly reflected by the distance ratio of the point features in the support region to the line features.

[0083] Optionally, obtaining the line feature matching result sequence specifically includes:

[0084] Constructing the point-line geometric invariant of the full support region, and the expression is:

[0085]

[0086] In the formula, is the distance from the point feature p i in the left support region of the line feature l1 on image I i to l1. For I i The point feature p within the right support region of the line feature l1 on the image i The distance to l1 For I j The point feature p within the left support region of the line feature l2 on the image j The distance to l2 For I j The point feature p within the right support region of the line feature l2 on the image j The distance to l2, where i = 1, 2, 3, …, n and j = 1, 2, 3, …, n.

[0087] Calculate the ratio of the distance between the point feature matching pairs and the line feature in the distance result sequence Q1 to the distance between the point feature matching pairs and the line feature in Q3, and the ratio of the distance between the point feature matching pairs and the line feature in Q2 to the distance between the point feature matching pairs and the line feature in Q4 to obtain the ratio result sequence of the full support region; according to the point-line geometric invariant, eliminate the distance ratios of the corresponding point features and line features that do not meet the point-line geometric invariant threshold in the ratio result sequence, and retain the distances between the point features and line features that meet the point-line geometric invariant threshold to obtain the line feature matching result sequence {Q’1, Q’2, Q’3, Q’4}

[0088] Specifically, the point-line geometric invariant threshold is generally set to 0.95, and the line feature matching result sequence {Q’1, Q’2, Q’3, Q’4} is obtained based on the ratio threshold

[0089] Optionally, according to the line feature matching result sequence, the calculation formula for the final point-line geometric invariant of the full support region of the line feature is

[0090]

[0091] In the formula And And Respectively represent the sum of the distances from all point features in image I1 and image I2 to the corresponding line features

[0092] Optionally, calculate the point-line geometric invariant within the semi-support region, obtain the ratio result sequence of the semi-support region, and calculate the variance of the ratio result sequence, specifically including

[0093] Based on the distance result sequence of the semi-support region of the line feature, calculate the point-line geometric invariant within the semi-support region, and the formula is

[0094]

[0095] In the formula And Are the corresponding point features p in image I1 and image I2i Distance to line feature l; λ1 to λ n Represents the ratio of the distance from each point feature to its corresponding line feature;

[0096] According to the point-line geometric invariants in the semi-support region, the ratio result sequence of the semi-support region is obtained as q(λ1,λ2,…,λ n ), the variance of the ratio result sequence of the semi-support domain is var(q).

[0097] Specifically, when it is in the semi-support region of the line feature, see Figure 2 (b) The point feature correspondence within half of the neighborhood can be obtained, so only two pairs of point features to corresponding line features can be obtained. According to the point-line feature matching invariant, the ratio of the distance from each corresponding point to the line feature should be a constant value.

[0098] Specifically, by calculating the ratio of the distances from points to lines within the support region, we can effectively construct geometric invariants that are not affected by transformations such as image scaling, rotation, or translation, thereby improving the accuracy of matching when facing different geometric changes during image processing.

[0099] Specifically, for any straight line segment in the semi-support domain to be matched, the calculation formula of the point-line geometric invariant of the semi-support domain is used to obtain a set of ratio result sequences of the distances between point features and line features. Since the point feature matching may be incorrect, the data sequence may contain obvious mismatching points, resulting in the appearance of outliers. To this end, the median of the data sequence is first calculated, and the abnormal proportion values ​​are eliminated with three times the median as the interval range. Subsequently, the variance of the data sequence is calculated, and the corresponding score is obtained accordingly. Finally, the obtained score is filled into the score matrix S in the subsequent matching matrix according to the formula distance scoring formula, see Figure 2 (d).

[0100] S103: Determine the distance score between the line feature in the image I1 and the line feature in the image I2 according to the variance of the ratio result sequence in the semi-support domain and the point-line geometric invariant in the full support domain.

[0101] Optionally, the angle score between the line feature in the image I1 and the line feature in the image I2 is calculated according to the angle value, specifically including: based on the angle value α between the line feature direction vector v1 and the direction vector v2 of the mapped line feature, the angle score between the line feature in the image I1 and the line feature in the image I2 is obtained, and the formula is:

[0102]

[0103] When the angle between the line feature direction vector and the line feature direction vector is between 175 and 180 degrees, or between 0 and 5 degrees, the line feature matching score is socoreangle = 1, when the included angle value is between 5 and 175 degrees, the line feature matching score is

[0104] S104: Establish an image plane coordinate system, calculate the line feature direction vector using the homogeneous coordinates of the two endpoints of the line feature in image I1, and map the line feature direction vector of image I1 to image I2; calculate the included angle value between the mapped line feature direction vector and the corresponding line feature direction vector in image I2, and calculate the angle score between the line feature in image I1 and the line feature in image I2 according to the included angle value.

[0105] Optionally, the angle value between the line feature direction vector v1 of I1 and the direction vector v2 of the line feature of I2 after homography mapping, and score the line feature according to the angle, specifically including:

[0106] Calculate the angle value α between the line feature direction vector v1 and the direction vector v2 of the line feature, see Figure 2 (c), which is a schematic diagram of the included angle between the line feature direction vector v1 and the direction vector v2 of the line feature, and the formula is:

[0107]

[0108] Based on the angle value between the line feature direction vector v1 and the direction vector v2 of the line feature, the scoring formula for line feature matching is:

[0109]

[0110] In the formula, is the score of the angle invariant result of the line feature matching pair, indicating the angle matching score between the i-th line feature in I1 and the j-th line feature in I2.

[0111] Specifically, the calculation of the direction vector is based on the endpoint coordinates of the line feature. Each line feature has two endpoints, and the coordinates of these endpoints can be used to calculate the direction vector of the line feature. This direction vector is determined by the coordinate difference between the two endpoints and can indicate the direction of the line feature in the image two-dimensional coordinate system. By calculating the direction vector of the line feature, the orientation information of the line feature can be obtained, which provides strong support for the construction of the subsequent matching score matrix. Establish an image plane coordinate system, and calculate and map the line feature direction vector v1 using the line endpoint coordinates. If the homography mapping is accurate and the detection and extraction of the line feature in the original image are accurate, then theoretically, the matched line feature should "coincide" at the corresponding position in the other image after the homography mapping (here, coincidence means that the projection directions of the same line in the space represented by the two images under the ideal projection relationship are the same, the distance between the endpoints of the line feature is 0, and the included angle is 0).

[0112] S105: Construct a two-dimensional line feature matching score matrix according to the distance score and the angle score; the element value in the two-dimensional line feature matching score matrix represents the score between the line feature in image I1 and the line feature in image I2.

[0113] Optionally, constructing the two-dimensional line feature matching score matrix specifically includes:

[0114] Based on the distance score and the angle score between the line feature in image I1 and the line feature in image I2, construct a score matrix S; where the score score in the two-dimensional line feature matching score matrix ij The calculation formula of is:

[0115] socore ij = w1 * socore angle + w2 * socore distance ;

[0116] w1 + w2 = 1;

[0117] In the formula, w1 is the weight of the angle constraint invariant, w2 is the weight of the distance constraint invariant, socore angle is the angle score, and socore distance is the distance invariant score.

[0118] S106: Through the spatial geometric constraint information between line features, eliminate the wrongly matched line feature pairs in the line feature matching pairs in the two-dimensional line feature matching score matrix to obtain the line feature matching result.

[0119] Optionally, the spatial geometric constraint information between line features includes: the constraint (d1, d2) of the distance from the endpoints of the line feature pair to the corresponding line feature and the constraint d of the distance between the midpoints of the line feature pair m , and the formula is:

[0120]

[0121] In the formula, (a, b, c) is the direction vector of the line feature, and (u1, v1, 1) and (u2, v2, 1) are the homogeneous coordinates of the two endpoints of the line feature. d1 and d2 are the distances from the endpoints of the line feature to the line, is the midpoint coordinate of the corresponding line feature, is the midpoint coordinate on the straight line l1, is the midpoint coordinate on the straight line l2.

[0122] Through the above method steps, in this embodiment, image pairs in different situations are selected, see Figure 3 (including situations such as low texture, scale change, view change, rotation, occlusion, etc.), which can be used to evaluate the algorithm provided by the present invention from different angles.

[0123] The parameter settings of the method in this embodiment: To determine the values of w1 and w2 in the scoring formula of the two-dimensional line feature matching scoring matrix, verification experiments were carried out using the data sequences (A) to (F) in the dataset. The value of w1 was changed back and forth within the range of 0.1 to 0.9. Refer to Figure 4 (a) and Figure 4 (b), and the qualitative and quantitative results were obtained respectively; Figure 4 (a) shows the number of line feature matching pairs corresponding to each sequence under different weights, while Figure 4 (b) shows the accuracy of line feature matching corresponding to each sequence under different weights; According to Figure 4 (a) and Figure 4 (b), combined with the comparison of the results of multiple data sequences, when w1 and w2 are 0.5, the number and accuracy of feature matching are basically the highest. Therefore, the weight values of w1 = 0.5 and w2 = 0.5 are directly selected here; Refer to Figure 5 , the visual comparison results of line feature matching under some scene images are given, including LineJunction Line and Hybrid Matching and the method provided by the present invention, which respectively show the matching results in indoor low-texture and weak-texture scenes.

[0124] In summary, the present invention provides a novel line feature matching strategy based on point-line invariants, aiming to improve the accuracy and robustness of line feature matching, especially having significant advantages in the performance of line feature matching in complex scenes. By combining the geometric relationship between point features and line features and using point-line invariants, this algorithm can effectively ensure the accuracy and robustness of the matching results in the presence of interferences such as noise, illumination changes, scale scaling, and perspective changes.

[0125] The core steps and principles of this algorithm are as follows:

[0126] First, in order to obtain accurate and reliable point feature matching pairs, the method of the present invention adopts a robust point feature matching algorithm, specifically, the Grid-based Motion Statistics (GMS) algorithm can be used. The GMS algorithm is an advanced point matching method that can achieve matching by dividing the image into grids and statistically analyzing the motion relationship between point features in each grid. It has strong robustness, especially in the case of perspective changes, scale changes, and partial occlusions in the image. This algorithm can effectively obtain a set of point feature matching pairs, and these matching point pairs provide important information for line feature matching in subsequent steps.

[0127] Next, the present invention divides the line feature support region into left and right regions by establishing and partitioning it. During this partitioning process, assuming there is a pair of adjacent matching images, for each pair of images, the ratio of the distance from the point feature in the corresponding support region to the line feature is calculated. Theoretically, this ratio is a constant. The core of this step is based on the geometric relationship between points and lines, ensuring that during the matching process, even under different perspectives, the geometric constraints between point features and line features remain stable, thereby avoiding matching errors caused by image transformation.

[0128] In addition, the method of the present invention also combines the direction information of line features, especially the direction vector of line features. The direction vector of each line is calculated from the coordinates of its endpoints in a two-dimensional image and can effectively describe the spatial direction of the line. Through this direction vector, the accuracy and consistency of matching are further improved.

[0129] After constructing the point-line invariant, the algorithm calculates the initial matching matrix, which contains all candidate line feature matching pairs. Subsequently, by screening the geometric attributes of the matching pairs, the preliminary line feature matching results are gradually extracted. The verification of geometric attributes includes checking the distance, angle, and endpoint geometric relationship between line features, thereby eliminating incorrect matches that do not meet the geometric constraints.

[0130] Finally, to verify the effectiveness of the algorithm, the present invention conducts a comparative analysis with the existing line feature matching algorithm (Line Junction Line) and hybrid matching algorithm (Hybrid Matching) through a large amount of experimental data. The experimental results show that the proposed point-line invariant matching strategy exhibits extremely high matching accuracy on multiple data sets. In addition, when dealing with complex images, the algorithm can effectively cope with various interference factors, demonstrating strong robustness and adaptability. Whether under illumination changes, perspective transformations, or partial occlusions, the method can maintain a high matching accuracy.

[0131] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as the scope recorded in the present invention.

Claims

1. A line feature matching method based on point-line geometric invariants, characterized in that, Including: Detect and extract a certain number of point features from image I1, match the point features of image I1 to image I2, and obtain the point features of image I2; In the two-dimensional image plane, calculate the distances between the line features in image I1 and image I2 and each point feature within its corresponding support region respectively through the two-dimensional coordinates of each point feature and the line feature equation; according to the said distances, obtain the distance result sequences of the full support regions and semi-support regions of the line features in image I1 and image I2 respectively; construct the point-line geometric invariant of the full support region according to the distance result sequence of the full support region of the line features in image I1 and image I2, and according to the point-line geometric invariant of the full support region, eliminate the distances with incorrect matches in the distance result sequence to obtain the line feature matching result sequence of the full support region; calculate the final point-line geometric invariant of the full support region of the line features according to the line feature matching result sequence; obtain the ratio result sequence of the semi-support region according to the ratios between the corresponding feature points in the distance result sequence of the semi-support region of the line features in image I1 and image I2, and calculate the variance of the ratio result sequence; Determine the distance score between the line features in image I1 and the line features in image I2 according to the variance of the ratio result sequence of the semi-support region and the point-line geometric invariant of the full support region; Establish an image plane coordinate system, calculate the line feature direction vector using the homogeneous coordinates of the two endpoints of the line feature in image I1, and map the line feature direction vector of image I1 to image I2; calculate the included angle value between the mapped line feature direction vector and the corresponding line feature direction vector in image I2, and calculate the angle score between the line features in image I1 and the line features in image I2 according to the included angle value; Construct a two-dimensional line feature matching score matrix according to the distance score and the angle score; the element value in the two-dimensional line feature matching score matrix represents the score between the line features in image I1 and the line features in image I2; Eliminate the incorrectly matched line feature matching pairs in the two-dimensional line feature matching score matrix through the spatial geometric constraint information between the line features to obtain the line feature matching result.

2. The line feature matching method based on point-line geometric invariants according to claim 1, characterized in that The calculation of the distances between the line features in image I1 and image I2 and each point feature within its corresponding support region through the two-dimensional coordinates of each point feature in image I1 and image I2 and the line feature equation is as follows: In the formula, is the point feature p within the line feature support region on the image I i to the line feature l j distance, i = 1, 2, 3, …, j = 1, 2, 3, …, is the transposed form of the homogeneous coordinates of the point feature, is the line feature equation (ax + by + c = 1) in the two-dimensional coordinate system, where the equation coefficients (a, b, c) are unit vectors, i = 1, 2, 3, …, n, k = 1, 2, 3, …, n.

3. A line feature matching method based on point-line geometric invariant according to claim 1, characterized in that The expression of the distance result sequence of the full support region of the line features in image I1 and image I2 is: In the formula, Q1, Q2, Q3, and Q4 are respectively the distance result sequences of the point features within the left and right support regions of the line features in the image pair to be matched to the line features. Q1 is the distance sequence of the point features within the left support region of line feature l1 in image I1 to line feature l1, Q2 is the distance sequence result of the point features within the right support region of line feature l1 in image I1 to line feature l1, Q3 is the distance sequence of the point features within the left support region of line feature l2 in image I2 to line feature l2, and Q4 is the distance sequence of the points within the right support region of line feature l2 in image I2 to the line feature; The expression of the distance result sequence of the line feature full support regions in the images I1 and I2 is as follows: In the formula, and are respectively the distance result sequences calculated from the line features on the I1 image and the I2 image and the point features within their corresponding semi-support regions.

4. A line feature matching method based on point-line geometric invariants as described in claim 1, characterized in that The obtaining of the line feature matching result sequence specifically includes: Construct the point-line geometric invariant of the full support region through the distance between the point feature and the line feature in the distance result sequence, and the expression is: Wherein, is I i the distance from the point feature p within the left support region of the line feature l1 on the I i image to l1, is I i the distance from the point feature p within the right support region of the line feature l1 on the I i image to l1, is I j the distance from the point feature p within the left support region of the line feature l2 on the I j image to l2, is I j the distance from the point feature p within the right support region of the line feature l2 on the I j image to l2, i = 1, 2, 3, …, n, j = 1, 2, 3, …, n; Calculate the ratio of the distance between the point feature and the line feature in the distance result sequence Q1 to the distance between the point feature and the line feature in Q3, and the ratio of the distance between the point feature and the line feature in Q2 to the distance between the point feature and the line feature in Q4, to obtain the ratio result sequence of the full support region; According to the point-line geometric invariant, eliminate the ratios in the ratio result sequence of the full support region that do not meet the point-line geometric invariant threshold, and retain the distance between the point feature and the line feature in the remaining ratio result sequence, to obtain the line feature matching result sequence {Q’1, Q’2, Q’3, Q’4}.

5. A line feature matching method based on point-line geometric invariants as described in claim 1, characterized in that, The calculation formula for the final point-line geometric invariant of the line feature full support region according to the line feature matching result sequence is: In the formula, and and respectively represent the sum of the distances from all point features in image I1 and image I2 to the corresponding line features.

6. The line feature matching method based on point-line geometric invariants according to claim 1, wherein The obtaining of the ratio result sequence of the semi-support region specifically includes: Based on the distance result sequence of the line feature semi-support region, calculate the point-line geometric invariant within the semi-support region, and the formula is: In the formula, and is the distance from the corresponding point feature p in the images I1 and I2 i to the line feature l; λ1 to λ n represents the ratio of the distance from each point feature to its corresponding line feature; According to the ratio between corresponding feature points in the distance result sequence of the line feature semi-support regions in image I1 and image I2, the ratio result sequence of the semi-support regions is obtained as q(λ1, λ2, …, λ n ).

7. A line feature matching method based on point-line geometric invariants according to claim 1, characterized in that, According to the variance of the line feature matching result sequence and the point-line geometric invariant of the full support region, determine the distance score between the line feature in image I1 and the line feature in image I2, specifically including: According to the ratio result sequence q(λ1, λ2, …, λ n ), take the median of the sequence as the lower limit of the interval and three times the median as the upper limit of the interval, and remove the outliers in the distance sequence from point features to line features; Calculate the variance of the ratio result sequence of the semi-support region, and based on the variance of the ratio result sequence of the semi-support region and the final point-line geometric invariant of the line feature full support region, obtain the distance score between the line feature in image I1 and the line feature in image I2, and the formula is: where, socore distance is the distance score between the line features in image I1 and the line features in image I2, and var(q) is the variance of the ratio result sequence of the semi-support region.

8. A line feature matching method based on point-line geometric invariants according to claim 1, characterized in that According to the included angle value, calculate the angle score between the line feature in image I1 and the line feature in image I2, specifically including: Based on the angle value α between the direction vector v1 of the line feature and the direction vector v2 of the mapped line feature, obtain the angle score between the line feature in image I1 and the line feature in image I2, and the formula is: When the included angle between the line feature direction vector and the line feature direction vector is in the range of 175 degrees to 180 degrees, or in the range of 0 degrees to 5 degrees, the line feature matching score is socore angle = 1. When the included angle value is between 5 and 175 degrees, the line feature matching score is 9. A line feature matching method based on point-line geometric invariants according to claim 1, characterized in that The construction of the two-dimensional line feature matching score matrix specifically includes: Construct a scoring matrix S based on the distance score and angle score between the line features in image I1 and the line features in image I2; wherein, the score score in the two-dimensional line feature matching scoring matrix ij is calculated by the formula: socore ij = w1 * socore angle + w2 * socore distance ; w1 + w2 = 1; Wherein, w1 is the weight of the angle constraint invariant, w2 is the weight of the distance constraint invariant, socore angle is the angle score, socore distance is the distance invariant score.

10. A line feature matching method based on point-line geometric invariants according to claim 1, characterized in that, The spatial geometric constraint information between the line features includes: the constraints (d1, d2) of the distances from the endpoints of the line feature matching pairs to the corresponding line features and the constraint d of the distance between the midpoints of the line feature matching pairs m , and the formula is: Where (a, b, c) is the direction vector of the line feature, and (u1, v1, 1) and (u2, v2, 1) are the homogeneous coordinates of the two endpoints of the line feature. d1 and d2 are the distances from the endpoints of the line feature to the line, is the coordinate of the midpoint of the corresponding line feature, is the coordinate of the midpoint on the straight line l1, is the coordinate of the midpoint on the straight line l2.

Images