A Line Feature Matching Method Based on Dynamic Matching and Median Absolute Deviation

Abstract

A line feature matching method based on dynamic matching and absolute median difference includes the following steps: 1) receiving RGB frames captured by an RGB-D camera as visual input images; 2) extracting line features using the EDLines line detector; 3) parameterizing line features using endpoints; 4) describing the parameterized line features based on discrete key points; 5) dynamically matching line features extracted from previous and next frames using a dynamic programming algorithm; and 6) estimating the standard deviation of displacement changes for all line feature matches using the absolute median difference method, and using this estimate to eliminate erroneous or low-quality line matches. This method can extract a sufficient number of correct line feature matches in scenes with occlusion, rotation, and appearance changes.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and navigation positioning, and in particular to a line feature matching method based on dynamic matching and absolute median difference. Background Art

[0002] Image matching algorithms extract relevant information from images acquired from different perspectives or sensors, such as similarities in texture, grayscale, and structure, and then match two or more images using a matching algorithm. High-precision image matching and low computational cost are well-suited for use in visual SLAM navigation systems.

[0003] Image matching algorithms can be primarily categorized as grayscale-based and feature-based. Feature-based matching algorithms are more robust to image object rotation, illumination variations, and occlusion. Grayscale-based image matching algorithms typically achieve higher matching accuracy by utilizing extreme values of correlation and covariance functions. Because grayscale information is sensitive to illumination variations, feature-based image matching remains the mainstream research approach.

[0004] SIFT has attracted widespread attention from researchers due to its rotation and scale invariance, making it a common method for image feature matching. However, the SIFT feature point extraction process takes a significant amount of time, resulting in long matching times and making it difficult to implement in real-time applications. The recently proposed ORB feature points not only share SIFT's rotation and scale invariance properties but also offer a lower time cost, enabling rapid and accurate image feature matching. Consequently, researchers have applied them to visual SLAM with promising results. However, when point features are scarce or unevenly distributed in an environment, extracting high-quality features that accurately represent the environment is difficult, resulting in inability to achieve high-precision image matching and hindering practical application. Furthermore, most current feature-based image matching algorithms struggle to simultaneously achieve high real-time and accuracy requirements. Furthermore, with changing lighting, uneven feature distribution, and the presence of dynamic occlusion, image feature matching suffers from high error rates, making it difficult to achieve good performance in practical applications. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a line feature matching method based on dynamic matching and absolute median difference, which can achieve high-precision and real-time image feature matching in scenes with occlusion, lighting changes, and perspective changes.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A line feature matching method based on dynamic matching and absolute median difference, the specific steps are as follows:

[0008] 1) The vehicle-mounted RGB-D camera collects RGB images as the input visual image, and uses the EDLines line feature extraction algorithm to detect line features in the current input RGB image;

[0009] 2) Parameterizing the extracted line features to obtain two sets of line feature sets;

[0010] 3) Sample NL evenly spaced discrete key points along each line feature and describe the line feature based on the discrete key points;

[0011] 4) Use the dynamic programming algorithm to perform data association on the line features extracted from the previous and next frames to obtain dynamic matching results;

[0012] 5) The absolute median difference method is used to estimate the standard deviation of the displacement changes of all line feature matches, and the dynamic matching results are optimized to obtain the final high-quality line matching pairs.

[0013] As a further improvement of the present invention, the midline feature parameterization method in step 2) is as follows:

[0014] Based on the EDLines detector, the two sets of line features extracted from the reference image I1 and the current image I2 are:

[0015] L1={ 1 l i | 1 l i =( 1 g i,0 , 1 g i,1 ),i=1,2,…,M1}

[0016] L2={ 2 l j | 2 l j =( 2 g j,0 , 2 g j,1 ),j=1,2,…,M2}

[0017] Among them, M1 and M2 represent the number of extracted line features in images I1 and I2 respectively, ( 1 g i,0 , 1 g i,1 )and( 2 g j,0 , 2 g j,1 ) represent the I1 midline characteristics1 l i and I2 midline characteristics 2 l j endpoint.

[0018] As a further improvement of the present invention, the midline feature description method in step 3) is as follows:

[0019] For two sets of line features, L1 and L2, N samples are sampled along each line feature L Evenly spaced points, including the two endpoints, which means that the detected line features are described as a series of discrete key points, for which the 1 l i and 2 l j Two sequences of sampled points:

[0020] 1 Q i ={ 1 q i,t |t=1,2,…,N L}

[0021] 2 Q j ={ 2 q j,t |t=1,2,…,N L}

[0022] This line description strategy improves robustness to occlusion, viewpoint changes, and endpoint shortening. Even if part of the line is occluded, the keypoints on the non-occluded part will store sufficient line feature details.

[0023] As a further improvement of the present invention, in step 4), dynamic matching is achieved by flexibly matching two key point sequences through a dynamic programming algorithm to find the optimal key point correspondence. The specific process is as follows:

[0024] 1) Line features are described by uniformly sampling 6 key points along each line feature, with a minimum distance of 7 pixels between each key point;

[0025] 2) Based on the key point descriptor, the key point sequence of the I1 midline feature is dynamically matched with the key point sequence of the I2 midline feature in sequence. During the dynamic matching process, when matching two key point sequences, each point can be matched to another point or skipped. The matching score of the two points depends on the similarity of their descriptors. Therefore, the higher the similarity, the higher the score. When a point is skipped, it will be penalized with a gap score of 0.01, so as to adjust the skipping of points with low similarity to better match points with high similarity.

[0026] 3) The total score of line feature matching is the sum of all skip and match operations of key points on the line feature. The score matrix is filled row by row through the dynamic programming algorithm to maximize the total score:

[0027]

[0028] S(t1,t2)=max{S(t1-1,t2)+gap,S(t1,t2-1)+gap,S(t1-1,t2-1)+1 / SimH( 1 d i,t1 , 2 d j,t2 )}

[0029] in, 1 d i,t1 and 2 d j,t2 Represent key points 1 q i,t1 and 2 q j,t2 Descriptor of SimH( 1 d i,t1 , 2 d j,t2 ) represents the similarity between descriptors measured by Hamming distance, Represents the exclusive OR operator;

[0030] 4) Once the matrix S is filled, the highest score in the grid is selected as the candidate line pair 1 l i and 2 l j Finally, each line detected in image I1 is matched with the line in image I2 that has the maximum matching score.

[0031] As a further improvement of the present invention, in step 5), the absolute median difference method is as follows:

[0032] 1) Assume that the result set of dynamic matching is:

[0033] LM={( 1 l h(d) , 2 l f(d) ),d=1,2,...,N M}

[0034] in( 1 l h(d) , 2 l f(d) ) represents a pair of matching lines, h(d)∈[1,M1],f(d)∈[1,M2], N MRepresents the number of line feature matches, where the midpoints of these matching lines are:

[0035]

[0036] 2) For each pair of matching line midpoints in LM, a rotation angle is calculated as:

[0037]

[0038] 3) Consider the midpoint rotation angle of all matching line segments, Φ={Φ1,…,Φ NM}, the absolute median difference is defined as:

[0039] MAD=median(|Φ d -median(Φ)|)

[0040] Where median(·) is the median function, and the standard deviation of Φ is σ=1.4286*MAD

[0041] 4) Based on the mean value of Φ, i.e., mean(Φ), a single threshold function between true and false line matches is described as:

[0042]

[0043] If the matching line segment satisfies True, the line feature matching is retained; if the matching line segment satisfies False, it is discarded.

[0044] The beneficial effects brought about by adopting the above technical solution are:

[0045] This paper proposes a line feature matching method based on dynamic matching and absolute median difference. It first dynamically matches line features using keypoint correspondences, then refines the matching quality using absolute median difference based on rotational consistency constraints. This method effectively improves line feature matching accuracy and reduces image matching time in scenarios with occlusion, low texture, rotation, viewpoint, and appearance changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a system flow chart of the line feature matching method based on dynamic matching and absolute median difference in the present invention;

[0047] Figure 2 a is a schematic diagram showing the correspondence relationship between key points constructed by midline features of the present invention;

[0048] Figure 2 b is a schematic diagram of the score matrix expression for filling dynamic matching in the present invention;

[0049] Figure 3Schematic diagram of image line feature matching results in scenes with perspective changes and occlusions in the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0051] Reference Figure 1 As shown, the line feature matching method of the present invention mainly includes four stages: line feature parameterization, line feature description, dynamic matching and absolute median difference estimation.

[0052] The line feature matching method based on dynamic matching and absolute median difference of the present invention specifically includes:

[0053] 1) The vehicle-mounted RGB-D camera captures an RGB image as the visual input image. The EDLines line feature extraction algorithm is used to detect line features in the current input RGB image.

[0054] 2) Parameterize the extracted line features to obtain two sets of line feature sets.

[0055] 3) Sample N features along each line L The line features are described based on the discrete key points.

[0056] 4) Use the dynamic programming algorithm to perform data association on the line features extracted from the previous and next frames to obtain dynamic matching results.

[0057] 5) The absolute median difference method is used to estimate the standard deviation of the displacement changes of all line feature matches, and the dynamic matching results are optimized to obtain the final high-quality line matching pairs.

[0058] First, based on the EDLines detector, the two sets of line features extracted from the reference image I1 and the current image I2 are:

[0059] L1={ 1 l i | 1 l i =( 1 g i,0 , 1 g i,1 ),i=1,2,…,M1}

[0060] L2={ 2 l j | 2 l j =( 2 g j,0 , 2 g j,1 ),j=1,2,…,M2}

[0061] Among them, M1 and M2 represent the number of extracted line features in images I1 and I2 respectively, ( 1 g i,0 , 1 g i,1 )and( 2 g j,0 , 2 g j,1 ) represent the I1 midline characteristics 1 l i and I2 midline characteristics 2 l j endpoint.

[0062] Then, for two sets of line features, L1 and L2, we sample N along each line feature L points evenly spaced (including the two endpoints), which means that the detected line features are described as a series of discrete key points. To this end, the 1 l i and 2 l j Two sequences of sampled points:

[0063] 1 Q i ={ 1 q i,t |t=1,2,…,N L}

[0064] 2 Q j ={ 2 q j,t |t=1,2,…,N L}

[0065] This line description strategy can improve the robustness to occlusion, viewpoint change and endpoint shortening. Even if part of the line is occluded, the keypoints on the non-occluded part will store sufficient line feature details.

[0066] like Figure 2 As shown in Figure 1, two key point sequences are flexibly matched through a dynamic programming algorithm to find the optimal key point correspondence to achieve dynamic matching. The specific process is as follows:

[0067] 1) Line features are described by uniformly sampling 6 key points along each line feature, and a minimum distance of 7 pixels is maintained between each key point.

[0068] 2) Based on the keypoint descriptors, the keypoint sequence of the midline feature of I1 is dynamically matched with the keypoint sequence of the midline feature of I2. During the dynamic matching process, when matching two keypoint sequences, each point can be matched to the other point or skipped. The matching score of two points depends on the similarity of their descriptors, with higher similarity resulting in a higher score. When a point is skipped, a 0.01 gap score penalty is applied to adjust the skipping of low-similarity points to better match high-similarity points.

[0069] 3) The total score of line feature matching is the sum of all skip and match operations on key points on the line feature. The score matrix S6×6 is filled row by row through a dynamic programming algorithm to maximize the total score:

[0070]

[0071] S(t1,t2)=max{S(t1-1,t2)+gap,S(t1,t2-1)+gap,S(t1-1,t2-1)+1 / SimH( 1 d i,t1 , 2 d j,t2 )}

[0072] in, 1 d i,t1 and 2 d j,t2 Represent key points 1 q i,t1 and 2 q j,t2 Descriptor of SimH( 1 d i,t1 , 2 d j,t2 ) represents the similarity between descriptors measured by Hamming distance, Represents the exclusive OR operator.

[0073] 4) If Figure 2 As shown in b, once the matrix S is filled, the highest score in the grid is selected as the candidate line pair 1 l i and 2 l j Finally, each line detected in image I1 is matched with the line in image I2 that has the maximum matching score.

[0074] Finally, the line feature matching quality is improved based on the absolute median difference method, as follows:

[0075] 1) Assume that the result set of dynamic matching is:

[0076] LM={(1 l h(d) , 2 l f(d) ),d=1,2,...,N M}

[0077] in( 1 l h(d) , 2 l f(d) ) represents a pair of matching lines, h(d)∈[1,M1],f(d)∈[1,M2]. N M Represents the number of line feature matches. The midpoints of these matching lines are:

[0078]

[0079] 2) For each pair of matching line midpoints in LM, a rotation angle is calculated as:

[0080]

[0081] 3) Consider the midpoint rotation angle of all matching line segments, The absolute median difference is defined as:

[0082] MAD=median(|Φ d -median(Φ)|)

[0083] Where median(·) is the median function, and the standard deviation of Φ is σ=1.4286*MAD

[0084] 4) Based on the mean value of Φ, i.e., mean(Φ), a single threshold function between true and false line matches is described as:

[0085]

[0086] If the matching line segment satisfies True, the line feature matching is retained; if the matching line segment satisfies False, the line feature matching is eliminated.

[0087] Figure 3 A schematic diagram showing the line feature matching results of this method on an image with occlusion and perspective changes. It can be seen that the method of the present invention is highly robust to complex scene images such as those with occlusion and perspective changes, and can match a sufficient number of line features more accurately.

[0088] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A line feature matching method based on dynamic matching and absolute median difference, comprising the following steps, characterized in that: Step 1) The vehicle-mounted RGB-D camera captures an RGB image as the input visual image. The EDLines line feature extraction algorithm is used to detect line features in the current input RGB image. Step 2) Parameterize the extracted line features to obtain two sets of line feature sets; Step 3) Sample NL evenly spaced discrete key points along each line feature and describe the line feature based on the discrete key points; Step 4) Use a dynamic programming algorithm to perform data association on the line features extracted from the previous and next frames to obtain dynamic matching results; In step 4), dynamic matching is achieved by flexibly matching two key point sequences through a dynamic programming algorithm to find the optimal key point correspondence. The specific process is as follows: 1) Line features are described by uniformly sampling 6 key points along each line feature, with a minimum distance of 7 pixels between each key point; 2) Based on the key point descriptor, the key point sequence of the I1 midline feature is dynamically matched with the key point sequence of the I2 midline feature in sequence. During the dynamic matching process, when matching two key point sequences, each point can be matched to another point or skipped. The matching score of the two points depends on the similarity of their descriptors. Therefore, the higher the similarity, the higher the score. When a point is skipped, it will be penalized with a gap score of 0.01, so as to adjust the skipping of points with low similarity to better match points with high similarity. 3) The total score of line feature matching is the sum of all skip and match operations of key points on the line feature, and the score matrix is filled row by row through the dynamic programming algorithm , maximize the total score: ； ； in, 1 d i,t1 and 2 d j,t2 Represent key points 1 q i,t1 and 2 q j,t2 Descriptor of SimH( 1 d i,t1 , 2 d j,t2 ) represents the similarity between descriptors measured by Hamming distance, ⊕ represents the exclusive OR operator; 4) Once the matrix S is filled, the highest score in the grid is selected as the candidate line pair 1 l i and 2 l j Finally, each line detected in image I1 is matched with the line in image I2 that has the largest matching score; Step 5) Use the absolute median difference method to estimate the standard deviation of the displacement changes of all line feature matches, optimize the dynamic matching results, and obtain the final high-quality line matching pairs.

2. The line feature matching method based on dynamic matching and absolute median difference according to claim 1, characterized in that: The method for parameterizing the midline feature in step 2) is as follows: Based on the EDLines detector, the two sets of line features extracted from the reference image I1 and the current image I2 are: L1 = { 1 l i | 1 l i = ( 1 g i,0 , 1 g i,1 ), i=1, 2, …, M1}; L2 = { 2 l j | 2 l j = ( 2 g j,0 , 2 g j,1 ), j=1, 2, …, M2}; Among them, M1 and M2 represent the number of extracted line features in images I1 and I2 respectively, ( 1 g i,0 , 1 g i,1 )and( 2 g j,0 , 2 g j,1 ) represent the I1 midline characteristics 1 l i and I2 midline characteristics 2 l j endpoint.

3. The line feature matching method based on dynamic matching and absolute median difference according to claim 1, characterized in that: The midline feature description method in step 3) is as follows: For two sets of line features, L1 and L2, N samples are sampled along each line feature L Evenly spaced points, including the two endpoints, which means that the detected line features are described as a series of discrete key points, for which the 1 l i and 2 l j Two sequences of sampled points: 1 Q i = { 1 q i, t | t=1, 2, …, N L }; 2 Q j = { 2 q j, t | t=1, 2, …, N L }; This line description strategy improves robustness to occlusion, viewpoint changes, and endpoint shortening. Even if part of the line is occluded, the keypoints on the non-occluded part will store sufficient line feature details.

4. The line feature matching method based on dynamic matching and absolute median difference according to claim 1, characterized in that: In step 5), the absolute median difference method is as follows: 1) Assume that the result set of dynamic matching is: ； in( 1 l h(d) , 2 l f(d) ) represents a pair of matching lines, , , N M Represents the number of line feature matches, where the midpoints of these matching lines are: ； 2) For each pair of matching line midpoints in LM, a rotation angle is calculated as: ； 3) Consider the midpoint rotation angles of all matching line segments, , the absolute median difference is defined as: ； in, is the median function, and the standard deviation of Φ is ; 4) Based on the average value of Φ, that is , a single threshold function between true and false line matches is described as: ； If the matching line segment satisfies True, the line feature matching is retained; if the matching line segment satisfies False, it is discarded.

Images