A line feature matching method based on density space clustering and hough transform

Abstract

The application discloses a line feature matching method based on density space clustering and Hough transformation, which comprises the following steps: detecting line features on an image by using LSD, and calculating LBD feature descriptors of the line features; obtaining initial line feature matching pairs and performing Hough transformation on the line feature matching pairs respectively; converting the "line feature" form of the feature matching pairs on a two-dimensional plane into the "point feature" form on a parameter space; normalizing the end point, middle point coordinates of the line feature pairs and the Hough transformation parameter space coordinates into the 0-1 interval; constructing a distance matrix between the line feature matching pairs, and iteratively performing a space density clustering operation; and obtaining robust and accurate line feature matching pairs through the line feature clustering results. The application solves the problem of accurately removing the wrong line feature matching pairs, avoids the iterative calculation of homography matrix and other geometric constraint conditions, and thus realizes the fast and efficient matching and screening of the line features.

Description

Technical Field

[0001] This invention relates to the fields of image processing and pattern recognition, and specifically to a line feature matching method based on density space clustering and Hough transform. Background Technology

[0002] Line features are another important visual feature in the field of computer vision image processing. Line feature matching, as a core component of computer vision, is crucial for practical tasks such as 3D reconstruction, simultaneous localization and mapping (SLAM), and pose estimation. Currently, line feature matching methods are developing rapidly, and paradigm matching based on deep learning models can also achieve good results. However, achieving high accuracy and robustness in line feature matching while simultaneously satisfying high precision and time efficiency, and without dependence on hardware platforms and sample data, still faces continuous challenges.

[0003] Most existing image line feature matching methods focus on designing line feature descriptors (MLSD, LBD, IILD, etc.), and then achieving feature matching based on the similarity of line feature descriptors and geometric constraints, point-line geometric projection invariants, or through homography matrix constraints between line features. This often relies on other geometric conditions or point features to obtain robust and numerous line feature matching pairs, while the geometric properties of the line features themselves are not fully utilized. Although deep learning-based line feature matching has also made significant progress, the generalization ability of the models needs improvement, and the hardware requirements are relatively demanding. In today's industrial production and daily life, line features are widely used in photogrammetry, pose estimation in computer vision, 3D reconstruction, SLAM, and industrial robotics. Finding an algorithm similar to the point feature matching RANSAC algorithm to accurately and efficiently match line features in images is a key step in robots' understanding and perception of scene structure information, thereby contributing to the automation and intelligence of industry. Summary of the Invention

[0004] The purpose of this invention is to provide a line feature matching method based on density space clustering and Hough transform. This method is based solely on image line feature descriptors and makes full use of the inherent attribute information of the line features. It seeks an algorithm similar to the random sampling consistency of point features to achieve efficient elimination of mismatched line feature pairs, thereby achieving accurate and efficient matching of line features. This invention has significant application value in the fields of image pose estimation and SLAM with multi-visual feature fusion, as well as 3D reconstruction.

[0005] The technical solution to achieve the purpose of this invention is: a method based on density space clustering and Hough transform line feature matching, comprising the following steps:

[0006] Step 1: The camera sensor acquires a sequence of color images in the scene, and the Line Feature Detection Operator (LSD) interface in OpenCV is used to extract line features from the image pairs; the OpenCV Line Feature Descriptor (LBD) interface is used to calculate and extract the binary descriptors of the line features.

[0007] Step 2: Perform bidirectional matching of line features based on the LBD binary descriptor to obtain the initial line feature matching pair results; obtain the coordinates of the feature endpoints and midpoints on the line features of the matching pair, and represent them as column vectors, with the indices of the vectors corresponding to the line feature matching pairs in turn;

[0008] Step 3: Perform Hough transform on the line feature matching pairs to convert the line segment form of the line features on the two-dimensional image into the "point" representation in the parameter space, and represent the coordinates of these transformed points as column vectors.

[0009] Step 4: Based on the initial line feature matching pairs and Hough transform, obtain a series of column vectors of line feature endpoints, midpoint coordinates and Hough transform points, and add corresponding noise terms at the end of these column vectors.

[0010] Step 5: Based on the initial line feature matching pair obtained in Step 4, the coordinates of the endpoints and midpoints and the point column vector form of the Hough transform parameter space are normalized to the 0-1 interval.

[0011] Step 6: Construct a distance matrix corresponding to the line feature matching pairs based on the coordinates of the endpoints and midpoints of the line features on the 2D image, and quantify the distance between the line features and their surrounding line features;

[0012] Step 7: Based on the point form of the parameter space after the Hough transform, construct the distance matrix corresponding to the matching pair, and quantify the distance between the line feature "point" form in the parameter space and its surrounding "point" forms again.

[0013] Step 8: Set the minimum number of clusters and the number of points in the neighborhood for spatial clustering, as well as the neighborhood radius and minimum number of points for DBSCAN clustering;

[0014] Step 9: Establish the weight matrix of the distance matrix constructed in Steps 6 and 7, and calculate the total distance of the final line feature matching pairs;

[0015] Step 10: Execute the DBSCAN spatial clustering algorithm to determine whether the number of correct matching pairs has reached the threshold, and determine whether convergence has been achieved based on the threshold.

[0016] Step 11: Recalculate the minimum number of points and neighborhood radius for spatial clustering, and execute the DBSCAN clustering algorithm again to obtain the final line feature matching pairs.

[0017] Furthermore, the initial matching of image pairs uses the line feature matching algorithm in OpenCV, which obtains the initial matching pairs by comparing the Hamming distance of the binary descriptors of the line features.

[0018] Furthermore, the initial line matching pairs' endpoint and midpoint line feature coordinate attribute information are uniformly represented as the following vectors:

[0019] (1)

[0020] (2)

[0021] in, , and These represent the coordinates of the start point, end point, and midpoint of the LSD line feature detected on image I1, forming an n-dimensional column vector. , and These represent n-dimensional column vectors composed of the starting point coordinates, ending point coordinates, and midpoint coordinates of the LSD line features detected on image I2, respectively. The indices of the corresponding elements in the vectors correspond one-to-one with the line feature matching pairs. , and These represent the coordinates of the start point, end point, and midpoint of the feature line for the i-th matching pair on the I1 image, respectively. , and These represent the coordinates of the start point, end point, and midpoint of the feature line for the i-th matching pair on the I2 image, respectively.

[0022] Furthermore, a Hough transform is performed on the line feature matching pairs, and the linear equation expression of the line feature is calculated based on the coordinates of the start and end points of the line feature in the two-dimensional image:

[0023] (3)

[0024] Here, a, b, and c are constants and Then, based on the Hough transform principle, the coordinates of the line's characteristic endpoints and midpoints are simultaneously represented in parametric space form:

[0025] (4)

[0026] This transforms the line segments of the linear features in the two-dimensional planar image coordinate system into the "point" form of the linear features in the parameter space. It means that among them This represents the distance from the origin to the line segment in the image coordinate system. The angle between a line segment and a coordinate axis in the image coordinate system is represented by the following parameter space vector:

[0027] (5)

[0028] (6)

[0029] , These represent the column vectors in parameter space that represent the LSD line features detected on image I1 after Hough transform. and These represent the elements corresponding to the i-th line feature matching pair in the column vector; , The LSD line features detected on image I2 are then transformed into column vectors in the parameter space by the Hough transform. and This represents the element corresponding to the i-th line feature matching pair in the LSD line feature column vector of image I2.

[0030] Furthermore, the point column vector form of the Hough transform parameter space in equations (1) and (2) above is normalized to the 0-1 interval. The characteristic is that, assuming the initial number of line feature matching pairs is N, the number of random noises is NoiseN and the total number of matches is NN, obtained from the following equations:

[0031] (7)

[0032] The following noise sequence was subsequently generated:

[0033] (8)

[0034] In the above formula, The vectors corresponding to the start, end, and midpoint coordinates of image I1 after adding noise, respectively. and These correspond to the vectors of the parameter space coordinates (ρ, θ) of the online features of image I1 after Hough transform, with noise added. These are the vectors containing the start, end, and midpoint coordinates of image I2 after adding noise. and These correspond to the vectors of the parameter space coordinates (ρ, θ) of the upper features of image I2 after Hough transform, with noise added. Representation generates a A 2x2 random number matrix This means generating a A ×1 order random number matrix (*) indicates repeated operations on the matrix. This represents the element-wise multiplication of matrices. It is then mapped to the interval [0,1] using appropriate normalization operations.

[0035] Furthermore, the line feature space distance matrix in step 6, for a given vector sequence, has the following expression:

[0036] (9)

[0037] (10)

[0038] Let represent the Euclidean distance between the midpoint of line feature i and the midpoint of line feature j in the normalized image I1. The distance matrix constructed from the midpoint geometric conditions of the feature matching pairs on image I1 represents the line feature matching. Let represent the Euclidean distance between the midpoint of line feature i and the midpoint of line feature j in the normalized image I2. The distance matrix is ​​constructed from the midpoints of the features on the I2 line of the image.

[0039] Furthermore, the line feature direction is calculated based on the coordinates of the endpoints and starting point of the line feature, and the cosine similarity between matching pairs can be calculated from the line feature direction:

[0040] (11)

[0041] (12)

[0042] (13)

[0043] and These represent the direction vectors from the starting point to the ending point of the line feature on images I1 and I2, respectively. ( * The cosine distance between the elements of two vectors is represented by . This represents the cosine distance between the line feature matching pairs of images I1 and I2, and forms the corresponding distance matrix. .

[0044] Furthermore, the line feature parameter space distance matrix in step 7, for a given sequence of parameter space vectors, specifically includes:

[0045] (15)

[0046] , (16)

[0047] (17)

[0048] , (18)

[0049] in and Belongs to the parameter space Distance matrix of values.

[0050] (19)

[0051] , (20)

[0052] (twenty one)

[0053] , (twenty two)

[0054] in and Belongs to the parameter space The distance matrix is ​​obtained from the above formula, which gives the total distance matrix in the parameter space:

[0055] (twenty three)

[0056] (twenty four)

[0057] Here [A.*B] 2 We define the product as the element-wise multiplication of corresponding elements in matrices A and B, followed by squaring, to obtain a new matrix. Here, * denotes the element-wise multiplication of corresponding elements. Thus, the line feature distance space matrix of the image to be matched is obtained. and .

[0058] Furthermore, based on the results of steps 6 and 7, the distance weight matrix is ​​calculated using the following formula:

[0059] (25)

[0060] in For parameter factors, The elements in , for Image online features i and The weight elements between the features j on the image are used to construct a weight matrix. .

[0061] Furthermore, assuming there are n line feature matching pairs, first calculate the total distance matrix from these line features to their respective line features:

[0062] (26)

[0063] Where D is the total spatial distance matrix of line feature matching pairs. The weight matrix constructed for formula (25) This represents the mathematical operation of multiplying corresponding elements of two matrices of the same dimension.

[0064] Furthermore, in step 8, the minimum number of spatial clusters and the number of points in the neighborhood are set, as well as the clustering cutoff condition. Specifically, the minimum number of spatial clusters is pre-generated based on the matching pairs of image line features. The iteration cutoff condition is that when more than 90% of the matching pairs can be obtained in the first clustering, it is considered to have converged.

[0065] Furthermore, if step 10 determines that the clustering has not converged, the clustering is re-executed. The minimum number of linear features and the neighborhood radius required by the clustering algorithm are adaptively determined as follows: Assuming the number of linear feature matching pairs is n, a minimum number of linear features K is first determined here.

[0066] (27)

[0067] Where N represents the number of any initial line feature matching pairs, Pct represents the pre-set proportion of inlier points in the line feature matching, and [*] represents the rounding symbol in mathematics. BL represents the initial threshold setting, which adaptively sets the minimum number of features within a neighborhood of any linear feature in the space, thus forming a neighborhood set. For neighborhood radius Its characteristic is that it has the following formula:

[0068] (28)

[0069] here The value range is in the interval [0,1].

[0070] Compared with the prior art, the significant advantages of the present invention are: the present invention does not rely on the image fundamental matrix or homography matrix, but solves the accuracy and robustness problem of image line feature matching only through the initial line feature descriptor, and achieves accurate matching of line features by combining the initial line feature matching pairs with the line feature distribution density and DBSCAN clustering iteration. Attached Figure Description

[0071] Figure 1 This is a flowchart of the linear feature matching method based on density space clustering and Hough transform of the present invention.

[0072] Figure 2 This is an example image of image point feature matching for the method of the present invention.

[0073] Figures 3-6 This is an example diagram of feature matching pairs based on the cascaded matching line of the present invention.

[0074] Figure 7 This is an example diagram of a 3D reconstruction generated based on the line feature matching pairs of this invention. Detailed Implementation

[0075] This invention proposes a line feature matching method based on density space clustering and Hough transform. The method mainly includes the following steps: acquiring optical or aerial photographic digital image sequences; using the OpenCV open-source computer vision library, employing the LSD algorithm to detect and extract line features from image sequence pairs, and calculating binary descriptors for image point features; comparing the similarity of line feature descriptors based on bidirectional matching of descriptor distance to obtain initial line feature matching pairs; obtaining the coordinates of the endpoints and midpoints of the line features based on the initial line feature matching pairs; calculating the line feature equations based on the two-dimensional coordinates of the line feature images; introducing the Hough transform to convert the line features into parameter space coordinates, mapping the endpoint and midpoint coordinates of the line features, as well as the parameter coordinates of the line feature space, to the 0-1 interval; setting corresponding noise terms and DBSCAN clustering parameters; and using the DBSCAN algorithm to perform cluster analysis on the point features to obtain line feature matching pairs. This invention does not rely on the fundamental matrix or homography geometric constraints between image pairs, but first solves the image point feature matching using only the basic point feature descriptors, and then completes robust matching of line features based on the multimodal matching quantity of line features. This method has strong applicability in the field of image processing pose estimation.

[0076] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0077] Combination Figure 1 A line feature matching method based on density space clustering and Hough transform is proposed. First, initial line feature matching pairs are obtained using binary line feature descriptors. Then, the line feature density distribution and adaptive parameter settings are combined to construct a distance matrix between line feature matching pairs. Iterative clustering is then performed to obtain correct line feature matches, achieving robust matching of line features. The method includes the following steps:

[0078] Step 1: The camera sensor acquires a sequence of color images in the scene, and the Line Feature Detection Operator (LSD) interface in OpenCV is used to extract line features from the image pairs; the OpenCV Line Feature Descriptor (LBD) interface is used to calculate and extract the binary descriptors of the line features.

[0079] Step 2: Perform initial matching of line features based on the LBD binary descriptor to obtain the initial point feature matching pair results; obtain the attribute information such as the coordinates of the feature endpoints and midpoints on the two-dimensional image of the matching pair line features, and represent them as column vectors. The indices of the vectors correspond to the line feature matching pairs in sequence, as shown below:

[0080] (1)

[0081] (2)

[0082] in, , and These represent the coordinates of the start point, end point, and midpoint of the LSD line feature detected on image I1, forming an n-dimensional column vector. , and These represent n-dimensional column vectors composed of the starting point coordinates, ending point coordinates, and midpoint coordinates of the LSD line features detected on image I2, respectively. The indices of the corresponding elements in the vectors correspond one-to-one with the line feature matching pairs. , and These represent the coordinates of the start point, end point, and midpoint of the feature line for the i-th matching pair on the I1 image, respectively. , and These represent the coordinates of the start point, end point, and midpoint of the feature line for the i-th matching pair on the I2 image, respectively.

[0083] Step 3: Perform a Hough transform on the line feature matching pairs to convert the line segment form of the line features in the 2D image into a "point" representation in the parameter space, and represent the coordinates of these transformed points as column vectors; perform a Hough transform on the line feature matching pairs, and calculate the conventional straight line equation form of the line features based on the coordinates of the start and end points of the line features in the 2D image:

[0084] (3)

[0085] Here, a, b, and c are constants and Then, based on the Hough transform principle, the two-dimensional line segment is represented as follows:

[0086] (4)

[0087] This transforms the line segment form of the linear feature in the two-dimensional plane image coordinate system into the "point" form of the linear feature in the parameter space. It means that among them This represents the distance from the origin to the line segment in the image coordinate system. This represents the angle between a line segment and the coordinate axis in the image coordinate system, thus forming the following parameter space vector:

[0088] (5)

[0089] (6)

[0090] , These represent the column vectors in parameter space that represent the LSD line features detected on image I1 after Hough transform. and These represent the elements corresponding to the i-th line feature matching pair in the column vector; , The LSD line features detected on image I2 are then transformed into column vectors in the parameter space by the Hough transform. and This represents the element corresponding to the i-th line feature matching pair in the LSD line feature column vector of image I2.

[0091] Step 4: Based on the initial line feature matching pairs and Hough transform, obtain a series of column vectors containing the coordinates of the line feature endpoints, midpoints, and Hough transform points. Add a series of noise terms to the end of these column vectors. The resulting initial line feature matching pairs, in the form of a series of coordinate points and a point column vector in the Hough transform parameter space with noise terms, are normalized to the 0-1 interval. The key feature is that, assuming the initial number of matching pairs is N, the number of random noise points (NoiseN) and the total number of matching pairs (NN) are obtained using the following formula:

[0092] (7)

[0093] The following noise sequence was subsequently generated:

[0094] (8)

[0095] In the above formula, The vectors corresponding to the start, end, and midpoint coordinates of image I1 after adding noise, respectively. and These are the vectors with noise added to the parameter space coordinates (ρ, θ) of the online features in image I1 after Hough transform. , The vectors corresponding to the start, end, and midpoint coordinates of image I2 after adding noise are respectively... and Then, the vectors corresponding to the parameter space coordinates (ρ, θ) of the upper features of image I2 after Hough transform are respectively given with noise added. Representation generates a A 2x2 random number matrix This means generating a A ×1 order random number matrix (*) indicates repeated operations on the matrix. This represents the element-wise multiplication of matrices. It is then mapped to the interval [0,1] using appropriate normalization operations.

[0096] Step 5: Based on the initial line feature matching pair obtained in Step 4, a series of coordinate points (endpoints, midpoints) with noise terms and the point column vector form of the Hough transform parameter space are normalized to the 0-1 interval.

[0097] Step 6: Construct a distance matrix corresponding to the line feature matching pairs based on the coordinates of the endpoints and midpoints of the line features in the 2D image, quantifying the distance between the line features and their surrounding line features; the spatial distance matrix of the line features, for a given vector sequence, has the following expression:

[0098] (9)

[0099] (10)

[0100] Let represent the Euclidean distance between the midpoint of line feature i and the midpoint of line feature j in the normalized image I1. The distance matrix constructed from the midpoint geometric conditions of the feature matching pairs on image I1 represents the line feature matching. Let represent the Euclidean distance between the midpoint of line feature i and the midpoint of line feature j in the normalized image I2. The distance matrix is ​​constructed from the midpoints of the features on the I2 line of the image.

[0101] Step 7: Based on the point form of the parameter space after the Hough transform, construct the distance matrix corresponding to the matching pair, and quantify the distance between the line feature "point" form in the parameter space and its surrounding "point" forms again; calculate the line feature direction based on the coordinates of the endpoints and starting points of the line feature, and calculate the cosine similarity between the matching pairs from the line feature direction:

[0102] (11)

[0103] (12)

[0104] (13)

[0105] and These represent the direction vectors from the starting point to the ending point of the line feature on images I1 and I2, respectively. ( * The cosine distance between the elements of two vectors is represented by . This represents the cosine distance between the line feature matching pairs of images I1 and I2, and forms the corresponding distance matrix. .

[0106] The line feature parameter space distance matrix in step 7, for a given sequence of parameter space vectors, specifically includes:

[0107] (15)

[0108] , (16)

[0109] (17)

[0110] , (18)

[0111] in and Belongs to the parameter space Distance matrix of values.

[0112] (19)

[0113] , (20)

[0114] (twenty one)

[0115] , (twenty two)

[0116] in and Belongs to the parameter space The distance matrix is ​​obtained from the above formula, which gives the total distance matrix in the parameter space:

[0117] (twenty three)

[0118] (twenty four)

[0119] Here [A.*B] 2 We define the product as the element-wise multiplication of corresponding elements in matrices A and B, followed by squaring, to obtain a new matrix. Here, * denotes the element-wise multiplication of corresponding elements. Thus, the distance space matrix of the image is obtained. and .

[0120] Step 8: Set the minimum number of clusters and the number of points in the neighborhood for spatial clustering, as well as the cutoff conditions for clustering;

[0121] Step 9: Establish the weighting mechanism for the distance matrix constructed in Steps 6 and 7, and calculate the total distance of the final line feature matching pairs; the formula for calculating the distance weighting factor between Steps 6 and 7 is:

[0122] (25)

[0123] in For parameter factors, The elements in , for Image online features i and The weight elements of the image's online feature j are used to construct a weight matrix. .

[0124] Furthermore, assuming there are n line feature matching pairs, first calculate the total distance matrix from these line features to their respective line features:

[0125] (26)

[0126] Where D is the total spatial distance matrix of line feature matching pairs. The weight matrix constructed in formula (25) This represents the mathematical operation of multiplying corresponding elements of two matrices of the same dimension.

[0127] Step 10: Execute the DBSCAN spatial clustering algorithm to determine whether the number of correct matching pairs has reached the threshold, and determine whether convergence has been achieved based on the threshold.

[0128] Step 11: If convergence fails, adjust the weight parameters and execute the DBSCAN clustering algorithm again.

[0129] First, we need to determine a minimum value, K.

[0130] (27)

[0131] Where N represents the number of feature matching pairs for any initial line, and Pct represents the proportion of inliers. * The symbol ] represents the integer part of a mathematical function. BL represents the initial threshold setting, which adaptively sets the minimum number of points in a neighborhood of any linear feature within the space, thus forming a neighborhood set. For neighborhood radius Its characteristic is that it has the following formula:

[0132] (28)

[0133] here The value range is in the interval [0,1].

[0134] Example

[0135] The specific implementation process of the present invention will be briefly described below with reference to the accompanying drawings:

[0136] (1) In this embodiment, any pair of images to be matched is selected, LSD line features are detected and extracted respectively, and the corresponding line feature descriptors are calculated. The initial matching of line features is completed according to the Hamming distance of the feature descriptors, and the initial matching pair of line features is obtained.

[0137] (2) Select the initial line feature matching pair, calculate the linear equation of the image line feature according to the feature endpoint coordinates, and convert the two-dimensional form of the line feature into the point feature form of the parameter space. At the same time, set the addition of noise term and normalize the line feature endpoint coordinates to the 0-1 interval.

[0138] (3) Based on the coordinates of the endpoints of the line features and the coordinates of the parameter space, construct the distance matrix of the line features, construct the distance matrix between adjacent features based on the line feature matching pairs, and construct the distance matrix D after the line features are converted into the parameter space;

[0139] (4) Set the corresponding clustering parameters Pct=5%, BU=3, BL=10, γ=10, u=0.09, perform the first DBSCAN density clustering, obtain the line feature matching pairs of the first clustering, and update the total distance matrix D;

[0140] (5) Set the corresponding clustering parameters Pct=5%, BU=3, BL=10, γ=10, u=0.1 again, perform the second DBSCAN density clustering, obtain the line feature matching pairs after the second clustering, and take this line feature matching pair as the final line feature matching pair;

[0141] (6) Based on the final line feature matching pairs obtained by this method, the known camera projection matrix of the image to be matched can be used to obtain the three-dimensional sparse reconstruction map of the line features.

[0142] Figure 1 The specific steps and flow of the method of the present invention in actual operation are shown. Figure 2 The image shown is the original image to be matched in this example. Figure 2 The four sets of images are all representative scene images: the first set of images has obvious rotational changes, the second set is an indoor low-texture scene, the third set consists of two sets of images under different viewpoints, and the fourth set consists of two sets of images in a large outdoor scene. Figure 3 Demonstrated targeting Figure 2 The line feature matching results obtained from four sets of images. In actual implementation, directly using line feature descriptors to match these four sets of images does not yield ideal results, with many false matches. After applying the method of this invention,Figure 2 The first group of images matched a total of 329 pairs of line features, of which 327 pairs were correct, with an accuracy rate of 99.3%; the second group matched 197 pairs of line features, of which 196 pairs were correct, with an accuracy rate of 99.4%; the third group matched 821 pairs of line features, of which 816 pairs were correct, with an accuracy rate of 99.3%; and the fourth group matched 2584 pairs of line features, of which 2564 pairs were correct, with an accuracy rate of 99.2%.

[0143] Figure 2 The fourth group of original images to be matched provides the corresponding image projection matrix. This invention combines... Figures 3-6 The line feature matching results shown are used to generate corresponding 3D reconstructed line drawings of buildings using the projection matrix. It is clear from the figures that the image line feature matching pairs obtained based on this invention can accurately reconstruct the general outline of the building; this fully demonstrates the accuracy and robustness of this invention in image line feature matching results, and indirectly illustrates that this invention can meet the practical application needs of various subsequent applications based on visual feature matching results. Figure 7 As can be seen, the building reconstructed based on the matched line features and image projection matrix is ​​clear, with a sharp outline, complete structure, and good rendering of details such as wall edges and window frames. No obvious distortion or drift caused by mismatches is observed. This example further verifies that the method of the present invention not only has high accuracy in 2D line feature matching but also effectively supports high-precision 3D reconstruction tasks, meeting the practical engineering application needs of SLAM, UAV mapping, and industrial-grade visual positioning.

[0144] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Various modifications, substitutions, and improvements made by those skilled in the art to the technical solutions of the present invention based on the provided textual description and drawings, without departing from the design concept and spirit of the present invention, should all fall within the scope of protection of the present invention.

Claims

1. A linear feature matching method based on density space clustering and Hough transform, characterized in that, Includes the following steps: Step 1: The camera sensor acquires a sequence of color images in the scene, and the Line Feature Detection Operator (LSD) interface in OpenCV is used to extract line features from the image pairs; the OpenCV Line Feature Descriptor (LBD) interface is used to calculate and extract the binary descriptors of the line features. Step 2: Perform bidirectional matching of line features based on the LBD binary descriptor to obtain the initial line feature matching pair results; obtain the coordinates of the feature endpoints and midpoints on the line features of the matching pair, and represent them as column vectors, with the indices of the vectors corresponding to the line feature matching pairs in turn; Step 3: Perform Hough transform on the line feature matching pairs to convert the line segment form of the line features on the two-dimensional image into the "point" representation in the parameter space, and represent the coordinates of these transformed points as column vectors. Step 4: Based on the initial line feature matching pairs and Hough transform, obtain a series of column vectors of line feature endpoints, midpoint coordinates and Hough transform points, and add corresponding noise terms at the end of these column vectors. Step 5: Based on the initial line feature matching pair obtained in Step 4, the coordinates of the endpoints and midpoints and the point column vector form of the Hough transform parameter space are normalized to the 0-1 interval. Step 6: Construct a distance matrix corresponding to the line feature matching pairs based on the coordinates of the endpoints and midpoints of the line features on the 2D image, and quantify the distance between the line features and their surrounding line features; Step 7: Based on the point form of the parameter space after the Hough transform, construct the distance matrix corresponding to the matching pair, and quantify the distance between the line feature "point" form in the parameter space and its surrounding "point" forms again. Step 8: Set the minimum number of clusters and the number of points in the neighborhood for spatial clustering, as well as the neighborhood radius and minimum number of points for DBSCAN clustering; Step 9: Establish the weight matrix of the distance matrix constructed in Steps 6 and 7, and calculate the total distance of the final line feature matching pairs; Step 10: Execute the DBSCAN spatial clustering algorithm to determine whether the number of correct matching pairs has reached the threshold, and determine whether convergence has been achieved based on the threshold. Step 11: Recalculate the minimum number of points and neighborhood radius for spatial clustering, and execute the DBSCAN clustering algorithm again to obtain the final line feature matching pairs.

2. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, The initial matching of image pairs uses the line feature matching algorithm in OpenCV, which obtains the initial matching pairs by comparing the Hamming distance of the binary descriptors of the line features.

3. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, The initial line matching endpoint and midpoint line feature coordinate attribute information are uniformly represented as the following vectors: ， ， （1） ， ， （2） in, , and These represent the coordinates of the start point, end point, and midpoint of the LSD line feature detected on image I1, forming an n-dimensional column vector. , and These represent n-dimensional column vectors composed of the starting point coordinates, ending point coordinates, and midpoint coordinates of the LSD line features detected on image I2, respectively. The indices of the corresponding elements in the vectors correspond one-to-one with the line feature matching pairs. , and These represent the coordinates of the start point, end point, and midpoint of the feature line for the i-th matching pair on the I1 image, respectively. , and These represent the coordinates of the start point, end point, and midpoint of the feature line for the i-th matching pair on the I2 image, respectively.

4. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, Perform a Hough transform on the line feature matching pairs, and calculate the conventional linear equation form of the line features based on the coordinates of the start and end points of the line features in the 2D image: （3） Here, a, b, and c are constants and Then, based on the Hough transform, the two-dimensional line segment is represented as follows: （4） This transforms the line segment form of the linear feature in the two-dimensional plane image coordinate system into the "point" form of the linear feature in the parameter space. It means that among them This represents the distance from the origin to the line segment in the image coordinate system. The angles between line segments and coordinate axes in the image coordinate system are represented by the following parameter space vector: （5） （6） , These represent the column vectors in parameter space that represent the LSD line features detected on image I1 after Hough transform. and These represent the elements corresponding to the i-th line feature matching pair in the column vector; , The LSD line features detected on image I2 are then transformed into column vectors in the parameter space by the Hough transform. and This represents the element corresponding to the i-th line feature matching pair in the LSD line feature column vector of image I2.

5. A line feature matching method based on density space clustering and Hough transform according to claim 1, wherein the endpoint and midpoint coordinate vectors of the initial line feature matching pairs obtained in step 4, with noise terms, are normalized to the 0-1 interval, characterized in that, Assuming the initial number of matched pairs is N, the number of random noise items, NoiseN, and the total number of matches, NN, are obtained from the following formulas: （7） The following noise sequence was subsequently generated: （8） In the above formula, , These are the vectors containing the start, end, and midpoint coordinates of image I1 after adding noise. and These correspond to the vectors of the parameter space coordinates (ρ, θ) of the online features in image I1 after Hough transform, with noise added. These are the vectors containing the start, end, and midpoint coordinates of image I2 after adding noise. and The vectors corresponding to the parameter space coordinates (ρ, θ) of the upper features of image I2 after Hough transform are respectively modified by adding noise. Representation generates a A 2x2 random number matrix This means generating a A ×1 order random number matrix (*) indicates a repetitive operation on the matrix; it is mapped to the [0,1] interval by the corresponding normalization operation.

6. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, The distance matrix for the line feature matching pairs in step 6, for a given vector sequence, is expressed as follows: （9） （10） Let represent the Euclidean distance between the midpoint of line feature i and the midpoint of line feature j in the normalized image I1. The distance matrix constructed from the midpoint geometric conditions of the feature matching pairs on image I1 represents the line feature matching. Let represent the Euclidean distance between the midpoint of line feature i and the midpoint of line feature j in the normalized image I2. The distance matrix is ​​constructed from the midpoints of the line features on image I2; the line feature direction is calculated based on the coordinates of the endpoints and starting points of the line features, and the cosine similarity between matching pairs is calculated from the line feature direction. （11） （12） （13） and These represent the direction vectors from the starting point to the ending point of the line feature on images I1 and I2, respectively. (*) represents the cosine distance between the elements of two vectors. This represents the cosine distance between the line feature matching pairs of images I1 and I2, and forms the corresponding distance matrix. .

7. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, The distance matrix for constructing line feature matching pairs in step 7, for a given vector sequence, is expressed as follows: （15） ， （16） （17） ， （18） in and Belongs to the parameter space Distance matrix of values; （19） ， （20） （21） ， （22） in and Belongs to the parameter space The distance matrix is ​​obtained from the above formula, which gives the total distance matrix in the parameter space: （23） （24） Among them, [A] B] 2 Defined as multiplying corresponding elements of matrix A and matrix B, then squaring them to obtain a new matrix. Thus, the distance space matrix of the online features of the image to be matched is obtained. and .

8. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, The distance weight matrix in step 9 is then calculated based on the parameters from steps 6 and 7, and the calculation formula is as follows: （25） in For parameter factors, The elements in , for Image online features i and The weight elements of the image's online feature j are used to construct a weight matrix. ; Assuming there are n line feature matching pairs, calculate the total distance matrix from these points to their respective line features: （26） Where D is the total spatial distance matrix of line feature matching pairs. The weight matrix in formula (25) is... This represents the mathematical operation of multiplying corresponding elements of two matrices of the same dimension.

9. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, In step 8, the minimum number of spatial clusters and the number of points in the neighborhood are set, as well as the clustering cutoff condition. Specifically, the minimum number of spatial clusters is pre-generated based on the matching pairs of image line features. The iteration cutoff condition is that when more than 90% of the matching pairs can be obtained in the first clustering, it is considered to have converged.

10. The linear feature matching method based on density space clustering and Hough transform according to claim 1, characterized in that, If step 10 determines that the clustering has not converged, the clustering is re-executed. The minimum number of linear features and the neighborhood radius required by the clustering algorithm are adaptively determined as follows: Assuming the number of matching linear feature pairs is n, first determine the minimum neighborhood value K for each linear feature: （27） Where N represents the number of feature matching pairs for any initial line, and Pct represents the proportion of inliers. * The symbol ] represents the integer part of a mathematical function. BL represents the initial threshold setting, which adaptively sets the minimum number of points in a neighborhood of any linear feature within the space, thus forming a neighborhood set. For neighborhood radius The following formula is given: （28） in, The value range is in the interval [0,1].

Images