A point-line feature cascade matching method with adaptive kernel density clustering and multi-modal constraint
By adopting an adaptive kernel density clustering and multimodal constraint point-line feature cascade matching method, the accuracy and robustness issues of point-line feature matching in complex scenes are solved, achieving efficient pose estimation and 3D reconstruction in the field of image processing.
Patent Information
- Application Number
- CN202610350854.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-20
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies struggle to guarantee the robustness and accuracy of point and line feature matching in complex scenarios such as large-scale scaling, wide-viewpoint changes, occlusion, and lighting variations. In particular, line feature matching methods perform poorly in these scenarios, and methods based on deep learning models lack generalization and have high hardware requirements.
A point-line feature cascade matching method based on adaptive kernel density clustering and multimodal constraints is adopted. Point-line features are extracted using OpenCV operators, and initial matching is performed by combining point feature descriptors. Kernel density estimation and DBSCAN clustering are used to adjust parameters, and geometric similarity, radiation similarity and normalized cross-correlation similarity of line features are calculated. Finally, line feature matching is completed through a multi-hypothesis voting mechanism.
It achieves accuracy and robustness in point and line feature matching in complex scenes, improves the effect of pose estimation and 3D reconstruction in the field of image processing, and reduces the dependence on hardware.
Smart Images
Figure CN122368540A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and specifically to a point-line feature cascade matching method based on adaptive kernel density clustering and multimodal constraints. Background Technology
[0002] Feature matching, as a core component of computer vision, is crucial for practical applications such as 3D reconstruction, simultaneous localization and mapping (SLAM), and image registration. Although point feature matching methods are relatively mature, and paradigm matching based on deep learning models can achieve good results, high-precision and robust point-line feature matching, especially line feature matching, still faces continuous challenges when dealing with complex scenes such as large-scale scaling, wide-viewpoint changes, occlusion, lighting variations, and low texture.
[0003] Currently, most point and line feature matching methods are performed independently, meaning the algorithms either only perform point feature matching or only focus on line feature matching. Furthermore, most existing image line feature matching methods focus on designing line feature descriptors (MLSD, LBD, IILD, etc.) and then achieving feature matching based on the similarity of the line feature descriptors and geometric constraints, or through homography constraints between line features. This makes it difficult to obtain robust and numerous line feature matching pairs in complex scenes with large viewpoints, scaling, and occlusion. While deep learning-based line feature matching has made significant progress, the generalization ability of the models needs improvement, and the hardware requirements are relatively demanding. In today's industrial production and daily life, point and line features are widely used in photogrammetry, pose estimation in computer vision, 3D reconstruction, SLAM, and industrial robotics. Accurate and efficient matching of point and line features in images is a crucial step in robots' understanding and perception of scene structure information, thus contributing to the automation and intelligentization of industry. Summary of the Invention
[0004] The purpose of this invention is to provide a point-line feature cascade matching method based on adaptive kernel density clustering and multimodal constraints. This method is based solely on image point feature descriptors and does not rely on point feature geometric constraints. This invention also does not rely on line feature descriptors and completes line feature matching based on point features. This method has significant application value in the fields of image pose estimation, SLAM with multi-visual feature fusion, and 3D reconstruction.
[0005] The technical solution to achieve the objective of this invention is: an adaptive kernel density clustering and multimodal constraint point-line feature concatenation matching method, comprising the following steps:
[0006] Step 1: The camera sensor acquires a sequence of color images in the scene, and calls the SIFT point feature detection operator interface and LSD line feature detection interface in OpenCV to extract point and line features from the image pairs; then, the OpenCV point feature descriptor interface is called to calculate and extract the binary descriptors of the point features.
[0007] Step 2: Perform initial point feature matching based on the binary descriptor of the point features to obtain the initial point feature matching pair results; estimate the similarity transformation variables between the two views based on the initial point feature matching pairs: translation, rotation and scaling factors, and combine them with the preset point matching pair in-point ratio threshold to comprehensively determine whether the matching pair transformation operation needs to be performed.
[0008] Step 3: Set adaptive variable thresholds related to feature point matching, including translation threshold, rotation threshold, and scaling threshold; when at least one of the adaptive variable thresholds changes more than the corresponding preset threshold, and the proportion of in-points in the point feature matching mentioned in Step 2 is greater than the previously set threshold, the system determines that pre-transformation processing is required.
[0009] Step 4: Based on the initial point feature matching pairs, construct a feature vector expression that includes position and motion information;
[0010] Step 5: Analyze the spatial distribution of feature points on the image by kernel density estimation, and adaptively adjust the subsequent point feature matching parameters based on the spatial distribution of point features. Estimate the spatial distribution density of feature points on the image by using KDE.
[0011] Step 6: After completing the density calculation for all grid points, extract and calculate two key density features: global average density and density variation coefficient;
[0012] Step 7: Run the DBSCAN clustering algorithm to cluster points based on the motion of feature matching pairs and execute the feature point matching iterative algorithm.
[0013] Step 8: Parameterize the image line features, including calculating the direction vector, length, slope, and intercept of the line features; use spatial grid partitioning technology to divide the image into multiple grid cells and assign line segments to the grid cells that intersect with their bounding boxes;
[0014] Step 9: Calculate the geometric similarity between any two line features;
[0015] Calculate the radiation similarity between line features, define the average gray value on the left and the average gray value on the right of the reference line segment and the target line segment; evaluate the illumination consistency by comparing the gray value changes on both sides of the same position in the reference image and the target image, and determine the line feature matching pairs;
[0016] Calculate the normalized cross-correlation similarity between line features;
[0017] Compute the geometric invariants within the support domain of the local point set;
[0018] Step 10: The comprehensive similarity of line segment pairs is calculated by weighting geometric similarity, radial similarity, normalized cross-correlation similarity, and geometric invariants. A multi-hypothesis voting mechanism is used for the final decision to obtain the final line feature matching pair.
[0019] Furthermore, the point-to-point ratio refers to the number of correct matches divided by the total number of matches.
[0020] Furthermore, the rigid transformation model for the image pair is as follows:
[0021] (1)
[0022] Where T is the transformation matrix (t x ,t y ) is the translation vector, θ is the rotation angle, x and y represent the two-dimensional coordinates of the feature points on the image, and s is the scaling factor.
[0023] Furthermore, the matching of each initial point is represented as a feature vector containing both position and motion information, uniformly characterized as follows:
[0024] (2)
[0025] Define the difference components: .
[0026] Accurately measure the similarity between two matches based on a weighted distance function:
[0027] (3)
[0028] in Indicates taking a vector The first dimension to the second dimension, that is, the first two components, which are... The part; where the weighting coefficients are:
[0029] (4)
[0030] To adjust the parameters, the subscripts i and j represent the indices of the corresponding point feature matching pairs. , These are the two-dimensional coordinates of the feature point image; this distance function comprehensively considers spatial location differences and motion vector differences, and emphasizes the motion consistency of spatially neighboring point pairs through adaptive weights; in the distance calculation, the spatial location differences are measured using Euclidean distance, while the motion vector differences are balanced by adjusting parameters.
[0031] Furthermore, KDE (Kernel Density Estimation) is used to estimate the spatial distribution density of feature points on the image plane. First, equally spaced grid points are generated in both the width and height directions of the image to cover the entire image. After the image grid is constructed, the bandwidth parameter h for kernel density estimation is calculated using the following formula:
[0032] (5)
[0033] in The number of feature points, The standard deviation estimate of the feature point coordinates is calculated. and The average of the directional standard deviations is obtained as follows:
[0034] (6)
[0035] Based on the defined grid and bandwidth parameters, calculate the value for each grid point. The density value; for any grid point, first calculate the Euclidean distance from all feature points to that point:
[0036] (7)
[0037] x k For the first For each feature point, the density contribution of that feature point to the grid point is calculated using a Gaussian kernel function, and the average value is taken to obtain the final density estimate f. ij :
[0038] (8)
[0039] Furthermore, global average density It is the arithmetic mean of the density values of all grid points:
[0040] (9)
[0041] The coefficient of variation (CVD) is the ratio of the standard deviation to the mean of the grid point density, used to characterize the uniformity of the characteristic distribution.
[0042] (10)
[0043] Based on the extracted density features, the algorithm establishes a parameter adaptive adjustment mechanism; based on the global average density... The size is set by different adjustment factors for key parameters of the subsequent DBSCAN clustering algorithm. Then, adaptive noise points are generated based on the density distribution. The number of noise points is determined by the total number of feature points and the global density.
[0044] (11)
[0045] Indicates the number of noise points; the noise generation weights are generated for each grid cell. Set as a normalization of the inverse of the mesh density:
[0046] (12)
[0047] in Used to ensure numerical stability.
[0048] Furthermore, the feature clustering in step 7, for a given matching point Its neighborhood is defined as:
[0049] (13)
[0050] express The neighborhood set of a point Represents the Euclidean distance between feature points. Characterizing the distance threshold, if ,but Marked as the core point, This represents the minimum number of feature points in this category. To improve the algorithm's robustness under high false matching rates, an iterative optimization strategy is employed. First, in the full matching set... Preliminary clustering is performed to obtain the initial interior point set. Then based on the interior set Recalculate each point - Neighborhood range, after performing a second DBSCAN clustering, the results are:
[0051] (14)
[0052] Indicates the results after clustering The neighborhood set of a point, and the stable and accurate point matching results, provide a reliable foundation of point features for subsequent line matching.
[0053] Furthermore, for each line segment, a threshold is defined, and its support region is divided into two semi-neighborhoods, left and right. Points falling within each semi-neighborhood are searched and matched, and these points are associated with the line segment, thus constructing a local point set context for each line segment. The support region of the line feature is determined by constructing a rectangle using the length of the line feature and half of the line feature length as its support region. This process calculates the perpendicular distance from points to line segments in batches based on the following distance formula:
[0054] (15)
[0055] in The coordinates of the feature point, and For line segments The slope and intercept; for vertical line segments ( ), distance simplified to ,in Let x be the x-coordinate of the vertical line segment.
[0056] The angle between the direction vectors of two line features is calculated using formulas (16)-(17), where... To adjust the parameters:
[0057] (16)
[0058] (17)
[0059] By calculating the included angle between the two line segments Assess directional consistency. Use The formula calculates geometric similarity; the smaller the angle, the higher the similarity.
[0060] Furthermore, the radiative similarity between the two line features, and the average gray value of the left side of the corresponding support domain of the matching line segment are respectively... and The average gray value on the right is and The formula for calculating radiation similarity is:
[0061] (18)
[0062] in: , , The support domain for the line feature is determined by constructing a rectangle using the length of the line feature and half of the line feature length as its support domain.
[0063] Furthermore, regarding the Normalized Correlation Value (NCC), when scaling or rotation transformations exist between images, the correspondence between local image patches changes, leading to inconsistent NCC similarity and thus affecting the reliability of line feature matching. This invention first preprocesses the images to correct for scaling or rotation transformations between images to compensate for the shortcomings of NCC under complex changes. For candidate line segment pairs, uniform sampling is performed along the line segments. Each sampling point pair is used to extract a local image patch centered on that point. and The NCC calculation formula is:
[0064] (19)
[0065] in and Image blocks and The pixel mean is then calculated. The pixel normalization value for the line segment pair is obtained by averaging the NCC values of all sampled points. ,in These represent the k-th image patch corresponding to the sampling point of the line segment matching pair:
[0066] (20)
[0067] Furthermore, when there are enough matching pairs within the support domain of the line features, the support domain is determined by constructing a rectangular support domain using the length of the line feature and half the line length. Assuming there are n such rectangular support domains, the distance ratios from these points to their respective line features are calculated, and the median is taken as the geometric invariant.
[0068] (twenty one)
[0069] in and For a pair of matching feature points, )and These represent the distances from a point to its corresponding line segment, and median(.) is the mathematical symbol for median operation.
[0070] Furthermore, the comprehensive similarity in step 10 is specifically as follows:
[0071] (twenty two)
[0072] in , , The weights for each component are dynamically generated based on the global features of the image pair.
[0073] A multi-hypothesis voting mechanism is used for the final decision; the voting matrix is initialized. Matrix elements The value represents the first value in image 2. The line segment and the first line in Figure 1 The total confidence obtained by taking line segments as correct matching pairs; for each initial candidate matching pair Score its similarity The summation is applied to the voting matrix elements corresponding to all spatially compatible matching pairs:
[0074] (twenty four)
[0075] This is the set of spatially compatible matching pairs, containing all pairs that are compatible with the current initial candidate. Other pairs of line segments that are compatible in spatial geometry .
[0076] Compared with existing technologies, the significant advantages of this invention are: This invention does not rely on the image's fundamental matrix or homography matrix, but solves the accuracy and robustness issues of image point-line feature matching solely through initial point feature descriptors. Accurate point feature matching is achieved through an initial point feature matching degree, combined with point feature distribution density and DBSCAN clustering iterations. Subsequently, based on point feature matching, a series of line feature matching condition variables are constructed, and efficient and accurate line feature matching is achieved through iterative convergence based on weighted summation. This method has strong applicability in the field of image processing pose estimation. Attached Figure Description
[0077] Figure 1 This is a schematic diagram of a specific implementation scheme of the present invention.
[0078] Figures 2-3 This is an example image of image point feature matching for the method of the present invention.
[0079] Figures 4-7 This is an example diagram of feature matching pairs based on the cascaded matching line of the present invention. Detailed Implementation
[0080] This invention proposes an adaptive weighted clustering and multi-conditional constraint point-line feature cascade matching method. The method mainly includes the following steps: acquiring optical or aerial photographic digital image sequences; using the OpenCV open-source computer vision library, employing SIFT and LSD algorithms to detect and extract point-line features from image sequence pairs, and calculating binary descriptors of image point features; comparing the similarity of point feature descriptors based on bidirectional matching of descriptor distance to obtain initial point feature matching pairs; calculating the transformation model matrix between image pairs based on the initial point feature matching pairs, and performing rotation and scaling correction preprocessing on the images; constructing feature motion quantities based on the two-dimensional coordinates of point feature images; introducing the density distribution formula for point feature image pairs, calculating the global and local density coefficients of point features; setting noise terms and adaptive matching parameters; using the DBSCAN algorithm to perform cluster analysis on point features to obtain point feature matching pairs; calculating the length, direction, and image plane line feature equation coefficients of line features based on the two-dimensional coordinates of line feature images; calculating the geometric similarity of line features (angle between line features, length, etc.); calculating the radiation similarity of line features; calculating the NCC (Normalized Cross-Correlation) of line features. The method involves calculating the NCC (Neural Coefficient of Characterization) value; constructing and calculating the geometric invariants of line features based on the distance from points to line features; and finally, integrating the multimodal line feature quantities and completing line feature matching through voting. This invention does not rely on the fundamental matrix or homography geometric constraints between image pairs. It first solves image point feature matching using only basic point feature descriptors, and then completes robust line feature matching based on the multimodal matching quantities of line features. This method has strong applicability in the field of image processing pose estimation.
[0081] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.
[0082] Combination Figure 1 An adaptive weighted clustering and multi-condition constraint-based point-line feature cascade matching method is proposed. First, initial point feature matching pairs are obtained using binary point feature descriptors. Then, point feature density distribution and adaptive parameter settings are combined to iteratively cluster and obtain point feature matches. Based on the point feature matching pair relationships and subsequent combinations of multimodal quantities of line features, robust matching of line features is achieved. The method includes the following steps:
[0083] Step 1: The camera sensor acquires image sequences from the actual scene, calls the SIFT point feature detection operator interface and LSD line feature detection interface in OpenCV to detect and extract point and line features in the image pairs, and then calls the OpenCV point feature descriptor interface to calculate and extract the binary descriptor of the point features.
[0084] Step 2: Based on the binary descriptor of point features, call the OpenCV point feature matching interface to perform initial point feature matching and obtain the initial matching point feature pair results; the point feature initial matching refers to using the binary descriptor of point features to directly obtain point feature matching pairs using the OpenCV matching interface.
[0085] Step 3: Calculate the initial transformation matrix between image pairs based on point features, and simultaneously set adaptive thresholds for feature point matching, including translation threshold, rotation threshold, and scaling threshold. When at least one of the changes is greater than the corresponding set threshold and the proportion of points within the feature is greater than the set threshold, this invention performs a rigid transformation on the image pair;
[0086] Step 4: Since correctly matched point pairs typically exhibit local motion consistency, while mismatches show random distribution characteristics, based on the initial point feature matching pairs, construct a feature vector expression that includes position and motion information:
[0087] (1)
[0088] Define the difference components: .
[0089] in These represent the two-dimensional coordinates of the feature point matching pairs in the image. This represents the motion vector of a point matching pair. Here, a weighted distance function is used to accurately measure the similarity between two point feature matching pairs.
[0090] Step 5: Analyze the spatial distribution of feature points in the image using kernel density estimation (KDE), and adaptively adjust the parameters for subsequent point feature matching based on the spatial distribution of point features. KDE is used to estimate the spatial distribution density of feature points on the image plane. Specifically, for a given set of feature points, this invention calculates its density value at each location, thereby obtaining the point feature density distribution index (CVD) across the entire image plane.
[0091] (2)
[0092] The density contribution of each feature point to the grid point is calculated, and the average value is taken to obtain the final density estimate f. ij Global average density It is the arithmetic mean of the density values of all grid points.
[0093] Step 6: After completing the density calculation of all grid points, the algorithm of this invention extracts two key density features: global average density and density variation coefficient.
[0094] Step 7: After setting the matching parameters and noise points from the above steps, perform operations based on the point features.
[0095] The dynamic vector runs the DBSCAN clustering algorithm, which clusters based on the motion of point feature matching pairs, then performs feature point matching, and finally obtains all point feature matching results.
[0096] Step 8: Parametrically represent the detected line segments in the image, including calculating geometric attributes such as the direction vector, length, slope, and intercept of the line features. For efficient processing, the system employs spatial grid partitioning technology, dividing the image into multiple grid cells and assigning line segments to grids intersecting their bounding boxes. This partitioning strategy ensures that subsequent intersection point calculations and neighborhood point searches are performed only within adjacent grids, significantly improving computational efficiency.
[0097] Step 9: Calculate geometric similarity By comparing the direction vectors of the two line segments and The included angle ,in To adjust the parameters:
[0098] (3)
[0099] (4)
[0100] By calculating the included angle between the two line segments And based on formula (4), the directional consistency of line feature matching pairs is evaluated. The larger the angle, the smaller the angle, and the higher the similarity of the line features to be matched.
[0101] Step 10: Calculate the radiation similarity between line features. This invention calculates gray-level statistical features in the neighborhood of a line feature by comparing the gray-level statistical features on both sides of the line segment. It also calculates the gray-level statistical features based on the neighboring pixel values of the line feature. The average gray-level value on the left and the average gray-level value on the right of the reference and target line segments are defined, and the radiation similarity is calculated. Line feature matching pairs are determined by comparing the gray-level value changes on both sides of the same position in the reference and target images.
[0102] and These are the average gray values on the left side of the reference line segment and the target line segment, respectively. and The average gray value to the right of the reference line segment and the target line segment is then used. The radiation similarity is calculated based on formula (5):
[0103] (5)
[0104] in: , ,
[0105] Step 11: Calculate the normalized cross-correlation (NCC) similarity within the neighborhood of the line feature. This is used to enhance the matching reliability of texture-rich regions and further improve the accuracy of line feature matching. Here, the normalized cross-correlation similarity calculation formula is:
[0106] (6)
[0107] in and Image blocks and The mean value, where the subscripts i,j represent the pixel indices of the image block.
[0108] Step 12: Calculate the invariants of line feature matching within the support domain of the local point set to further enhance matching reliability: For a pair of correctly matched line segments and the corresponding matching point pair, the distance ratio remains approximately unchanged under most transformations. The calculation formula is as follows:
[0109] (7)
[0110] in and For a pair of matching feature points, )and These represent the distances from a point to its corresponding line segment, and median(.) is the mathematical symbol for median operation.
[0111] Step 13: The comprehensive similarity of the line segment pairs is calculated by weighting the above four components. Based on this, the system adopts a multi-hypothesis voting mechanism to make the final decision and obtain the final line feature matching pairs. The formula for the weighted calculation of the four components in this invention is (8):
[0112] (8)
[0113] in , , The weights for each component are dynamically generated based on the global features of the image pair. For example, it consists of three sub-factors: point matching quality factor (The higher the quality of the point matching, the better) (The larger the value), the better the line length factor. (The longer the line segment, the better) (The larger the value), the better the gradient consistency factor. (The stronger the gradient consistency of the line segment, the better) The larger the value).
[0114] The voting mechanism here is achieved by initializing the voting matrix. Matrix elements The value represents the first value in image 2. The line segment and the first line in Figure 1 The line segments are the total confidence obtained as correct matching pairs. For each initial candidate matching pair Score its similarity The summation is applied to the voting matrix elements corresponding to all spatially compatible matching pairs:
[0115] (9)
[0116] This is the set of spatially compatible matching pairs, containing all pairs that are compatible with the current initial candidate. Other line segment pairs that are compatible (i.e. not contradictory) in spatial geometry Compatibility is usually determined based on whether the implicit local geometric transformations (such as scaling and rotation) of the two matching pairs are consistent.
[0117] Example
[0118] The following description, in conjunction with the accompanying drawings, briefly introduces the specific implementation process of the present invention:
[0119] (1) In this embodiment, challenging image pairs (outdoor scene + viewpoint change, occlusion) are selected, point features are detected and extracted respectively, and corresponding point feature descriptors are calculated. Point feature matching is performed based on the Hamming distance of the feature descriptors, and initial matching pairs of line features are obtained.
[0120] (2) Select initial point feature matching pairs, calculate image point feature motion vectors according to feature endpoint coordinates, set global density parameters and local point feature density parameters according to point feature density distribution, add noise terms, and realize point feature matching based on DBSCAN clustering;
[0121] (3) Calculate the line feature direction, length, two-dimensional plane equation and other parameter variables based on the line feature detection results, establish the line feature neighborhood, and facilitate the calculation of subsequent multi-condition constraint matching parameters (geometric invariants, radiation, normalized NCC value, etc.) of the line feature.
[0122] (4) Based on the calculated line feature radiance, geometric invariants, normalized NCC value, and combined with the point feature matching accuracy and line feature length, the weights of the above matching conditions are adaptively set respectively.
[0123] (5) Establish a line feature matching voting mechanism and obtain the final line feature matching pairs based on the adaptive line feature weights and the final total similarity score; Figure 2 , Figure 3The number of point feature matches in the two images from different scenes were 91 and 115 respectively, with matching accuracies of 0.99 and 0.98. (Corresponding...) Figures 4-7 The number of line feature matches between the two images in the scene is 64 and 136 pairs, respectively. The accuracy rates of the above line feature matching pairs obtained by this invention are 0.99 and 1.0, respectively.
[0124] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Various modifications, substitutions, and improvements made by those skilled in the art to the technical solutions of the present invention based on the provided textual description and drawings, without departing from the design concept and spirit of the present invention, should all fall within the scope of protection of the present invention.
Claims
1. A point-line feature concatenation matching method based on adaptive weighted clustering and multimodal constraints, characterized in that, Includes the following steps: Step 1: The camera sensor acquires a sequence of color images in the scene, and calls the SIFT point feature detection operator interface and LSD line feature detection interface in OpenCV to extract point and line features from the image pairs; then, the OpenCV point feature descriptor interface is called to calculate and extract the binary descriptors of the point features. Step 2: Perform initial point feature matching based on the binary descriptor of the point features to obtain the initial point feature matching pair results; estimate the similarity transformation variables between the two views based on the initial point feature matching pairs: translation, rotation and scaling factors, and combine them with the preset point matching pair in-point ratio threshold to comprehensively determine whether the matching pair transformation operation needs to be performed. Step 3: Set the threshold values for adaptive variables related to feature point matching, including translation threshold, rotation threshold, and scaling threshold; When at least one of the adaptive variable thresholds changes more than the corresponding preset threshold, and the proportion of points within the point feature matching pair mentioned in step 2 is greater than the previously set threshold, the system determines that pre-transformation processing is required. Step 4: Based on the initial point feature matching pairs, construct a feature vector expression that includes position and motion information; Step 5: Analyze the spatial distribution of feature points on the image by kernel density estimation, and adaptively adjust the subsequent point feature matching parameters based on the spatial distribution of point features. Estimate the spatial distribution density of feature points on the image by using KDE. Step 6: After completing the density calculation for all grid points, extract and calculate two key density features: global average density and density variation coefficient; Step 7: Run the DBSCAN clustering algorithm to cluster points based on the motion of feature matching pairs and execute the feature point matching iterative algorithm. Step 8: Parameterize the image line features, including calculating the direction vector, length, slope, and intercept of the line features; use spatial grid partitioning technology to divide the image into multiple grid cells and assign line segments to the grid cells that intersect with their bounding boxes; Step 9: Calculate the geometric similarity between any two line features; Calculate the radiation similarity between line features, define the average gray value on the left and the average gray value on the right of the reference line segment and the target line segment; evaluate the illumination consistency by comparing the gray value changes on both sides of the same position in the reference image and the target image, and determine the line feature matching pairs; Calculate the normalized cross-correlation similarity between line features; Compute the geometric invariants within the support domain of the local point set; Step 10: The comprehensive similarity of line segment pairs is calculated by weighting geometric similarity, radial similarity, normalized cross-correlation similarity, and geometric invariants. A multi-hypothesis voting mechanism is used for the final decision to obtain the final line feature matching pair.
2. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 1, characterized in that, The rigid transformation model for the image pair is: （1） Where T is the transformation matrix (t x ,t y ) is the translation vector, θ is the rotation angle, x and y represent the two-dimensional coordinates of the feature points on the image, and s is the scaling factor.
3. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 1, characterized in that, Each initial point matching is represented as a feature vector containing position and motion information, uniformly characterized as: （2） Define the difference components: ; Accurately measure the similarity between two matches based on a weighted distance function: （3） in Indicates taking a vector The first two components, that is The part; where the weighting coefficients are: （4） To adjust the parameters, the subscripts i and j represent the indices of the corresponding point feature matching pairs. , These are the two-dimensional coordinates of the feature point image; this distance function comprehensively considers spatial location differences and motion vector differences, and emphasizes the motion consistency of spatially neighboring point pairs through adaptive weights; in the distance calculation, the spatial location differences are measured using Euclidean distance, while the motion vector differences are balanced by adjusting parameters.
4. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 1, characterized in that, KDE is used to estimate the spatial distribution density of feature points on the image plane. First, equally spaced grid points are generated in the width and height directions of the image to cover the entire image. After the image grid is constructed, the bandwidth parameter h for kernel density estimation is calculated, and its calculation formula is as follows: （5） in The number of feature points, The standard deviation estimate of the feature point coordinates is calculated. and The mean value of the directional standard deviation is: （6） Based on the defined grid and bandwidth parameters, calculate the value for each grid point. The density value; for any grid point, first calculate the Euclidean distance from all feature points to that point: （7） x k For the first For each feature point, the density contribution of that feature point to the grid point is calculated using a Gaussian kernel function, and the average value is taken to obtain the final density estimate f. ij : （8）。 5. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 4, characterized in that, Global average density It is the arithmetic mean of the density values of all grid points: （9） The coefficient of variation (CVD) is the ratio of the standard deviation to the mean of the grid point density, used to characterize the uniformity of the characteristic distribution. （10） Based on the extracted density features, an adaptive parameter adjustment mechanism is established; based on the global average density... The size is set by different adjustment factors for key parameters of the subsequent DBSCAN clustering algorithm. Then, adaptive noise points are generated based on the density distribution. The number of noise points is determined by the total number of feature points and the global density. （11） Indicates the number of noise points; the noise generation weights are generated for each grid cell. Set as a normalization of the inverse of the mesh density: （12） in Used to ensure numerical stability.
6. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 1, characterized in that, The feature clustering in step 7, for a given matching point Define the set of points in its neighborhood as The specific expression is as follows: （13） Represents the Euclidean distance between feature points. Characterizing the distance threshold, if ,but Marked as the core point, This represents the minimum number of feature points in that category. An iterative optimization strategy is adopted, starting with the initial matching set. Preliminary clustering is performed to obtain the initial interior point set. Then based on the interior set Recalculate each point - Neighborhood range, perform a second DBSCAN clustering, where After clustering, any point Neighborhood set: （14）。 7. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 1, characterized in that, For each line segment, a threshold is defined and its supported region is divided into two semi-neighborhoods, left and right. By searching for point matching pairs of points that fall into each semi-neighborhood and associating these points with the line segment, a local point set context is constructed for each line segment. Calculate the perpendicular distance from a point to a line segment in batches using the following distance formula: （15） in The coordinates of the feature point, and For line segments The slope and intercept; for a perpendicular line segment, the distance simplifies to... ,in Let x be the x-coordinate of the vertical line segment; The angle between the direction vectors of the line feature pairs is calculated in formulas (16)-(17). To adjust the parameters: （16） （17） By calculating the included angle between the two line segments Assess consistency in direction; use The formula calculates geometric similarity; the smaller the angle, the higher the similarity.
8. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 7, characterized in that, The radiative similarity between two line features, and the average gray value of the left side of the corresponding support domain of the matched line segments are respectively and The average gray value on the right is and The formula for calculating radiation similarity is: （18） in: , , The line feature support domain is constructed by using a rectangle formed by the length of the line feature and half the length of the line feature as its support domain. The normalized cross-correlation similarity method first preprocesses the images to correct for scaling or rotation variations; for candidate line segment pairs, it samples uniformly along the line segments. For each pair of sampling points, extract the local image patch centered on it. and The NCC similarity calculation formula is: （19） in and Image blocks and The pixel mean is calculated, where i and j are the indices of pixels within the image patch; then, the pixel normalized value for the line segment pair is obtained by averaging the NCC values of all sampling points. , in These represent the k-th image patch corresponding to the sampling point of the line segment matching pair: （20）。 9. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 8, characterized in that, When there are enough matching pairs within the support domain of the line features (assuming there are n pairs), first calculate the distance ratio from these points to their respective line features, and take the median as the geometric invariant: （21） in and For a pair of matching feature points, )and These represent the distances from a point to its corresponding line segment, and median(.) is the mathematical symbol for median operation.
10. The adaptive weighted clustering and multimodal constraint point-line feature concatenation matching method according to claim 9, characterized in that, The comprehensive similarity in step 10 is as follows: （22） in , , The weights for each component are dynamically generated based on the global features of the image pair. A multi-hypothesis voting mechanism is used for the final decision; the voting matrix is initialized. Matrix elements The second image represents the first one. The line segment and the first line in Figure 1 The total confidence obtained by taking line segments as correct matching pairs; for each initial candidate matching pair Score its similarity The summation is applied to the voting matrix elements corresponding to all spatially compatible matching pairs: （24） This is the set of spatially compatible matching pairs, containing all pairs that are compatible with the current initial candidate. Other pairs of line segments that are compatible in spatial geometry .