A visual localization method based on Hungarian matching

Abstract

This invention provides a visual positioning method based on Hungarian matching, belonging to the field of visual positioning technology. It solves the technical problems of traditional visual positioning, such as the Hungarian matching cost matrix having a single consideration dimension, low accuracy, poor noise resistance, and insufficient accuracy and efficiency due to the lack of a fine matching step. This method takes BEV visual perception results and high-precision map results as input, extracts distance, shape, and type-related feature information of lane lines and lane elements, defines distance, shape, and type similarity and corresponding weights, calculates the total similarity through linear weighting, and uses the difference between 1 and the total similarity as the value of the Hungarian matching cost matrix to complete the cost matrix creation. Based on this cost matrix, Hungarian association matching of lane lines and lane elements is achieved, and then ICP fine matching is performed using the point cloud information of the matched pairs to output high-precision positioning results. This invention optimizes the creation logic of the Hungarian matching cost matrix.

Description

Technical Field

[0001] This invention belongs to the field of visual positioning technology, and more specifically, relates to a visual positioning method based on Hungarian matching. Background Technology

[0002] As one of the core technologies in the aforementioned fields, visual positioning directly determines the overall performance of the system through its positioning accuracy, matching efficiency, and noise resistance. Currently, existing visual positioning technologies are mainly divided into four categories, each with varying degrees of defects and shortcomings, making it difficult to simultaneously meet the requirements of high-precision, high-efficiency, and high-robust visual positioning.

[0003] Semantic SLAM technology introduces semantic information on the basis of traditional SLAM to distinguish between static and dynamic objects. Although it improves the robustness of localization and the ability to understand the semantic meaning of the map, this technology consumes a lot of computing resources, and the errors generated during the semantic segmentation process will continue to propagate, ultimately affecting the localization accuracy.

[0004] Multi-sensor fusion technology integrates data from multiple sensors such as cameras, LiDAR, IMU, and GNSS, leveraging the advantages of each sensor to compensate for the shortcomings of a single sensor. However, this technology has high hardware costs, complex calibration processes between sensors, and the fusion of multi-sensor data significantly increases the overall complexity of the system, making troubleshooting and maintenance more difficult.

[0005] Multi-target tracking and data association technology is designed to solve the problem of continuous tracking of multiple moving targets in a video sequence. It achieves correct matching of detection boxes in different frames through data association. However, its core technology focuses only on moving target tracking and does not optimize for matching static road features such as lane lines and lane elements. Therefore, it cannot be directly applied to high-precision visual positioning scenarios.

[0006] End-to-end visual navigation technology relies on deep learning models to directly output navigation actions based on visual input and language commands, simplifying the traditional visual positioning process. However, the model interpretability and reliability of this technology are poor, its generalization ability is insufficient, and model training requires massive amounts of labeled data, resulting in extremely high data acquisition and labeling costs.

[0007] Furthermore, existing visual positioning technologies that use the Hungarian algorithm for lane line and lane element association matching generally suffer from a single-dimensional consideration in cost matrix creation. Relying solely on a single feature information to construct the cost matrix leads to low accuracy and susceptibility to environmental noise interference. Moreover, most schemes lack an effective fine-matching step after association matching, ultimately resulting in insufficient matching accuracy and low matching efficiency in visual positioning, failing to meet the actual needs of high-precision visual positioning in scenarios such as autonomous driving. To address these technical shortcomings, there is an urgent need for an optimized Hungarian matching cost matrix creation method that combines a fine-matching step to achieve high-precision and high-efficiency visual positioning. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a visual positioning method based on Hungarian matching. This method solves the technical problems in existing visual positioning methods where the Hungarian algorithm is used to correlate and match lane lines and lane elements. These problems include a single dimension in the cost matrix creation process, low accuracy, susceptibility to noise, and a lack of fine matching, resulting in insufficient positioning accuracy and low matching efficiency.

[0009] A visual localization method based on Hungarian matching, applied to visual localization scenarios, takes BEV visual perception results and high-precision map results as input, and includes the following steps:

[0010] Extract the point cloud, type, and shape information of lane lines and lane elements from the BEV visual perception results, as well as the corresponding information of point cloud, type, and shape of lane lines and lane elements from the high-precision map results;

[0011] Define distance similarity (distance_sim), shape similarity (shape_sim), and type similarity (type_sim), as well as the corresponding distance similarity weight (distance_weight), shape similarity weight (shape_weight), and type similarity weight (type_weight);

[0012] Calculate the distance_sim, shape_sim, and type_sim between the corresponding lane lines and lane elements in the BEV visual perception results and the high-precision map results, respectively;

[0013] According to the formula:

[0014] total_sim = distance_weight * distance_sim + shape_weight * shape_sim + type_weight * type_sim calculates the total similarity of lane lines and lane elements;

[0015] Use 1-total_sim as the value of the cost matrix for Hungarian matching to complete the creation of the Hungarian matching cost matrix;

[0016] Based on the created cost matrix, Hungarian association matching is performed on lane lines and lane elements in the BEV visual perception results and high-precision map results to obtain matching pairs of all lane lines and lane elements;

[0017] Based on the point cloud information in the matching pair, ICP fine matching is performed to output a high-precision positioning result for visual positioning.

[0018] Preferably, the distance_sim is the similarity between the center point distance of the corresponding lane line and lane element in the BEV visual perception result and the high-precision map result.

[0019] Preferably, the shape_sim is the shape similarity (IOU) value between the corresponding lane lines and lane elements in the BEV visual perception result and the high-precision map result.

[0020] Preferably, the type_sim is the similarity of the type attributes of the corresponding lane lines and lane elements in the BEV visual perception results and the high-precision map results, and the type attributes include the dashed and solid line types of the lane lines.

[0021] Preferably, the information extracted in step 1) also includes the color information of lane lines and lane elements in the BEV visual perception results and high-precision map results, and the color information is incorporated into the calculation process of the corresponding similarity.

[0022] Preferably, the distance_weight, shape_weight, and type_weight are all non-zero positive weights, and the values ​​of each weight can be adjusted according to the actual application scenario of visual positioning.

[0023] Preferably, the matching objects of the Hungarian association matching in step 6) are all lane lines and lane elements in the BEV visual perception results and high-precision map results, with no missing matching targets.

[0024] Preferably, before performing ICP fine matching in step 7), noise removal is first performed on the point cloud information of the matching pair, and ICP fine matching is performed only on the valid point cloud information after noise removal.

[0025] Preferably, the method reduces the impact of environmental noise on Hungarian matching by incorporating multi-dimensional information of lane lines and lane elements into a cost matrix according to weights, thereby improving the matching accuracy of visual positioning by 0.5cm.

[0026] Preferably, the method performs association matching through an optimized Hungarian matching cost matrix, which saves 50ms of matching time compared to traditional visual positioning matching methods.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] This invention adds an ICP fine matching step to the Hungarian association matching, and provides an accurate matching pair basis for fine matching through multi-dimensional feature-optimized association matching, ultimately improving the matching accuracy by 0.5cm, which meets the core requirements of high-precision visual positioning in scenarios such as autonomous driving.

[0029] Significantly improve matching efficiency: This invention optimizes the creation of the cost matrix by using a multi-dimensional feature weighting method, making the Hungarian algorithm more targeted in matching and greatly reducing invalid matching calculations. Compared with traditional methods, it saves 50ms in matching efficiency and improves the real-time performance of visual positioning.

[0030] Improving the accuracy of the cost matrix: This invention incorporates multi-dimensional information such as lane lines, lane element distance, shape, and type into the creation process of the Hungarian cost matrix with different weights, breaking through the limitations of the traditional cost matrix's single-dimensional consideration, and more realistically reflecting the comprehensive matching degree of the object to be matched, effectively increasing the accuracy of the cost matrix.

[0031] Enhanced noise resistance of matching results: The weighted fusion of multi-dimensional features makes the creation of the cost matrix no longer dependent on single feature information. It can effectively offset the interference of environmental noise such as light and shadow, road stains, and obstacles on single features, reduce the impact of noise on matching results, and improve the robustness of visual localization under complex road conditions.

[0032] The solution boasts high adaptability and ease of implementation: the cost matrix weight parameters of this invention can be flexibly adjusted according to actual application scenarios (such as urban roads, highways, severe weather, etc.) to adapt to different visual positioning needs; at the same time, the technical solution takes BEV visual perception results and high-precision map results as inputs, the data interface is universal, there is no need to make major modifications to the existing visual perception and high-precision map systems, and it is easy to deploy and promote in the existing visual positioning system. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall structure of the present invention. Detailed Implementation

[0034] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0035] This invention relates to the field of visual positioning technology, specifically to a visual positioning method based on Hungarian matching. It is applicable to visual positioning scenarios based on BEV (bird's-eye view) visual perception and high-precision map fusion, aiming to solve the technical problems of traditional visual positioning, such as the Hungarian matching cost matrix having a single consideration dimension, low matching accuracy, poor noise resistance, and insufficient matching efficiency. This embodiment details the technical implementation process of the invention. The embodiments described are only for explaining the invention and do not constitute a limitation on the scope of protection of the invention. Those skilled in the art can make parameter adaptations and detail adjustments based on actual application scenarios based on this embodiment, all of which fall within the protection scope of this invention.

[0036] The core technical concept of this invention is as follows: using BEV visual perception results and high-precision map results as dual inputs, multi-dimensional feature information of lane lines and lane elements is extracted. The creation logic of the Hungarian matching cost matrix is ​​optimized by fusing distance similarity, shape similarity, and type similarity and assigning different weights. Then, the Hungarian algorithm is used to complete the association matching of lane lines and lane elements. Finally, ICP (Iterative Closest Point) fine matching is performed based on the point cloud information of the matching pairs to achieve high-precision and high-efficiency visual positioning. Among them, color information is used as the initial feature screening basis, which can reduce invalid matching calculations and further improve matching efficiency and noise resistance.

[0037] 1. Preparation of visual positioning input data:

[0038] The visual positioning system of this invention takes two types of data as input: BEV visual perception results and high-precision map results. Both types of data contain complete feature information of lane lines and lane elements in the road scene, wherein:

[0039] BEV visual perception results are obtained by collecting road images from onboard visual sensors and then converting them into bird's-eye view and extracting features. The results include information such as point cloud coordinates, type (dashed and solid lines, single / double yellow lines, guide lines, etc.), shape and outline, color (yellow, white, red, etc.), and spatial location of lane lines / lane elements.

[0040] The high-precision map result is pre-stored high-precision electronic map data of roads, including standardized information such as standard point cloud coordinates, calibration type, standard shape outline, calibration color, and road topology of lane lines / lane elements.

[0041] The feature information of the two types of input data is uniformly formatted to ensure that the lane lines and lane elements to be matched correspond one-to-one in the feature dimension, laying the foundation for subsequent similarity calculation and matching.

[0042] 2. Initial screening and extraction of lane line / lane element features:

[0043] Before performing similarity calculation, the lane lines / lane elements in the BEV visual perception results and high-precision map results are initially screened using color information: candidate pairs of lane lines / lane elements with completely different colors in the two types of data are removed, and only candidate pairs with the same or highly similar colors are retained for subsequent similarity calculation.

[0044] For the candidate pairs after initial screening, extract the core feature information respectively:

[0045] Point cloud information: a set of three-dimensional spatial coordinates of lane lines / lane elements, used to calculate center point coordinates, Euclidean distance, and subsequent ICP fine matching;

[0046] Type information: The labeling type of lane lines / lane elements, such as solid lines, dashed lines, double yellow lines, white guide lines, etc., used to calculate type similarity;

[0047] Shape information: the outline, direction, and geometry of lane lines / lane elements, used to calculate the shape similarity (IOU) value;

[0048] Spatial location information: The relative position of lane lines / lane elements in the road scene, which helps to verify the rationality of the matching.

[0049] 3. Creation of the Hungarian matching cost matrix:

[0050] The core improvement of this invention lies in the creation logic of the cost matrix in the Hungarian algorithm. The creation of the optimized cost matrix is ​​achieved through four steps: defining multi-dimensional similarity parameters, setting weight coefficients, calculating the total similarity, and assigning the cost matrix. Specifically:

[0051] 3.1 Definition and Calculation of Similarity Parameters:

[0052] For the lane line / lane element candidate pairs after initial screening, three core parameters are calculated: distance similarity (distance_sim), shape similarity (shape_sim), and type similarity (type_sim). All parameters are normalized and range from 0 to 1. The closer the parameter value is to 1, the higher the similarity, and the closer it is to 0, the lower the similarity.

[0053] Distance similarity (distance_sim) is calculated from the Euclidean distance between the center points of lane lines / lane elements. The calculation steps are as follows:

[0054] ① Calculate the center point coordinates P1(x1,y1,z1) and P2(x2,y2,z2) of the point cloud coordinates of the lane line / lane element to be matched in the BEV visual perception results and high-precision map results respectively;

[0055] ② Calculate the Euclidean distance d between the two points. ;

[0056] ③ Normalize the Euclidean distance d to obtain distance_sim. The normalization formula is:

[0057] ,in The minimum Euclidean distance between the candidate pairs to be matched is given. This represents the maximum Euclidean distance between the candidate pairs to be matched. The smaller the Euclidean distance, the closer distance_sim is to 1, indicating a higher distance similarity between the candidate pairs.

[0058] Shape similarity (shape_sim): shape_sim is the Intersection over Union (IOU) value of the lane lines / lane elements. IOU (Intersection over Union) is the ratio of the intersection area to the union area of ​​the shape contours of the pairs to be matched. The calculation formula is:

[0059] .

[0060] The closer the shapes of the pairs to be matched are, the larger the intersection area, and the closer shape_sim is to 1, the higher the shape similarity; if the shapes do not overlap at all, shape_sim=0.

[0061] Type similarity (type_sim) is assigned a value based on the type consistency of lane lines / lane elements, according to the following rules:

[0062] ① If the types of the pairs to be matched are completely identical (such as solid lines and solid lines, double yellow lines and double yellow lines), then typesim=1;

[0063] ② If the types of the pairs to be matched are similar (this is not further subdivided in this embodiment, but can be expanded according to the actual scenario), then 0 <typesim<1;

[0064] ③ If the types of the pairs to be matched are completely different (e.g., solid lines and dashed lines, yellow lines and white lines), then typesim=0. Type similarity is a hard constraint feature that can effectively eliminate invalid matching pairs with mismatched types.

[0065] 3.2 Weight Parameter Settings:

[0066] Define weight coefficients corresponding to the three similarity parameters: distance similarity weight (distance_weight), shape similarity weight (shape_weight), and type similarity weight (type_weight). All weight coefficients are non-zero positive real numbers and can be dynamically adjusted according to the actual visual positioning scenario (such as urban roads, highways, rural roads, or adverse environments such as rainy days, foggy days, and nighttime). They are used to characterize the importance ratio of each dimension of similarity in the total similarity.

[0067] In the basic embodiment of the present invention, the default weight value is set as follows: =0.3、 =0.4、 =0.3 (weights sum to 1 for easy calculation and understanding); in scenarios with more stable shape features (such as highways), shape_weight can be increased; in scenarios where visual perception is disturbed (such as rainy days), type_weight can be increased and distance_weight decreased to improve the noise resistance of matching.

[0068] 3.3 Calculation of total similarity:

[0069] Based on the preset weight coefficients and the calculated multi-dimensional similarity parameters, the total similarity (total_sim) of the pairs to be matched is calculated using a linear weighted formula, as follows:

[0070] ;

[0071] The value of total_sim ranges from 0 to 1. The closer the value is to 1, the higher the overall matching degree of the pair to be matched, and the more suitable it is as the optimal matching pair for Hungarian matching.

[0072] 3.4 Assigning values ​​to the cost matrix:

[0073] The Hungarian algorithm's matching logic is minimum cost matching. Therefore, it uses the 1-total_sim of the pair to be matched as the element value at the corresponding position in the cost matrix to complete the creation of the cost matrix. ;

[0074] Where Cost(i,j) is the value of the element in the i-th row and j-th column of the cost matrix, i corresponds to the lane line / lane element in the BEV visual perception result, and j corresponds to the lane line / lane element in the high-precision map result; the closer the Cost(i,j) value is to 0, the higher the comprehensive matching degree between the i-th BEV perception target and the j-th high-precision map target, and the smaller the matching cost.

[0075] The dimension of the cost matrix is ​​the same as the number of lane lines / lane elements to be matched, ensuring that all candidate pairs are covered and no target is missed.

[0076] 4. Hungarian Algorithm for Association Matching:

[0077] Input the optimized cost matrix created in the above steps into the Hungarian algorithm to perform multi-objective association matching of lane lines / lane elements: The Hungarian algorithm is based on the principle of minimum cost matching. It traverses all element values ​​in the cost matrix, solves for the matching combination with the minimum total cost, and outputs the optimal matching pairs of all lane lines and lane elements, ensuring that the matching results are without duplication or omission and have the highest overall matching degree.

[0078] If a lane line / lane element has no valid matching pair (e.g., Cost(i,j) are all close to 1), it is marked as unmatched and will not participate in subsequent ICP fine matching.

[0079] 5. Output of ICP fine matching and high-precision positioning results:

[0080] Based on the lane line / lane element matching pairs output by the Hungarian algorithm, the point cloud information of each matching target is extracted, and ICP (Iterative Nearest Point) fine matching is performed on each pair of point clouds. The specific steps are as follows:

[0081] The point cloud in the high-precision map result is used as the target point cloud, and the point cloud in the BEV visual perception result is used as the source point cloud.

[0082] The optimal transformation matrix (translation matrix, rotation matrix) from the source point cloud to the target point cloud is solved by iterative calculation to minimize the Euclidean distance error between the source point cloud and the target point cloud.

[0083] The positional deviation of the BEV visual perception point cloud is corrected based on the optimal transformation matrix, and the corrected lane line / lane element point cloud coordinates are obtained.

[0084] Based on the point cloud coordinates after ICP fine matching, combined with the road topology, a high-precision visual positioning result is finally output, completing the entire visual positioning process.

[0085] 6. Scene adaptation and parameter optimization expansion:

[0086] The cost matrix creation method of the present invention has high scene adaptability. In addition to the basic embodiment described above, the following extensions and optimizations can be made according to actual application needs:

[0087] Dynamic weight adjustment: By using machine learning or offline calibration, a mapping relationship between different scenarios and weight coefficients is established to achieve adaptive adjustment of weights during the positioning process;

[0088] Similarity dimension expansion: Features such as the length, width, and curvature of lane lines / lane elements can be included in the similarity calculation to further improve the accuracy of the overall similarity.

[0089] Color similarity fusion: Upgrades color information from initial screening features to similarity calculation features, increases color similarity and corresponding weights, and further reduces the impact of environmental noise;

[0090] Post-match validation: Perform post-match validation on the results of Hungarian matching to remove matching pairs with low overall matching scores, thereby improving the effectiveness of ICP fine matching.

[0091] 7. Verification of the technical effects of the present invention:

[0092] In a real-world visual positioning test scenario on a main urban road, the optimized cost matrix of this invention was compared with the traditional Hungarian matching cost matrix (considering only the distance dimension). The experimental results are as follows:

[0093] Matching accuracy: This invention adds ICP fine matching to the Hungarian association matching, which improves the matching accuracy by 0.5cm compared with the traditional method, meeting the requirements of high-precision visual positioning;

[0094] Matching efficiency: By reducing invalid calculations through color screening, the optimized cost matrix reduces the number of matching iterations in the Hungarian algorithm, saving 50ms of the overall matching time compared to traditional methods;

[0095] Noise resistance: By incorporating multi-dimensional features into the cost matrix according to weights, the impact of environmental noise such as light and shadow, obstacles, and road stains on the matching results is effectively reduced, and the robustness of matching under complex road conditions is greatly improved.

[0096] Cost matrix accuracy: The fusion of multi-dimensional features enables the cost matrix to more accurately reflect the overall matching degree of the pairs to be matched, reducing the proportion of invalid pairs by more than 80%.

[0097] In summary, this invention optimizes the creation logic of the Hungarian matching cost matrix, integrates multi-dimensional features of lane lines / lane elements and assigns dynamic weights, and combines ICP fine matching to achieve high precision, high efficiency and high robustness of visual positioning. It solves the technical problems of low matching accuracy, poor noise resistance and insufficient efficiency in traditional visual positioning methods. Moreover, the implementation logic is simple and easy to deploy, making it suitable for widespread application in vehicle-mounted visual positioning systems.

[0098] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A visual localization method based on Hungarian matching, characterized in that, The method is applied to visual positioning scenarios, taking BEV visual perception results and high-precision map results as inputs, and includes the following steps: 1) Extract the point cloud, type, and shape information of lane lines and lane elements from the BEV visual perception results, as well as the corresponding information of point cloud, type, and shape of lane lines and lane elements from the high-precision map results; 2) Define distance similarity (distance_sim), shape similarity (shape_sim), and type similarity (type_sim), as well as the corresponding distance similarity weight (distance_weight), shape similarity weight (shape_weight), and type similarity weight (type_weight); 3) Calculate the distance_sim, shape_sim, and type_sim between the corresponding lane lines and lane elements in the BEV visual perception results and the high-precision map results, respectively; 4) According to the formula: total_sim = distance_weight * distance_sim + shape_weight * shape_sim + type_weight * type_sim calculates the total similarity of lane lines and lane elements; 5) Use 1-total_sim as the value of the cost matrix for Hungarian matching to complete the creation of the Hungarian matching cost matrix; 6) Based on the created cost matrix, perform Hungarian association matching on lane lines and lane elements in the BEV visual perception results and high-precision map results to obtain matching pairs of all lane lines and lane elements; 7) Perform ICP fine matching based on the point cloud information in the matching pair, and output high-precision positioning results for visual positioning.

2. The method according to claim 1, characterized in that, The distance_sim is the similarity between the center point distance of the corresponding lane lines and lane elements in the BEV visual perception results and the high-precision map results.

3. The method according to claim 1 or 2, characterized in that, The shape_sim is the IOU value of the shape similarity between the corresponding lane lines and lane elements in the BEV visual perception results and the high-precision map results.

4. The method according to any one of claims 1-3, characterized in that, The type_sim is the similarity of the type attributes of the corresponding lane lines and lane elements in the BEV visual perception results and the high-precision map results. The type attributes include the dashed and solid line types of the lane lines.

5. The method according to any one of claims 1-4, characterized in that, The information extracted in step 1) also includes the color information of lane lines and lane elements in the BEV visual perception results and high-precision map results, and the color information is incorporated into the calculation process of the corresponding similarity.

6. The method according to any one of claims 1-5, characterized in that, The distance_weight, shape_weight, and type_weight are all non-zero positive weights, and the values ​​of each weight can be adjusted according to the actual application scenario of visual positioning.

7. The method according to any one of claims 1-6, characterized in that, In step 6), the Hungarian association matching targets are all lane lines and lane elements in the BEV visual perception results and high-precision map results, with no missing matching targets.

8. The method according to any one of claims 1-7, characterized in that, Before performing ICP fine matching in step 7), noise removal is first performed on the point cloud information of the matching pair, and ICP fine matching is performed only on the valid point cloud information after noise removal.

9. The method according to any one of claims 1-8, characterized in that, The method reduces the impact of environmental noise on Hungarian matching by incorporating multi-dimensional information of lane lines and lane elements into a cost matrix according to weights, thereby improving the matching accuracy of visual positioning by 0.5cm.

10. The method according to any one of claims 1-9, characterized in that, The method performs association matching using an optimized Hungarian matching cost matrix, saving 50ms of matching time compared to traditional visual positioning matching methods.

Images