Multi-source visual feature fused sparse texture scene positioning method and system
By fusing multi-source visual features and multiple constraint verification methods, the visual localization problem in sparse texture scenes is solved, realizing a high-precision, low-cost localization solution suitable for complex scenarios such as underground parking lots.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-25
- Publication Date
- 2026-03-31
AI Technical Summary
In closed or semi-closed scenarios, existing visual-based localization methods suffer from inaccurate localization and poor robustness due to sparse textures and highly repetitive environments. In particular, in scenarios such as underground parking lots, traditional local feature extraction and matching are difficult, resulting in incomplete maps and localization failures.
By employing a method that integrates multi-source visual features, combining traditional local feature extraction, optical flow tracking, and deep learning feature matchers, a high-precision map is constructed using a binocular camera. Furthermore, a multi-verification mechanism based on topological relationships and physical height constraints is introduced to improve the robustness and accuracy of localization.
It significantly improves the adaptability, accuracy, and robustness of positioning in sparse texture scenarios, reduces hardware costs, and ensures the continuity and reliability of positioning trajectories, achieving high-precision positioning results.
Smart Images
Figure CN121761868A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of intelligent driving and computer vision technology. Specifically, it relates to a method and system for achieving high-precision and robust positioning in closed scenarios (such as underground parking lots) where Global Navigation Satellite System (GNSS) signals are missing. This is achieved by fusing multiple visual feature sources. Background Technology
[0002] With the rapid development of intelligent vehicle technology, high-precision positioning has become the cornerstone for realizing advanced autonomous driving functions such as automatic parking and autonomous navigation. In open outdoor environments, centimeter-level high-precision positioning can usually be achieved using real-time dynamic (RTK) differential GNSS technology. However, in enclosed or semi-enclosed scenarios such as underground parking lots, tunnels, and large indoor stadiums, GNSS signals may be completely or partially lost, preventing vehicles from obtaining accurate location information.
[0003] To address the positioning challenges in enclosed environments, the industry has proposed several alternative solutions. For example, radio positioning schemes based on ultra-wideband (UWB) or Wi-Fi, but these methods typically offer low positioning accuracy (meter-level) and require large-scale deployment of base stations and other hardware in advance, resulting in high costs and complex maintenance. Simultaneous localization and mapping (SLAM) schemes based on LiDAR can provide higher accuracy, but the high cost of LiDAR sensors limits their widespread application in consumer vehicles.
[0004] Therefore, visual positioning solutions based on vehicle cameras have become a highly promising technological approach due to their low hardware cost and rich perceptual information. This approach typically follows a "map first, then localize" model, where visual information of the scene is pre-collected to build a high-precision map, and the vehicle's pose is determined during localization by matching the real-time acquired images with the map. For example, visual SLAM solutions based on traditional local features such as ORB (Oriented Fast and Rotated BRIEF) can achieve good results in scenes with rich textures.
[0005] However, in engineering practice, it has been found that the aforementioned visual localization methods relying on single traditional features still have significant limitations when applied to particularly challenging scenarios such as underground parking lots. This is mainly because underground parking lots typically feature large areas of solid-color walls, epoxy floors, and load-bearing columns with uniform paint, resulting in extremely sparse surface textures and high scene repetition. Traditional local feature extraction algorithms that rely on image grayscale gradient changes (such as FAST corner detection based on ORB) struggle to extract a sufficient number of stable feature points with high discriminative power in such areas. This directly leads to two key problems:
[0006] 1) During the map building phase, a large amount of information is lost in sparse texture areas, making the map incomplete and creating "positioning blind spots".
[0007] 2) During the localization phase, the number of feature matches between real-time images and maps decreases sharply, resulting in poor matching quality and a high likelihood of erroneous pose estimations. This severely impacts the accuracy and robustness of localization, and can even lead to complete localization failure. (Refer to...) Figure 1 .
[0008] Therefore, overcoming the interference of sparse textured environments on visual feature extraction and matching, and providing a low-cost, high-precision localization method that is robust enough in various complex and closed scenarios, is a technical problem that urgently needs to be solved in the field of intelligent driving. Summary of the Invention
[0009] To address the shortcomings of existing technologies, this application proposes a sparse texture scene localization method and system that integrates multi-source visual features. By collaboratively utilizing traditional local features, optical flow tracking, and a deep learning-based feature matcher, it significantly improves mapping quality and localization performance in challenging scenarios. Furthermore, it effectively solves the problems of incomplete map information, poor localization robustness, and low accuracy caused by the difficulty of feature extraction and matching in sparse texture and highly repetitive scenes such as underground parking lots, where single-vision localization methods often fail.
[0010] The technical solution adopted in this invention is as follows:
[0011] A method for constructing high-precision maps of sparse textured scenes by fusing multi-source visual features includes the following steps:
[0012] S1. Use a binocular camera to acquire images of the target scene, define each pair of stereo images as a map node, and record the physical acquisition order of each node.
[0013] S2. Perform multi-source feature extraction and fusion on each map node image to form a fusion feature set that maintains high robustness under various texture conditions;
[0014] S3. For each matching point pair in the fusion feature set, the three-dimensional coordinates of the pair are obtained in the camera coordinate system using the intrinsic and extrinsic parameters of the binocular camera and the principle of triangulation. The three-dimensional coordinates of all fusion feature points of each node are stored to form the scene structure layer of that node.
[0015] S4. Determine the pose of each map node in the global unified coordinate system, and combine the poses of all nodes to form the node trajectory, thus completing the map construction.
[0016] Furthermore, the method for S2 to obtain the fused feature set is as follows:
[0017] S2.1. For the left-eye image of each map node, the ORB algorithm is used to extract the initial local features;
[0018] S2.2. The initial local feature set is verified and refined using the LK pyramid optical flow method to obtain a high-quality feature set for the nodes;
[0019] S2.3 If the number of feature points in the high-quality feature set is lower than the preset density threshold, feature supplementation is performed, and finally a fused feature set is generated for each node;
[0020] S2.4 First, cluster all fused features of all map nodes to form a visual dictionary; use the bag-of-words (BOW) model to generate a global descriptor for each node.
[0021] Furthermore, in S2.3, feature supplementation is initiated using the deep learning-based LoFTR method. The left and right eye images of the node are input into the pre-trained LoFTR model, and the model outputs a series of high-confidence matching point pairs. The matching points output by LoFTR are then filtered. If the distance between the coordinates of any side of a LoFTR matching point and the coordinates of a point in the existing high-quality feature set is less than 5 pixels, the supplemented point is discarded. After feature supplementation, a fused feature set is generated for each node.
[0022] Furthermore, in S2.4, the process of generating a global descriptor for each node using the bag-of-words approach is as follows: for the left eye image of each node, all its ORB feature descriptors are matched with the words in the visual dictionary, and the frequency of each word is counted to construct a k-dimensional word frequency histogram vector, which is the global descriptor for this node.
[0023] Furthermore, the method for constructing node trajectories in S4 is as follows:
[0024] S4.1 For key nodes in the path, directly measure their pose relative to the global coordinate system. ;
[0025] S4.2 For other nodes on the straight path, their global pose is recursively calculated by calculating the relative pose changes between adjacent nodes;
[0026] S4.3, Global pose of all nodes The set constitutes the node trajectory.
[0027] A sparse texture scene localization method that integrates multi-source visual features, based on the map constructed using the above method, the localization process is as follows:
[0028] Step 1: Acquire real-time images of the vehicle to be located using a monocular camera, extract its features and generate a bag-of-words descriptor. By comparing the descriptor with the global descriptor of all nodes in the map using Hamming distance, quickly match the map node that is closest to the vehicle's current location.
[0029] Step 2: Match the local features of the real-time image with the 3D feature points of the nearest map node to construct a "2D-3D" matching pair, and use the PnP model to solve for the vehicle's relative pose to the node.
[0030] Step 3: Combine the calculated relative pose with the global pose of the node trajectory layer to obtain the final positioning result of the vehicle in the global coordinate system, and perform multiple constraint verifications to eliminate outliers.
[0031] Furthermore, the calculated relative pose of the vehicle with respect to the node is... Global pose of the nearest node Perform coordinate transformation to obtain the vehicle's final pose in the global map coordinate system. , denoted as:
[0032]
[0033]
[0034] Where R is a 3×3 rotation matrix consisting of rotation components, and t is a 3×1 translation vector consisting of translation components.
[0035] Furthermore, constraint verification includes, but is not limited to, topological relationship constraints and physical height constraints. Topological relationship constraints limit the search range to the current node and its immediate neighbors based on the vehicle's direction and speed. If the nearest matched node does not conform to the expected topological order, the match is considered abnormal and rejected. Physical height constraints are determined by the vertical component of the calculated global pose. ,like If the location result is invalid, then the location result is determined to be invalid. For the installation height of the vehicle-mounted camera, This is the threshold for height deviation.
[0036] Furthermore, if the positioning result fails any of the above constraints, the relative pose calculated this time is retained, and the valid positioning result of the previous moment can be selectively output to ensure the smoothness of the trajectory; if the calculation results fail to pass the verification multiple times in a row, the system outputs the "positioning lost" status.
[0037] A positioning system for a vehicle, comprising:
[0038] The vehicle-mounted image acquisition unit is used to acquire environmental image data;
[0039] Multi-layer map pre-storage unit: Pre-stores a high-precision multi-layer map of the target scene;
[0040] The localization unit uses the acquired environmental image data to call the pre-stored map data, calculates the relative pose, and combines it with the global pose of the node trajectory layer to obtain the final localization result of the vehicle in the global coordinate system. The localization result is then subjected to multiple constraint verifications to eliminate outliers and obtain the final localization result.
[0041] The beneficial effects of this invention are:
[0042] (1) Significantly improve scene adaptability: By integrating three visual features based on different principles, ORB, LK optical flow and LoFTR, this invention can utilize efficient traditional features in texture-rich areas and automatically switch to robust deep learning features in sparse texture areas, effectively solving the "localization blind spot" problem and greatly enhancing the algorithm's adaptability to various scenes.
[0043] (2) Significantly improved positioning robustness: The introduction of a dual verification mechanism of "topological relationship constraint" and "physical height constraint" can effectively identify and eliminate gross positioning errors caused by environmental interference (such as changes in lighting, dynamic object occlusion) or scene repetition from both logical and physical levels, ensuring the continuity and reliability of the positioning trajectory.
[0044] (3) Ensuring high accuracy and low cost: While improving robustness, high-precision positioning with an average error rate of less than 5% is achieved through refined mapping and PnP solution. The entire solution requires only one low-cost monocular camera in the positioning stage, which greatly reduces the hardware deployment cost and facilitates commercialization.
[0045] (4) The integrity of map information has been optimized: The multi-source feature fusion strategy adopted in the mapping stage ensures that the constructed map can contain more comprehensive and denser structural information in the scene, laying a solid foundation for subsequent high-precision positioning. Attached Figure Description
[0046] Figure 1 This diagram illustrates the failure of existing visual localization methods in sparse textured scenes such as underground parking lots.
[0047] Figure 2 This is a schematic diagram of the overall process of a localization method that integrates multi-source visual features, provided in an embodiment of the present invention.
[0048] Figure 3 This invention utilizes deep learning feature matching methods in Figure 1 The diagram shows the effect of successful feature matching in a sparse texture scene.
[0049] Figure 4 This is a schematic diagram illustrating the principle of the filtering rules used in this invention to avoid overlapping with existing feature points when supplementing features.
[0050] Figure 5 This is a schematic diagram illustrating the principle of using optical flow to track common local features between adjacent map node images in an embodiment of the present invention.
[0051] Figure 6 This is a schematic diagram illustrating the principle of optimizing results using topological relationship constraints when performing node-level localization in this invention. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 2-6 The present invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.
[0053] Example 1 presents a method for constructing a high-precision map of a sparse textured scene by fusing multi-source visual features. The method aims to pre-generate a high-precision map containing rich and reliable visual information, typically completed offline within the target scene (e.g., an underground parking lot). It employs a "binocular mapping" approach to obtain accurate 3D scene information. The method includes the following steps.
[0054] S1. Data Acquisition and Node Division.
[0055] Target scene data acquisition: A binocular camera (e.g., a ZED2 camera with a baseline length of 120mm) is mounted on a data acquisition vehicle, which travels along a planned path within an underground parking lot. A pair of synchronized left and right stereo images are acquired at fixed intervals (e.g., every 2 meters), and each pair of stereo images is defined as a map node. Simultaneously, the physical acquisition order of each node is recorded for subsequent topology construction.
[0056] S2. Based on the map nodes collected in S1, extract and store a set of visual features for each map node that maintains high robustness under different texture environments, thus completing the construction of the fusion feature layer. The process is as follows:
[0057] S2.1 Extract the initial local feature set of each map node.
[0058] For the left-view image of each map node, the ORB algorithm is used to extract initial local features. The ORB algorithm combines FAST corner detection and BRIEF feature description. First, the FAST-16 algorithm is used to detect candidate corners in the image, and the image pyramid is used to ensure the scale invariance of the features. Then, the principal direction is calculated for each detected corner to ensure rotation invariance. Finally, a 256-bit BRIEF binary descriptor is generated around each corner. This yields the initial local feature set for each node.
[0059] S2.2 Feature Verification and Refinement.
[0060] Considering that the parameters of the left and right cameras of a stereo camera are similar and the baseline is fixed, the images captured by the left and right cameras can be regarded as two consecutive frames obtained after the camera undergoes a slight translation in the baseline direction. Utilizing this prior knowledge, this step uses the LK (Lucas-Kanade) pyramid optical flow method to verify and refine the initial local feature set.
[0061] Specifically, each ORB feature point extracted from the left eye image is used as a tracking target, and its matching position is predicted in the corresponding right eye image. The basic assumptions of the optical flow method are grayscale invariance and infinitesimal motion. For a 3×3 pixel window around the feature point, an overdetermined linear equation system can be established for the 9 pixels within the window to solve for a uniform motion vector (u, v), as shown in the formula:
[0062]
[0063] In the formula, , These are the gradients of the image in the x and y directions, respectively. Let be the gradient of the image in the time direction (i.e., between the left and right eyes). By solving this system of equations using the least squares method, the optimal motion vector (u,v) can be obtained.
[0064] The matching points obtained through LK optical flow calculation are compared with the points obtained through BRIEF descriptor matching in step 1. The error E(B,F) between the pixel coordinates of the two is calculated. A pixel error threshold is set; if the error is greater than the threshold, the matching pair is considered unreliable and is removed from the initial local feature set. After this step, a high-quality feature set of nodes verified by geometric constraints is obtained.
[0065] S2.3, Feature Supplement
[0066] After LK optical flow verification, the number of high-quality feature sets for the current node is checked. If the number of feature points is below a preset density threshold, for example, less than 200 points, the node image is determined to correspond to a sparse texture region. At this time, feature supplementation based on the deep learning-based LoFTR method is initiated. LoFTR is a detector-free local feature matching method that uses the self-attention mechanism of Transformer to find pixel-level dense correspondences between two images, performing well in low-texture regions. The left and right images of the map node are input into the pre-trained LoFTR model, and the model directly outputs a series of high-confidence matching point pairs. To avoid the supplemented feature points being too dense or overlapping with existing ORB feature points, the matching points output by LoFTR are filtered: if the distance between any side coordinate of a LoFTR matching point and the coordinate of a point in the existing high-quality feature set is less than 5 pixels, the supplemented point is discarded. Through this conditional supplementation mechanism, a fused feature set is finally generated for each node.
[0067] S2.4 Generation of Node Global Descriptors
[0068] To achieve rapid scene recognition during the localization phase, a Bag of Words (BOW) model is used to generate a global descriptor for each node.
[0069] First, all fused features (specifically, ORB feature descriptors) of all map nodes are aggregated and clustered using the k-means++ algorithm. Each cluster center is a "visual word", and all visual words constitute a visual dictionary.
[0070] Then, for the left eye image of each node, all its ORB feature descriptors are matched with the words in the dictionary, and the frequency of each word is counted to construct a k-dimensional word frequency histogram vector. This vector is the global descriptor of this node and is stored in the fusion feature layer.
[0071] S3. Construct the scene structure layer.
[0072] The scene structure layer stores the 3D spatial information of each feature point in the fused feature set. For each matching point pair in the fused feature set, its 3D coordinates (X, Y, Z) in the camera coordinate system can be recovered using the intrinsic and extrinsic parameters of the stereo camera and the principle of triangulation. Its 3D coordinates can be derived from the parallax. Direct calculation, , These are the horizontal pixel coordinates of the points in the point pair in the left and right eye image coordinate systems, as shown in the formula:
[0073]
[0074] In the formula, The baseline length of the binocular camera; Let x be the focal length of the camera in the x and y directions; The coordinates of the principal point of the camera image; This represents the horizontal pixel coordinates of the point in the point pair in the left-eye image coordinate system. This represents the vertical pixel coordinates of a point in the point pair within the coordinate system of the left-eye image. Storing the 3D coordinates of all fused feature points for each node constitutes the scene structure layer for that node.
[0075] S4. Construction of the node trajectory layer.
[0076] The node trajectory layer defines the pose of each map node's camera coordinate system relative to a globally unified map coordinate system, including the rotation matrix. Translation vector Its construction method is as follows:
[0077] S4.1 Calibrating Key Nodes: For key nodes such as intersections and turns in the path, use tools such as handheld laser rangefinders to manually and accurately measure their pose relative to the global coordinate system. .
[0078] S4.2, Inter-node pose recursion: For other nodes on the straight path, their global pose is recursively calculated by determining the relative pose changes between adjacent nodes. The relative pose can be calculated by using optical flow to track common feature points between two nodes, or by using the PnP / ICP algorithm after feature matching.
[0079] S4.3 Finally, the global pose of all nodes. The collection constitutes the node trajectory layer.
[0080] Example 2: Based on the sparse texture scene high-precision map constructed in Example 1 above, combined with... Figure 2 This invention also proposes a sparse texture scene localization method that integrates multi-source visual features, the specific steps of which are as follows:
[0081] Step 1: Node-level coarse positioning
[0082] Acquire real-time images captured by the vehicle-mounted monocular camera. Using the same method as during map construction, extract local features of the image center point as the global features of that frame. Then, utilize the existing visual dictionary in the map to transform these global features into a bag-of-words descriptor. Then, calculate. Global descriptor for all nodes in the map The Hamming distance between the nodes is the smallest, and the node with the smallest Hamming distance is the map node closest to the vehicle's current position.
[0083] The Hamming distance formula is shown below:
[0084]
[0085] In the formula, For XOR operation, This is the size of the dictionary.
[0086] Step 2, meter-level precision positioning
[0087] After determining the map node closest to the vehicle's current position, the fused feature set of the real-time image is matched with the 3D point cloud stored in the scene structure layer of that node. This forms a set of correspondences between 2D image points and 3D spatial points. Using this set of matching pairs, the PnP algorithm is employed to solve for the camera pose. The goal of the PnP problem is to find a camera rotation matrix R and translation vector t that projects 3D points onto corresponding 2D image points. This model can be expressed as the formula...
[0088]
[0089] In the formula: Scale factor; The first matrix represents the pixel coordinates of the local features to be matched in the real-time image to be localized; the second matrix is the intrinsic parameter matrix A of the vehicle-mounted monocular camera, where... Let be the focal length of the camera in the x and y directions. The first matrix represents the coordinates of the principal point of the camera image; these parameters can be obtained in advance through camera calibration. The second 3×4 matrix represents the camera extrinsic parameter matrix to be solved. Where R is the rotation component The 3×3 rotation matrix is formed by the translation components. The resulting 3×1 translation vector. This extrinsic parameter matrix describes the position and orientation relationship of the camera coordinate system relative to the map node coordinate system, i.e., the relative pose we need to solve for; (X,Y,Z) are the three-dimensional spatial coordinates corresponding to this local feature stored in the scene structure layer.
[0090] By providing at least three sets of matching points, typically selecting the top 15% of matching quality in this embodiment, and having more than six sets to improve robustness, and combining this with robust estimation algorithms such as RANSAC, the relative pose of the vehicle with respect to the nearest map node can be accurately solved. .
[0091] Step 3: Combine the calculated relative pose with the global pose of the node trajectory layer to obtain the final localization result of the vehicle in the global coordinate system. Before outputting the final result, perform multiple constraint verifications to remove outliers. The process is as follows:
[0092] The relative pose of the vehicle with respect to the node obtained in the previous step. Global pose of the nearest node Perform coordinate transformation to obtain the vehicle's final pose in the global map coordinate system. .
[0093]
[0094]
[0095] In this final pose Before outputting, perform the following two constraint verifications:
[0096] (1) Topological relation constraints
[0097] This constraint leverages the physical continuity of vehicle movement. The system maintains a node index for the vehicle's current location. During new node-level coarse localization, instead of globally searching all nodes, the search is limited to the current node and a few neighboring nodes based on the vehicle's direction and speed. If the nearest matched node deviates significantly from the expected topological order, the match is considered anomalous and rejected.
[0098] (2) Physical height constraints
[0099] This constraint leverages the physical prior that the vertical coordinates of a vehicle's Z-axis (or Y-axis, depending on the coordinate system definition) should remain relatively stable when the vehicle is driving in a flat parking lot. Let the installation height of the vehicle-mounted camera be... Then the vertical component in the calculated global pose Should be in Nearby. Set a height deviation threshold. The positioning results are then verified. If the location result is invalid, then the location result is deemed invalid.
[0100] If the positioning result fails any of the above constraints, the system will discard the current calculation result and may selectively output the valid positioning result from the previous moment to ensure trajectory smoothness. If multiple consecutive calculation results fail the verification, the system will output a "positioning lost" status.
[0101] Through the above specific implementation methods, the present invention can effectively integrate the advantages of multiple visual feature sources to construct a map with complete information, and achieve high-precision and high-robust positioning in sparse texture scenes through multi-scale matching and multi-constraint optimization during positioning.
[0102] Example 3, a positioning system for a vehicle, comprising:
[0103] The vehicle-mounted image acquisition unit is used to acquire environmental image data;
[0104] A multi-layer map pre-storage unit pre-stores a high-precision multi-layer map of the target scene constructed using the method in Example 1.
[0105] The positioning unit, based on the acquired environmental image data, calls the pre-stored map data and uses the method described in Embodiment 2 above to calculate the relative pose and combine it with the global pose of the node trajectory layer to obtain the final positioning result of the vehicle in the global coordinate system; and performs multiple constraint verification on the positioning result to eliminate outliers to obtain the final positioning result.
[0106] The multi-layer map pre-storage unit and the positioning unit are integrated into the vehicle-mounted computing platform. The multi-layer map pre-storage unit uses the non-volatile storage medium of the vehicle-mounted computing platform for data storage, while the positioning unit uses the high-performance processor of the vehicle-mounted computing platform to perform logical calculations and multiple constraint verifications for the positioning algorithm, thereby achieving real-time, high-precision position calculation.
[0107] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.
Claims
1. A method for constructing a high-precision map of a sparse textured scene by fusing multi-source visual features, characterized in that, Includes the following steps: S1. Use a binocular camera to acquire images of the target scene, define each pair of stereo images as a map node, and record the physical acquisition order of each node. S2. Perform multi-source feature extraction and fusion on each map node image to form a fusion feature set that maintains high robustness under various texture conditions; S3. For each matching point pair in the fusion feature set, the three-dimensional coordinates of the pair are obtained in the camera coordinate system using the intrinsic and extrinsic parameters of the binocular camera and the principle of triangulation. The three-dimensional coordinates of all fusion feature points of each node are stored to form the scene structure layer of that node. S4. Determine the pose of each map node in the global unified coordinate system, and combine the poses of all nodes to form the node trajectory, thus completing the map construction.
2. The method for constructing a high-precision map of a sparse textured scene by fusing multi-source visual features according to claim 1, characterized in that, The method for S2 to obtain the fused feature set is as follows: S2.
1. For the left-eye image of each map node, the ORB algorithm is used to extract the initial local features; S2.
2. The initial local feature set is verified and refined using the LK pyramid optical flow method to obtain a high-quality feature set for the nodes; S2.3 If the number of feature points in the high-quality feature set is lower than the preset density threshold, feature supplementation is performed, and finally a fused feature set is generated for each node; S2.4 First, cluster all fused features of all map nodes to form a visual dictionary; use the bag-of-words (BOW) model to generate a global descriptor for each node.
3. The method for constructing a high-precision map of a sparse textured scene by fusing multi-source visual features according to claim 2, characterized in that, In S2.3, the feature supplementation method based on deep learning LoFTR is initiated. The left and right eyes of the node are input into the pre-trained LoFTR model, and the model outputs a series of high-confidence matching point pairs. The matching points output by LoFTR are filtered. If the distance between the coordinates of any side of a LoFTR matching point and the coordinates of a point in the existing high-quality feature set is less than 5 pixels, the supplemented point is discarded. After feature supplementation, a fused feature set is generated for each node.
4. The method for constructing a high-precision map of a sparse textured scene by fusing multi-source visual features according to claim 2, characterized in that, In S2.4, the process of generating a global descriptor for each node using the bag-of-words approach is as follows: For the left eye image of each node, all its ORB feature descriptors are matched with the words in the visual dictionary, and the frequency of each word is counted to construct a k-dimensional word frequency histogram vector, which is the global descriptor for this node.
5. The method for constructing a high-precision map of a sparse textured scene by fusing multi-source visual features according to claim 1, characterized in that, The method for constructing node trajectories in S4 is as follows: S4.1 For key nodes in the path, directly measure their pose relative to the global coordinate system. ; S4.2 For other nodes on the straight path, their global pose is recursively calculated by calculating the relative pose changes between adjacent nodes; S4.3, Global pose of all nodes The set constitutes the node trajectory.
6. A sparse texture scene localization method that integrates multi-source visual features, characterized in that, The map constructed based on the high-precision map construction method for sparse textured scenes that integrates multi-source visual features as described in claim 1 has the following localization process: Step 1: Acquire real-time images of the vehicle to be located using a monocular camera, extract its features and generate a bag-of-words descriptor. By comparing the descriptor with the global descriptor of all nodes in the map using Hamming distance, quickly match the map node that is closest to the vehicle's current location. Step 2: Match the local features of the real-time image with the 3D feature points of the nearest map node to construct a "2D-3D" matching pair, and use the PnP model to solve for the vehicle's relative pose to the node. Step 3: Combine the calculated relative pose with the global pose of the node trajectory layer to obtain the final positioning result of the vehicle in the global coordinate system, and perform multiple constraint verifications to eliminate outliers.
7. The sparse texture scene localization method based on multi-source visual features according to claim 6, characterized in that, The calculated relative pose of the vehicle to the node Global pose of the nearest node Perform coordinate transformation to obtain the vehicle's final pose in the global map coordinate system. , denoted as: Where R is a 3×3 rotation matrix composed of rotation components, and t is a 3×1 translation vector composed of translation components.
8. The sparse texture scene localization method fusing multi-source visual features according to claim 6, characterized in that, Constraint verification includes, but is not limited to, topological constraints and physical height constraints. Topological constraints limit the search range to the current node and its immediate neighbors based on the vehicle's direction and speed. If the nearest matched node does not conform to the expected topological order, the match is considered abnormal and rejected. Physical height constraints are determined by the vertical component of the calculated global pose. ,like If the location result is invalid, then the location result is determined to be invalid. For the installation height of the vehicle-mounted camera, This is the threshold for height deviation.
9. The sparse texture scene localization method based on multi-source visual features according to claim 8, characterized in that, If the positioning result fails any of the above constraints, the calculated relative pose is discarded, and the valid positioning result of the previous moment can be selectively output to ensure the smoothness of the trajectory; if the calculation results fail to pass the verification multiple times in a row, the system outputs the "positioning lost" status.
10. A positioning system for a vehicle, characterized in that, include: The vehicle-mounted image acquisition unit is used to acquire environmental image data; Multi-layer map pre-storage unit: Pre-stores a high-precision multi-layer map of the target scene; The localization unit uses the acquired environmental image data to call the pre-stored map data, calculates the relative pose, and combines it with the global pose of the node trajectory layer to obtain the final localization result of the vehicle in the global coordinate system. The localization result is then subjected to multiple constraint verifications to eliminate outliers and obtain the final localization result.