A complex intersection map making method of a multi-sensor fusion scheme

Abstract

This invention discloses a method for creating complex intersection maps using a multi-sensor fusion scheme, belonging to the field of intelligent driving technology. The method includes: preprocessing single-trip data to generate an odometer trajectory and aligning it with a GPS positioning trajectory to obtain a single-trip trajectory; selecting point cloud frame pairs whose center point distance is less than a preset distance threshold from different single-trip trajectories to form a set of potential cross-trip matching point cloud pairs; performing two-stage registration for each point cloud frame pair in the set based on a registration network and algorithm to solve the relative pose of each point cloud frame; constructing a graph model based on the relative poses of all point cloud frames, and jointly optimizing it by introducing odometer trajectory constraints within the same trip and cross-trip relative pose constraints; fusing the point clouds from each trip based on the optimized poses to generate an intersection point cloud map. This invention enables the construction of complex intersection point cloud maps with high precision, strong robustness, automation, and low cost, and is suitable for applications such as intelligent transportation and autonomous driving that require high map freshness and accuracy.

Description

Technical Field

[0001] This invention belongs to the field of map production technology, specifically relating to a method for producing complex intersection maps using a multi-sensor fusion scheme. Background Technology

[0002] To meet the demands of precise environmental perception and decision-making necessary for achieving advanced autonomous driving, the construction and maintenance of high-precision maps has become a core supporting technology, serving as the data foundation for accurate positioning, reliable perception, and forward-looking planning in autonomous driving systems. Currently, existing high-precision map production technologies mainly fall into two categories: vision-based crowdsourcing solutions and crowdsourcing solutions based on multi-sensor fusion and update strategies. Both of these solutions have achieved certain successes in open environments.

[0003] A vision-based crowdsourcing solution uses vehicle-mounted cameras as the primary sensor, acquiring image sequences from multiple vehicles through crowdsourcing, and then constructing maps using computer vision technology. For example, patent document CN110060343A proposes a map construction method whose process mainly includes image acquisition → local map construction → similar image detection and relative pose estimation → global map creation. The core of this method relies on the extraction and matching of visual features, aiming to achieve the construction and updating of large-scale maps at a low cost.

[0004] Crowdsourcing solutions based on multi-sensor fusion and update strategies: To overcome the limitations of single sensors, existing technologies integrate GNSS, inertial navigation systems, and visual sensors, and introduce intelligent update strategies. For example, patent document CN110412635A proposes a compact combination method that uses a priori visual feature point cloud map as an environmental beacon, providing positioning constraints through visual matching when GNSS signals are obstructed. Patent document CN116089445A proposes a dynamic crowdsourcing update strategy based on contribution discrimination for specific scenarios, aiming to achieve efficient, on-demand incremental map updates and reduce interference with continuous vehicle operation. The core of these solutions lies in seeking a balance between positioning accuracy and update efficiency through multi-source information complementarity and process optimization.

[0005] While the above solutions are feasible to some extent in general road scenarios, they still present the following significant problems in complex intersection scenarios such as elevated bridges, multi-level interchanges, and large roundabouts: (1) Visual matching fails under repetitive textures: Repetitive textures are likely to exist in scenarios such as overpasses, such as continuous and consistent median strips. Visual features cannot be clearly distinguished, which can easily lead to mismatches, causing local map stitching errors, and ultimately resulting in ghosting and structural misalignment in the global map.

[0006] (2) Multi-sensor fusion is constrained by signal degradation: GNSS signals at complex intersections are easily affected by obstruction and multipath effects, resulting in positioning drift. Directly aligning and overlaying point cloud maps will introduce system errors, causing misalignment and ghosting between map layers collected at different times.

[0007] (3) The update strategy is not adapted to the complexity and real-time requirements of intersections: the existing incremental update mechanism is difficult to accurately identify real changes and perceived noise in scenarios with complex intersection structures and dense traffic flow. If the update timing is inappropriate or the update content is inaccurate, it is very easy to cause navigation decision errors.

[0008] In summary, existing crowdsourced map building and updating solutions still have significant shortcomings in complex intersection scenarios, particularly in visual feature matching, multi-sensor fusion positioning accuracy, and intelligent update strategies in highly dynamic environments. These shortcomings hinder the reliable application of high-precision maps in autonomous driving systems. Therefore, there is an urgent need for a map production method that can effectively integrate multi-source perception data, adapt to the structural characteristics of complex intersections, and possess strong robustness and real-time update capabilities. Summary of the Invention

[0009] In view of the above, the purpose of this invention is to provide a method for creating complex intersection maps using a multi-sensor fusion scheme. This method utilizes lidar data collected by a fleet of ordinary passenger vehicles, fuses single-trip data collected by multiple vehicles passing through the same intersection at different times, constructs cross-trip intelligent association, and builds a two-stage precise registration framework. Global optimization is performed by fusing odometer data and cross-trip matching constraints, thereby transforming fragmented crowdsourced data into a globally consistent 3D point cloud map of complex intersections. This achieves high-precision, high-robustness, automated, and low-cost intersection map construction, effectively supporting the accurate perception and positioning of complex scenes by intelligent driving systems.

[0010] To achieve the above-mentioned objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for creating complex intersection maps using a multi-sensor fusion scheme, comprising the following steps: Preprocess the single-trip data of each data collection vehicle passing through complex intersections to generate odometer tracks, and align the odometer tracks with the GPS positioning tracks to a unified coordinate system to obtain aligned single-trip tracks. Based on spatial proximity, point cloud frame pairs whose center point distance is less than a preset distance threshold are automatically selected from different single-trip trajectories to form a set of potential point cloud matching pairs for cross-trip matching. For each point cloud frame pair in the potential point cloud matching pair set, a registration network is first used for coarse registration, and then a registration algorithm is used for fine registration to solve the relative pose of each point cloud frame. A graph model is constructed based on the relative pose of all point cloud frames, and odometry trajectory constraints and cross-trip relative pose constraints are introduced. All poses are jointly optimized by minimizing the overall constraint residual. The point clouds of each single trip are fused using the optimized pose to generate an intersection point cloud map.

[0011] Preferably, the step of automatically selecting point cloud frame pairs with a center point distance less than a preset distance threshold from different single-trip trajectories based on spatial proximity to form a set of potential point cloud matching pairs for cross-trip matching includes: The center point of each frame of point cloud in the aligned single-pass trajectory is placed in a unified coordinate system. The Euclidean distance between the center points of any two frames of point cloud from different single passes is calculated. Point cloud frame pairs with a distance less than a preset distance threshold are selected as candidate matching pairs and added to the potential point cloud matching pair set.

[0012] Preferably, the coarse registration using a registration network includes: Each point cloud frame pair in the potential point cloud matching pair set is divided into a source point cloud and a target point cloud to be registered, and then input into a registration network built based on a deep learning network. In the registration network, the input point cloud is first downsampled at multiple levels using a downsampling layer to form a hierarchical structure containing the input point cloud, dense point cloud, and coarse point cloud. Then, the local features of each dense point cloud are extracted using a Transformer layer and concatenated to form the local features of the coarse point cloud. A Geometric Transformer layer is then used to extract geometric features based on the local features of the coarse point cloud. Next, an attention mechanism is used to fuse the geometric features of the coarse point cloud to obtain the source point cloud features and the target point cloud features. Subsequently, the Gaussian correlation matrix between the source point cloud features and the target point cloud features is calculated, and the Top K matching point pairs are selected based on the matrix element values. Finally, the local transformation matrix is ​​calculated based on the dense point cloud corresponding to each matching point pair, and the optimal local transformation matrix is ​​selected as the coarse registration output from the source point cloud to the target point cloud through a voting mechanism.

[0013] Preferably, the fine registration using a registration algorithm includes: Using the local transformation matrix obtained from coarse registration as the initial estimate, the random sampling consensus algorithm is used to remove mismatches from the point cloud correspondence to obtain the set of interior points; Based on the set of interior points, the iterative nearest point algorithm is used for iterative optimization, and the optimal relative pose transformation matrix is ​​finally solved as the fine registration output result from the source point cloud to the target point cloud.

[0014] Preferably, the step of constructing a graph model based on the relative pose of all point cloud frames, and introducing intra-trip odometer trajectory constraints and cross-trip relative pose constraints, includes: The relative poses of the point cloud frames collected by each data acquisition vehicle at each time point are defined as nodes in the graph model, and the pose constraints between nodes are defined as edges in the graph model. In this process, consecutive frame nodes belonging to the same single trip are connected by edge connections through the same trip odometer trajectory constraints, and frame nodes from different single trips that have been registered are connected by edge connections through cross-trip relative pose constraints.

[0015] Preferably, the joint optimization of all poses by minimizing the overall constraint residual includes: Define the optimization objective function as follows: , in, The pose represents the position of multiple trajectories and is an optimization variable of the objective function. Indicates the first The first trip and Odometer trajectory constraints for the same trip in two frames. Indicates the first Trip Frame and the Trip Frame-by-frame relative pose constraints. and These represent the trajectory constraint weights for the same trip and the relative pose constraint weights across trips, respectively. and These represent the sets of frame indices for the same trip and across trips, respectively, with superscripts... Indicates transpose; , in, Indicates the first The first trip and Relative pose measurements between two frames, Indicates the first The first trip and The relative pose estimation values ​​between the two frames. Indicates a unified coordinate system; , in, Indicates the first Trip Frame and the Trip The relative pose measurement of the frame. Indicates the first Trip Frame and the Trip The relative pose estimate of the frame; Optimize variables through iterative adjustment To minimize the objective function This allows for a globally consistent optimal pose estimate.

[0016] Preferably, the step of fusing the point clouds of each single-trip data using the optimized pose to generate an intersection point cloud map includes: The optimized pose is used to transform the original point cloud of each frame of each single trip to a unified world coordinate system and then superimpose them. The superimposed overall point cloud is then filtered and voxelized downsampled to generate the final 3D point cloud map of the complex intersection.

[0017] Secondly, the present invention provides a complex intersection map making device based on a multi-sensor fusion scheme, which is implemented using the above-mentioned complex intersection map making method based on the multi-sensor fusion scheme, including: a trajectory preliminary alignment module, a cross-trip point cloud association module, a relative pose solving module, a global pose optimization module, and a point cloud map generation module. The preliminary trajectory alignment module is used to preprocess the single-trip data of each data collection vehicle passing through complex intersections, generate odometer trajectory, and align the odometer trajectory with the GPS positioning trajectory to a unified coordinate system to obtain the aligned single-trip trajectory. The cross-trip point cloud association module is used to automatically filter point cloud frame pairs whose center point distance is less than a preset distance threshold from different single-trip trajectories based on spatial proximity, forming a set of potential point cloud matching pairs for cross-trip matching. The relative pose solving module is used to solve the relative pose of each point cloud frame for each point cloud frame pair in the potential point cloud matching pair set by first using a registration network for coarse registration and then using a registration algorithm for fine registration. The global pose optimization module is used to construct a graph model based on the relative poses of all point cloud frames, and introduces the same trip odometer trajectory constraint and cross trip relative pose constraint, and performs joint optimization of all poses by minimizing the overall constraint residual; The point cloud map generation module is used to fuse the point clouds of each single pass using the optimized pose to generate an intersection point cloud map.

[0018] Thirdly, an electronic device provided by an embodiment of the present invention includes a memory and one or more processors. The memory is used to store a computer program, and the processor is used to implement the above-described method for creating complex intersection maps using a multi-sensor fusion scheme when executing the computer program.

[0019] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for creating complex intersection maps using a multi-sensor fusion scheme.

[0020] Compared with the prior art, the beneficial effects of the present invention include at least the following: (1) This invention uses a two-stage registration strategy to perform multi-scale feature extraction and precise registration of point clouds, which can effectively cope with challenges such as noise, occlusion and repetitive structures, and has strong robustness. Furthermore, by combining the same-trip odometer trajectory constraint and cross-trip relative pose constraint for global optimization, the cross-trip constraint provides a global correction benchmark for the system, significantly suppressing cumulative errors, and realizing the construction of sub-decimeter level intersection maps with no ghosting and high consistency, ultimately improving the positioning accuracy by more than 60%.

[0021] (2) This invention significantly reduces the computational complexity of matching massive amounts of data through intelligent association based on spatial proximity, and achieves end-to-end automated generation from raw data to high-quality maps through full-process design, which significantly improves processing efficiency.

[0022] (3) This invention relies on easily accessible crowdsourced sensor data, without the need for professional data collection equipment and a large amount of manual intervention. It is highly practical and provides a complete technical solution that is feasible and low-cost for the continuous updating of high-precision intersection maps.

[0023] In summary, this invention provides a complete technical path for low-cost, high-precision map construction based on crowdsourced data, and is particularly suitable for application scenarios such as intelligent transportation and autonomous driving that have extremely high requirements for map freshness and accuracy. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a method for creating complex intersection maps using a multi-sensor fusion scheme, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of constructing cross-pass relative pose constraints provided by an embodiment of the present invention; Figure 3 It is a planar projection of the three-dimensional point cloud map of a complex intersection provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of a complex intersection map creation device based on a multi-sensor fusion scheme provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0027] The inventive concept of this invention is as follows: Addressing the significant shortcomings of existing map building schemes in complex intersection scenarios, particularly in visual feature matching, multi-sensor fusion positioning accuracy, and intelligent update strategies under high dynamic environments, this invention provides a method for creating complex intersection maps using a multi-sensor fusion scheme. Based on single-trip data from multiple vehicles, it first constructs a set of potential point cloud matching pairs through cross-trip spatial association. Then, it innovatively employs a two-stage registration strategy combining deep learning coarse and fine registration to effectively handle scenarios with large pose deviations and low overlap rates. Finally, through global pose graph optimization that integrates single-trip odometer constraints and cross-trip matching constraints, it effectively corrects accumulated errors, obtaining a globally consistent high-precision 3D point cloud map of the intersection.

[0028] like Figure 1 As shown in the figure, the embodiment provides a method for creating complex intersection maps using a multi-sensor fusion scheme, including the following steps: S1 preprocesses the single-trip data of each data collection vehicle passing through complex intersections to generate odometer tracks, and aligns the odometer tracks with the GPS positioning tracks to a unified coordinate system to obtain aligned single-trip tracks.

[0029] S1.1, construct the odometer trajectory for each single trip.

[0030] In this embodiment, for each single data pass, the trajectory of its point cloud sequence is obtained using a Lidar Odometry algorithm. Taking the first... Taking the data as an example, it is composed of The sequence consists of a laser point cloud, the first... odometer track of the trip The definition is as follows: , in, express The first trip The laser point cloud to the first The 6-DOF pose transformation of the trajectory origin is a 4×4 matrix.

[0031] S1.2, based on the GPS positioning trajectory of each single trip and the odometer trajectory in S1.1, aligns the odometer trajectory (with 6 degrees of freedom) to the GPS positioning trajectory (GPS only has position xy information, 2 degrees of freedom) coordinate system.

[0032] To integrate the relative accuracy of odometers with the absolute reference of GPS, a unified and consistent initial coordinate system needs to be established for multiple data sets.

[0033] In this embodiment, the odometry trajectory is given based on the Umeyama algorithm. GPS positioning trajectory Optimal transformation and ,in Represents the rotation matrix. To represent a translation vector, we can construct the following 4×4 transformation matrix: , Using the transformation matrix The odometer tracks of different single trips Aligned to a unified GPS coordinate system, it is represented as: , in, This represents the aligned single-pass trajectory.

[0034] S2, based on spatial proximity, automatically filters point cloud frame pairs whose center point distance is less than a preset distance threshold from different single-trip trajectories, forming a set of potential point cloud matching pairs for cross-trip matching.

[0035] Since the computational complexity of pairwise matching across all frames in multiple runs of massive point cloud data increases quadratically, it is completely impractical in engineering. This invention utilizes the coarse spatial consistency provided by GPS to transform the infinite global search problem into a problem of generating a finite set of local potential point cloud matching pairs. This constructs a high-quality set of potential point cloud matching pairs, thereby significantly reducing the computational load and improving the matching quality, providing a feasible computational foundation for subsequent precise registration.

[0036] In this embodiment, based on step S1.2, the different single-trip data have been aligned to a unified GPS coordinate system, and potential point cloud matching pairs can be found based on a distance threshold. For example, for data from the first... Trip Point cloud of a frame , and the Trip Point cloud of a frame When the distance between the center points of the two is less than the set distance threshold At that time, point cloud frames Added to the set of potential point cloud matching pairs In the embodiment, the distance threshold The value can be selected within the preferred range of 10m to 60m.

[0037] S3. For each point cloud frame pair in the potential point cloud matching pair set, a registration network is first used for coarse registration, and then a registration algorithm is used for fine registration to solve the relative pose of each point cloud frame.

[0038] After step S2, some point cloud registration pairs with relative poses to be solved have been obtained. Although these point cloud pairs have been aligned to the GPS coordinate system, they cannot be directly used for subsequent reconstruction due to the large positioning error of GPS. Therefore, this invention proposes a two-stage point cloud pair pose solving scheme.

[0039] S3.1 Point cloud matching based on registration network (coarse registration).

[0040] To address the inherent sensitivity of traditional Iterative Closest Point Algorithm (ICP) to initial values, this invention significantly expands the convergence domain by designing a deep learning-driven coarse registration stage, thus solving the registration challenge in scenarios with large displacements and small overlaps. Furthermore, considering that single-scale feature extraction cannot adapt to the geometric diversity of complex scenes, a tree-based multi-resolution structure is used to achieve cross-scale feature fusion from local details to global structure, realizing multi-scale geometric perception capabilities.

[0041] In this embodiment, each point cloud frame pair in the potential point cloud matching pair set is divided into a source point cloud (srcpointcloud) to be registered. With the target point cloud (ref pointcloud) Input a registration network built based on a deep learning network to obtain the 6DOF transformation from the source point cloud to the target point cloud. , .

[0042] Specifically, the registration network includes: (1) Multiple downsampling of point cloud: The original point cloud corresponds to the original resolution; the first downsampling of the original point cloud is denoted as the dense point cloud (resolution); the last downsampling is denoted as the coarse point cloud (resolution). A tree structure is adopted, with each coarse point cloud as the root node corresponding to a set of dense point clouds, forming a hierarchical structure containing the input point cloud, dense point cloud, and coarse point cloud.

[0043] (2) Rough point cloud feature extraction: a. Extracting local features from dense point clouds and coarse point clouds based on Transformer networks. It is composed of concatenated dense point cloud features, with a dimension of .

[0044] b. Extracting geometric features of coarse point clouds based on Geometric Transformer network , dimension .

[0045] c. Feature weighting based on self-attention and cross-attention mechanisms. In addition to the projection matrices corresponding to the query, key, and value in the traditional self-attention mechanism... To weight the geometric features extracted by the GeometricTransformer network together, a geometric projection matrix is ​​introduced. : , in, Indicates the first Local features of each point, dimension is ; Indicates the first The geometric features of two points, the dimension is ; Indicates the first The features learned from each point after the attention mechanism, with dimensions of _ ... Projection matrix Dimension is , which are the parameters that need to be learned in the network.

[0046] Finally, the point cloud feature matrix is ​​obtained. ， Dimension is ,Depend on points Composition. Specifically, for The source point cloud features are denoted as , dimension ,in The number of coarse point clouds corresponding to the source point cloud; the features of the target point cloud are denoted as... , dimension ,in The number of coarse point clouds corresponding to the target point cloud.

[0047] (3) Point cloud feature matching: Calculate source point cloud features Features of the target point cloud Gaussian correlation matrix Each element The calculation is as follows: , in, and These represent the first and second characteristic matrices of the source point cloud, respectively. The column and the first of the target point cloud feature matrix List, This represents a hyperparameter (in this example, the value is 1).

[0048] When the value is 1: , Based on the scores of each element of the Gaussian correlation matrix, the Top K (100 in this example) matching point pairs of the coarse point cloud are selected.

[0049] (4) Registration estimation: For each pair of coarse point cloud matches, a local transformation of the locally matched point cloud is calculated using its corresponding dense point cloud match and the Singular Value Decomposition (SVD) algorithm. , Then, all local transformations are voted on across the entire matched dense point cloud, and the optimal transformation (with the highest inlier ratio) is selected as the initial estimate, which is the coarse registration output from the source point cloud to the target point cloud.

[0050] Specifically, regarding the voting mechanism, the first Group dense point cloud matching is denoted as , Indicates the first The first point in the source point cloud of the group dense point cloud matching One point, Indicates the first In the group dense point cloud matching, the first target point cloud One point, Indicates the first The set of dense point clouds, the first Local transformations corresponding to group points (hereinafter referred to as) , where represent the rotation matrix and translation vector respectively, calculate The cost function is as follows, and a unique solution can be obtained using SVD decomposition: , in, This represents the L2 norm.

[0051] The set of all dense point cloud matching groups is denoted as ,have The transformation with the largest number of points within all groups of dense points is selected as the optimal transformation. , (hereinafter referred to as) : , in, This is the threshold hyperparameter for determining interior points. In actual use, it is set to 0.2, which means 0.2 meters.

[0052] S3.2, Point cloud registration solution based on RANSAC+ ICP algorithm (fine registration).

[0053] To effectively integrate the advantages of data-driven and geometric optimization, this invention further performs a fine registration step, combining the strong semantic understanding capability of deep learning with the precise numerical calculation of traditional geometric optimization to form a hybrid solution with complementary advantages.

[0054] In this embodiment, the local transformation matrix obtained from coarse registration is used as the initial estimate. First, the Random Sample Consensus (RANSAC) algorithm is used to remove mismatches from the point cloud correspondence to obtain the inlier set. Then, based on the inlier set, the Iterative Closest Point Algorithm (ICP) algorithm is used for iterative optimization, and finally the optimal 6DOF transformation from the source point cloud to the target point cloud is solved. , This serves as the output of the fine registration from the source point cloud to the target point cloud.

[0055] If the first Trip Frame 1 as the target point cloud, the first Trip If the frame is used as the source point cloud, then: , in, Indicates the first Trip Frame to the Trip The pose transformation matrix of the frame.

[0056] Experiments have verified that the two-stage strategy of coarse and fine registration described above can improve the registration success rate from <20% to >85% under extreme conditions such as an initial pose deviation of 30° and a translation of 5 meters. The convergence radius is increased by 3-5 times, and the final registration error is controlled within the range of 5-15 cm, meeting the stringent requirements of high-precision map reconstruction and autonomous driving positioning. Even in low-quality scenarios with a point cloud overlap rate of only 30%, stable registration performance is maintained, and the registration error in the overlapping area is reduced by 40-60%. Furthermore, the registration network of this invention can adapt to different scene types (urban roads, highways, underground parking lots, etc.) after training, without requiring scene-specific parameter tuning, and has cross-scene generalization capabilities.

[0057] S4 constructs a graph model based on the relative poses of all point cloud frames, and introduces odometer trajectory constraints for the same trip and relative pose constraints across trips. It then performs joint optimization of all poses by minimizing the overall constraint residual.

[0058] To achieve deep integration of multi-source constraints, this invention unifies the continuous constraints of single-trip odometer and the discrete constraints of cross-trip point cloud matching into a unified model, forming complementary advantages: single-trip odometer provides smooth local continuity, while cross-trip matching provides a global correction benchmark.

[0059] In this embodiment, a graph model is constructed using the poses of different vehicles at different times as nodes and the constraints between poses as edges. Nodes of consecutive frames belonging to the same trip are connected by edges constructed using the same trip odometer trajectory constraints of each trip's data in step S1. Nodes of frames from different trips that have been registered in step S3 are connected by edges constructed using cross-trip relative pose constraints. Figure 2 The diagram shows the establishment of cross-trip relative pose constraints. Different colored arrow sequences represent different single-trip trajectories, and the connected solid lines indicate the existence of corresponding cross-trip relative pose constraints.

[0060] The optimization objective is to minimize the residuals of all constraints, that is, to adjust the pose estimates of all nodes (different vehicles at different times) so that the constraint relationships of the entire graph are consistent to the greatest extent.

[0061] Define the optimization objective function as follows:

[0062] in, The 6DOF pose represents the position of multiple trajectories and is the optimization variable of the objective function.

[0063] , Multiple trajectories are represented in the world coordinate system. Below, among them Indicates the first trip 6DOF pose of a frame, multiple trajectories The initial value comes from the result of step S1.2, that is, the aligned single-pass trajectory. .

[0064] Indicates the first The first trip and Odometer trajectory constraints for the same trip in two frames. Indicates the first Trip Frame and the first trip Frame-by-frame relative pose constraints. and These represent the trajectory constraint weights for the same trip and the relative pose constraint weights across trips, respectively. and These represent the sets of frame indices for the same trip and across trips, respectively, with superscripts... This indicates transpose.

[0065] , in, Indicates the first The first trip and The relative pose measurement values ​​of the two frames are derived from step S1.1; Indicates the first The first trip and The relative pose estimates between the two frames are derived from the optimization variables. .

[0066] , in, Indicates the first Trip Frame and the Trip The relative pose measurement value of the frame is derived from the result of step S3; Indicates the first Trip Frame and the Trip The relative pose estimate of the frame is derived from the optimization variables. .

[0067] Optimize variables through iterative adjustment To minimize the objective function This allows for a globally consistent optimal pose estimate.

[0068] In summary, by introducing cross-trip relative pose constraints, a global correction anchor point is provided for single-trip odometers, effectively suppressing single-trip cumulative errors. Furthermore, the multi-constraint collaborative optimization ensures the spatial consistency of estimated trajectories for different vehicles and time periods in overlapping areas, eliminating the "ghosting" and "double-image" phenomena common in traditional methods, and effectively improving the consistency of multi-trip trajectories. Moreover, the two types of constraints form a redundant verification mechanism: when a trip's odometer experiences significant drift, cross-trip constraints provide reliable correction; conversely, odometer constraints can verify the effectiveness of cross-trip matching, thereby effectively enhancing the system's robustness.

[0069] S5 uses the optimized pose to fuse the point clouds of each single trip to generate an intersection point cloud map.

[0070] In this embodiment, after optimizing the global pose map, the precise poses of all vehicles at each moment in the world coordinate system are obtained. Based on this, in the point cloud reconstruction stage, all the original laser point clouds collected in a single trip are uniformly transformed and stitched into the global world coordinate system through their corresponding optimized poses.

[0071] Unlike simple splicing, this process fully integrates observation data from multiple trips and multiple perspectives: for the same physical area at the intersection, point cloud observations from different times and angles are precisely superimposed, significantly improving the integrity of scene coverage and point cloud density.

[0072] Subsequently, efficient point cloud filtering and voxelization downsampling algorithms were applied to effectively remove ghosting caused by dynamic objects (such as vehicles and pedestrians) while preserving key geometric features, and to significantly reduce data redundancy.

[0073] The final generated 3D point cloud map of the complex intersection, such as Figure 3 As shown, it not only possesses millimeter-level relative accuracy and rich scene details, but also global consistency, providing directly usable, high-quality basic map data for downstream tasks such as high-precision positioning and autonomous driving simulation. This process enables automated, high-precision reconstruction from multi-source heterogeneous data to standardized map products.

[0074] Based on the same inventive concept, this embodiment of the invention also provides a complex intersection map production device 400 with a multi-sensor fusion scheme, including: a trajectory preliminary alignment module 410, a cross-trip point cloud association module 420, a relative pose solving module 430, a global pose optimization module 440, and a point cloud map generation module 450.

[0075] The preliminary trajectory alignment module 410 is used to preprocess the single-trip data of each data collection vehicle passing through complex intersections, generate odometer trajectory, and align the odometer trajectory with the GPS positioning trajectory to a unified coordinate system to obtain the aligned single-trip trajectory.

[0076] The cross-trip point cloud association module 420 is used to automatically filter point cloud frame pairs whose center point distance is less than a preset distance threshold from different single-trip trajectories based on spatial proximity, forming a set of potential point cloud matching pairs for cross-trip matching.

[0077] The relative pose solving module 430 is used to solve the relative pose of each point cloud frame for each point cloud frame pair in the potential point cloud matching pair set. First, a registration network is used for coarse registration, and then a registration algorithm is used for fine registration.

[0078] The global pose optimization module 440 is used to construct a graph model based on the relative pose of all point cloud frames, and introduces the same trip odometer trajectory constraint and cross trip relative pose constraint, and performs joint optimization of all poses by minimizing the overall constraint residual.

[0079] The point cloud map generation module 450 is used to fuse the point clouds of each single pass using the optimized pose to generate an intersection point cloud map.

[0080] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described method for creating complex intersection maps using a multi-sensor fusion scheme when executing the computer program.

[0081] Based on the same inventive concept, this embodiment of the invention also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for creating complex intersection maps using a multi-sensor fusion scheme.

[0082] It should be noted that the complex intersection map making device, electronic device, and computer-readable storage medium provided in the above embodiments all belong to the same inventive concept as the complex intersection map making method of multi-sensor fusion scheme. For details of their specific implementation process, please refer to the embodiment of the complex intersection map making method of multi-sensor fusion scheme, which will not be repeated here.

[0083] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for creating complex intersection maps using a multi-sensor fusion scheme, characterized in that, Includes the following steps: Preprocess the single-trip data of each data collection vehicle passing through complex intersections to generate odometer tracks, and align the odometer tracks with the GPS positioning tracks to a unified coordinate system to obtain aligned single-trip tracks. Based on spatial proximity, point cloud frame pairs whose center point distance is less than a preset distance threshold are automatically selected from different single-trip trajectories to form a set of potential point cloud matching pairs for cross-trip matching. For each point cloud frame pair in the potential point cloud matching pair set, a registration network is first used for coarse registration, and then a registration algorithm is used for fine registration to solve the relative pose of each point cloud frame. A graph model is constructed based on the relative pose of all point cloud frames, and odometry trajectory constraints and cross-trip relative pose constraints are introduced. Joint optimization of all poses is performed by minimizing the overall constraint residuals, including: Define the optimization objective function as follows: ， in, The pose represents the position of multiple trajectories and is an optimization variable of the objective function. Indicates the first The first trip and Odometer trajectory constraints for the same trip in two frames. Indicates the first Trip Frame and the Trip Frame-by-frame relative pose constraints. and These represent the trajectory constraint weights for the same trip and the relative pose constraint weights across trips, respectively. and These represent the sets of frame indices for the same trip and across trips, respectively, with superscripts... Indicates transpose; ， in, Indicates the first The first trip and Relative pose measurements between two frames, Indicates the first The first trip and The relative pose estimation values ​​between the two frames. Indicates a unified coordinate system; ， in, Indicates the first Trip Frame and the Trip The relative pose measurement of the frame. Indicates the first Trip Frame and the Trip The relative pose estimate of the frame; Optimize variables through iterative adjustment To minimize the objective function This allows for a globally consistent optimal pose estimate. The point clouds of each single trip are fused using the optimized pose to generate an intersection point cloud map.

2. The method for creating complex intersection maps using a multi-sensor fusion scheme according to claim 1, characterized in that, The method of automatically selecting point cloud frame pairs with a center point distance less than a preset threshold from different single-trip trajectories based on spatial proximity to form a potential point cloud matching pair set for cross-trip matching includes: The center point of each frame of point cloud in the aligned single-pass trajectory is placed in a unified coordinate system. The Euclidean distance between the center points of any two frames of point cloud from different single passes is calculated. Point cloud frame pairs with a distance less than a preset distance threshold are selected as candidate matching pairs and added to the potential point cloud matching pair set.

3. The method for creating complex intersection maps using a multi-sensor fusion scheme according to claim 1, characterized in that, The coarse registration using a registration network includes: Each point cloud frame pair in the potential point cloud matching pair set is divided into a source point cloud and a target point cloud to be registered, and then input into a registration network built based on a deep learning network. In the registration network, the input point cloud is first downsampled at multiple levels using a downsampling layer to form a hierarchical structure containing the input point cloud, dense point cloud, and coarse point cloud. Then, the local features of each dense point cloud are extracted using a Transformer layer and concatenated to form the local features of the coarse point cloud. A Geometric Transformer layer is then used to extract geometric features based on the local features of the coarse point cloud. Next, an attention mechanism is used to fuse the geometric features of the coarse point cloud to obtain the source point cloud features and the target point cloud features. Subsequently, the Gaussian correlation matrix between the source point cloud features and the target point cloud features is calculated, and the Top K matching point pairs are selected based on the matrix element values. Finally, the local transformation matrix is ​​calculated based on the dense point cloud corresponding to each matching point pair, and the optimal local transformation matrix is ​​selected as the coarse registration output from the source point cloud to the target point cloud through a voting mechanism.

4. The method for creating complex intersection maps using a multi-sensor fusion scheme according to claim 1 or 3, characterized in that, The fine registration using a registration algorithm includes: Using the local transformation matrix obtained from coarse registration as the initial estimate, the random sampling consensus algorithm is used to remove mismatches from the point cloud correspondence to obtain the set of interior points; Based on the set of interior points, the iterative nearest point algorithm is used for iterative optimization, and the optimal relative pose transformation matrix is ​​finally solved as the fine registration output result from the source point cloud to the target point cloud.

5. The method for creating complex intersection maps using a multi-sensor fusion scheme according to claim 1, characterized in that, The relative pose graph model is constructed based on all point cloud frames, and odometry trajectory constraints and cross-trip relative pose constraints are introduced, including: The relative poses of the point cloud frames collected by each data acquisition vehicle at each time point are defined as nodes in the graph model, and the pose constraints between nodes are defined as edges in the graph model. In this process, consecutive frame nodes belonging to the same single trip are connected by edge connections through the same trip odometer trajectory constraints, and frame nodes from different single trips that have been registered are connected by edge connections through cross-trip relative pose constraints.

6. The method for creating complex intersection maps using a multi-sensor fusion scheme according to claim 1, characterized in that, The process of fusing the point clouds from each single pass using the optimized pose to generate an intersection point cloud map includes: The optimized pose is used to transform the original point cloud of each frame of each single trip to a unified world coordinate system and then superimpose them. The superimposed overall point cloud is then filtered and voxelized downsampled to generate the final 3D point cloud map of the complex intersection.

7. A complex intersection map creation device based on a multi-sensor fusion scheme, implemented using the complex intersection map creation method based on the multi-sensor fusion scheme according to any one of claims 1 to 6, characterized in that, include: The module includes a preliminary trajectory alignment module, a cross-trip point cloud association module, a relative pose solving module, a global pose optimization module, and a point cloud map generation module. The preliminary trajectory alignment module is used to preprocess the single-trip data of each data collection vehicle passing through complex intersections, generate odometer trajectory, and align the odometer trajectory with the GPS positioning trajectory to a unified coordinate system to obtain the aligned single-trip trajectory. The cross-trip point cloud association module is used to automatically filter point cloud frame pairs whose center point distance is less than a preset distance threshold from different single-trip trajectories based on spatial proximity, forming a set of potential point cloud matching pairs for cross-trip matching. The relative pose solving module is used to solve the relative pose of each point cloud frame for each point cloud frame pair in the potential point cloud matching pair set by first using a registration network for coarse registration and then using a registration algorithm for fine registration. The global pose optimization module is used to construct a graph model based on the relative poses of all point cloud frames, and introduces the same trip odometer trajectory constraint and cross trip relative pose constraint, and performs joint optimization of all poses by minimizing the overall constraint residual; The point cloud map generation module is used to fuse the point clouds of each single pass using the optimized pose to generate an intersection point cloud map.

8. An electronic device comprising a memory and one or more processors, the memory for storing a computer program, characterized in that, The processor is used to implement the complex intersection map production method of the multi-sensor fusion scheme according to any one of claims 1 to 6 when executing a computer program.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the method for creating complex intersection maps using the multi-sensor fusion scheme according to any one of claims 1 to 6.

Images