Static map construction method and device based on automatic driving

Abstract

The invention discloses a static map construction method and device based on automatic driving, and the method comprises the steps: carrying out the time synchronization and space registration of laser radar three-dimensional point cloud data and IMU motion information, and obtaining registration data; performing landmark extraction and data association based on the registration data to obtain a data association result; constructing a mixed factor graph model based on a data association result; jointly estimating vehicle pose information based on a mixed factor graph model, and performing local map construction by adopting a sparse increment voxel method to obtain a local point cloud map containing a dynamic object; and carrying out dynamic object filtering and map splicing on the local point cloud map to obtain a global static point cloud map without a dynamic object. According to the method, pose estimation is carried out by introducing a factor graph optimization technology, the local point cloud map containing the dynamic object is accurately constructed, and then dynamic object filtering and map splicing algorithms are combined, so that the accuracy and robustness of static map construction are effectively improved, and meanwhile, the adaptability to the dynamic environment is enhanced.

Description

Technical Field

[0001] This application relates to the field of map construction technology, and in particular to a method and apparatus for constructing static maps based on autonomous driving. Background Technology

[0002] In the field of autonomous driving, environmental perception and mapping are key tasks for achieving efficient navigation and localization. With the advancement of LiDAR technology, 3D point cloud data is increasingly being used for environmental modeling. However, relying solely on LiDAR data to build environmental maps may not effectively cope with interference from dynamic objects, resulting in map information with significant noise and inaccuracies. Summary of the Invention

[0003] The main objective of this application is to provide a static map construction method and apparatus based on autonomous driving, which aims to solve the technical problem that existing technologies that rely solely on LiDAR data to construct environmental maps cannot effectively cope with the interference of dynamic objects, resulting in poor accuracy of the obtained map information.

[0004] To achieve the above objectives, this application proposes a static map construction method based on autonomous driving. The static map construction method based on autonomous driving includes: Acquire three-dimensional point cloud data from a lidar radar and motion information from an IMU, and then synchronize and spatially register the three-dimensional point cloud data with the IMU motion information to obtain registration data. Based on the registration data, landmark extraction and data association are performed to obtain the data association results; Based on the data association results, a hybrid factor graph model is constructed, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights; Based on the hybrid factor graph model, the vehicle pose information is jointly estimated, and the local map is constructed using the sparse incremental voxel method according to the vehicle pose information, to obtain a local point cloud map containing dynamic objects. Dynamic object filtering and map stitching are performed on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0005] In one embodiment, the step of acquiring 3D point cloud data from a lidar radar and IMU motion information, and then performing time synchronization and spatial registration between the 3D point cloud data and the IMU motion information to obtain registration data, includes: Acquire 3D point cloud data from LiDAR and motion information from IMU; The three-dimensional point cloud data of the lidar is preprocessed to obtain preprocessed three-dimensional point cloud data. The point cloud preprocessing includes at least point cloud distortion correction, iterative error Kalman filtering, and noise point removal. The preprocessed 3D point cloud data is synchronized with the IMU motion information in time to obtain time synchronization data. Spatial registration is performed on the time synchronization data to obtain registered data, wherein spatial registration includes at least coarse registration based on initial pose estimation and fine registration based on iterative nearest point algorithm.

[0006] In one embodiment, the step of performing landmark extraction and data association based on the registration data to obtain data association results includes: Landmark features are extracted based on the registration data, wherein the landmark features include at least geometric features and semantic features; The extracted landmark features are described to obtain feature descriptors; Based on the feature descriptors, semantic similarity matching, geometric position similarity matching, and target distance similarity matching are performed to obtain semantic similarity, geometric position similarity, and target distance similarity. A comprehensive matching score is obtained by weighting and fusing the semantic similarity, geometric position similarity, and target distance similarity. Based on the comprehensive matching score, landmark filtering and data association are performed to obtain data association results, wherein the data association results include at least the association relationship and association confidence between landmarks.

[0007] In one embodiment, constructing a hybrid factor graph model based on the data association results includes: An initial hybrid factor graph is constructed based on the data association results, wherein the initial hybrid factor graph consists of a loosely coupled factor graph and a tightly coupled factor graph with preset weights; An adaptive factor graph weight algorithm based on dual-delay deep deterministic policy gradient is adopted to dynamically adjust the weights of loosely coupled and tightly coupled factor graphs in the initial mixed factor graph according to the real-time scene information during vehicle driving, so as to obtain the adjusted initial mixed factor graph model. A scene-aware, loosely coupled adaptive factor graph combination algorithm is adopted to dynamically select and combine different loosely coupled factors based on real-time scene information during vehicle operation to expand and optimize the adjusted initial hybrid factor graph model, thereby obtaining a hybrid factor graph model.

[0008] In one embodiment, constructing an initial mixture factor graph based on the data association results includes: Frameworks for constructing loosely coupled factor graphs and tightly coupled factor graphs based on the association relationships and association confidence between landmarks; Based on the vehicle's current pose, pose nodes are added to the frames of the loosely coupled factor graph and the tightly coupled factor graph respectively to obtain the initial loosely coupled factor graph and the initial tightly coupled factor graph containing pose information. Each pose node represents the vehicle's position and attitude at a certain moment. Add loose coupling factors to the initial loose coupling factor map to obtain a loose coupling factor map, wherein the loose coupling factors include at least semantic landmark observation factors, loop closure detection factors, and GNSS prior factors; Add a tight coupling factor to the initial tight coupling factor map to obtain a tight coupling factor map, wherein the tight coupling factor includes at least an IMU pre-integration factor, a laser odometry factor, and a point cloud registration factor; The loosely coupled factor map and the tightly coupled factor map are combined according to preset weights to obtain an initial hybrid factor map model. The preset weights are set based on the confidence of the association between landmarks, the importance of landmark features, and the complexity of the vehicle driving environment.

[0009] In one embodiment, the method employing a factor graph weight adaptive algorithm based on a dual-delay deep deterministic policy gradient dynamically adjusts the weights of the loosely coupled and tightly coupled factor graphs in the initial mixed factor graph according to real-time scene information during vehicle operation, to obtain an adjusted initial mixed factor graph model, including: A dual-delay deep deterministic policy gradient algorithm model is constructed, wherein the dual-delay deep deterministic policy gradient algorithm model includes an actor network and a critic network, the actor network is used to select actions according to the current policy, and the critic network is used to evaluate the value of the selected actions; The weights of the loosely coupled and tightly coupled factor maps in the initial mixed factor map are used as the action space of the dual-delay deep deterministic policy gradient algorithm model, and the real-time scene information during vehicle driving is used as the state space of the dual-delay deep deterministic policy gradient algorithm model. The actor network selects weighted adjustments to actions within the motion space; The value assessment result is obtained by evaluating the value of the selected weight adjustment action in the state space based on the critic network. Based on the value assessment results, the policy parameters of the actor network are updated using the policy gradient update algorithm to obtain the updated actor network. Based on the updated actor network, a new weight adjustment action is selected again in the action space, and the value of the selected weight adjustment action is continuously evaluated and optimized in the state space by the critic network until the preset convergence condition is reached, so as to obtain the optimal weight adjustment strategy. The weights of the loosely coupled and tightly coupled factor graphs in the initial mixed factor graph are dynamically adjusted according to the optimal weight adjustment strategy to obtain the adjusted mixed factor graph model.

[0010] In one embodiment, the combination algorithm of loosely coupled adaptive factor graph based on scene awareness dynamically selects and combines different loosely coupled factors according to real-time scene information during vehicle operation to expand and optimize the adjusted initial hybrid factor graph model, resulting in a hybrid factor graph model, including: Acquire real-time scene information during vehicle operation; Based on the real-time scene information, a rule-based decision tree algorithm is used to dynamically select loose coupling factors and tight coupling factors. Each node of the decision tree algorithm represents a scene feature, each branch represents a decision path based on the scene feature, and each leaf node represents the selected loose coupling factor or tight coupling factor. A temporary factor graph is constructed based on the selected loose coupling factor and tight coupling factor, wherein the temporary factor graph is composed of the selected loose coupling factor and tight coupling factor; The temporary factor graph is subjected to consistency checks and optimization to obtain an optimized temporary factor graph; The optimized temporary factor graph is fused with the adjusted initial hybrid factor graph model to obtain a hybrid factor graph model. The fusion process includes merging factor graphs, updating pose nodes, and re-establishing relationships between factors.

[0011] In one embodiment, the step of jointly estimating vehicle pose information based on the hybrid factor graph model, and constructing a local map using the sparse incremental voxel method based on the vehicle pose information to obtain a local point cloud map containing dynamic objects, includes: Based on the hybrid factor graph model, the vehicle pose information is jointly estimated using a factor graph optimization algorithm, wherein the factor graph optimization algorithm includes at least the Gauss-Newton algorithm and the Levenberg-Marquardt algorithm. Based on the vehicle pose information, the point cloud data during the vehicle's driving process is segmented in chronological order to obtain a series of point cloud frames. Each frame of point cloud data is voxelized to generate a voxel mesh, where each voxel in the voxel mesh represents a cubic region in space. The generated voxel grid is filtered using the sparse incremental voxel method, retaining non-empty voxels containing point cloud data to obtain a sparse voxel grid. Based on the sparse voxel mesh, adjacent point cloud frames are registered and fused according to the vehicle pose information to obtain a local point cloud map. The local point cloud map is used to identify and mark dynamic objects using a machine learning-based dynamic object recognition algorithm, resulting in a local point cloud map containing dynamic objects.

[0012] In one embodiment, the step of performing dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects includes: The dynamic objects marked in the local point cloud map containing dynamic objects are identified and classified. Based on the category and motion characteristics of the dynamic objects, the corresponding filtering strategy is used to remove the dynamic objects, so as to obtain a local point cloud map without dynamic objects. Feature points in the local point cloud map without dynamic objects are extracted and matched using a feature matching-based stitching algorithm to establish the relative pose relationship between different local point cloud maps without dynamic objects. An ICP-based stitching algorithm is used to optimize the relative pose relationship obtained by feature matching through an iterative nearest-point algorithm, resulting in an optimized relative pose relationship. Based on the optimized relative pose relationship, the local point cloud maps without dynamic objects are stitched together to obtain a global static point cloud map without dynamic objects.

[0013] Furthermore, to achieve the above objectives, this application also proposes a static map building device based on autonomous driving, which includes: The registration module is used to acquire three-dimensional point cloud data of the lidar and IMU motion information, and to perform time synchronization and spatial registration of the three-dimensional point cloud data and IMU motion information to obtain registration data. The association module is used to extract landmarks and associate data based on the registration data to obtain the data association results; A construction module is used to construct a hybrid factor graph model based on the data association results, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights; The estimation module is used to jointly estimate vehicle pose information based on the hybrid factor graph model, and to construct a local map using the sparse incremental voxel method based on the vehicle pose information to obtain a local point cloud map containing dynamic objects. The stitching module is used to perform dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0014] This application proposes one or more technical solutions to acquire 3D point cloud data from a LiDAR radar and IMU motion information, and to synchronize and spatially register the 3D point cloud data with the IMU motion information to obtain registered data. Based on the registered data, landmark extraction and data association are performed to obtain data association results. A hybrid factor graph model is constructed based on the data association results, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights. Vehicle pose information is jointly estimated based on the hybrid factor graph model, and a local map is constructed using the sparse incremental voxel method based on the vehicle pose information to obtain a local point cloud map containing dynamic objects. Dynamic object filtering and map stitching are performed on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects. Through the above methods, by introducing factor graph optimization technology for pose estimation, a local point cloud map containing dynamic objects is accurately constructed. Furthermore, by combining dynamic object filtering algorithms and map stitching algorithms, the accuracy and robustness of static map construction are effectively improved, while enhancing adaptability to dynamic environments. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating an embodiment of the static map construction method based on autonomous driving in this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the static map construction method based on autonomous driving in this application; Figure 3 This is a schematic diagram of the module structure of a static map building device based on autonomous driving according to an embodiment of this application.

[0018] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0020] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0021] The main solution of this application embodiment is as follows: acquire 3D point cloud data of LiDAR and IMU motion information, and perform time synchronization and spatial registration of the 3D point cloud data and IMU motion information to obtain registration data; perform landmark extraction and data association based on the registration data to obtain data association results; construct a hybrid factor graph model based on the data association results, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights; jointly estimate vehicle pose information based on the hybrid factor graph model, and construct a local map using the sparse incremental voxel method according to the vehicle pose information to obtain a local point cloud map containing dynamic objects; perform dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0022] Relying solely on LiDAR data to build environmental maps may not be effective in dealing with interference from dynamic objects, resulting in map information with high noise and inaccuracy.

[0023] This application provides a solution that uses factor graph optimization technology for pose estimation to accurately construct a local point cloud map containing dynamic objects. Then, by combining dynamic object filtering algorithms and map stitching algorithms, it effectively improves the accuracy and robustness of static map construction while enhancing the adaptability to dynamic environments.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a static map building device based on autonomous driving. The following description uses a static map building device based on autonomous driving as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a method for constructing static maps based on autonomous driving, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the static map construction method based on autonomous driving in this application.

[0026] In this embodiment, the static map construction method based on autonomous driving includes steps S10 to S50: Step S10: Acquire the 3D point cloud data of the lidar and the motion information of the IMU, and perform time synchronization and spatial registration of the 3D point cloud data and the IMU motion information to obtain registration data.

[0027] It should be noted that the 3D point cloud data from the LiDAR is obtained by scanning the surrounding environment and contains rich spatial information, such as the position and shape of objects. The IMU motion information is acquired by the inertial measurement unit and contains dynamic information such as vehicle acceleration and angular velocity. Time synchronization and spatial registration align these two types of data to the same time and space reference frames to ensure data accuracy and consistency.

[0028] Understandably, before synchronizing and spatially registering 3D point cloud data with IMU motion information, it is necessary to preprocess the 3D point cloud data, such as denoising and filtering, to improve the data quality.

[0029] It is worth noting that time synchronization is typically achieved through timestamp matching to ensure that each data point has an accurate time stamp. Spatial alignment involves complex coordinate transformations and calibration processes to eliminate deviations and errors between different sensors.

[0030] In one feasible implementation, step S10 may include: acquiring three-dimensional point cloud data from a lidar and motion information from an IMU; performing point cloud preprocessing on the lidar three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data, wherein the point cloud preprocessing includes at least point cloud distortion correction, iterative error Kalman filtering, and noise point removal; synchronizing the preprocessed three-dimensional point cloud data with the IMU motion information in time to obtain time synchronization data; and performing spatial registration on the time synchronization data to obtain registration data, wherein the spatial registration includes at least coarse registration based on initial pose estimation and fine registration based on the iterative nearest point algorithm. It should be noted that in this embodiment, point cloud preprocessing includes point cloud distortion correction, iterative error Kalman filtering, and noise removal. Point cloud distortion correction can eliminate point cloud data distortion caused by uneven LiDAR scanning speed or vehicle movement; iterative error Kalman filtering can smooth point cloud data and reduce noise interference; noise removal can further clean the data and improve the accuracy of subsequent processing. Point cloud distortion correction can restore the true spatial position of the raw point cloud data obtained by LiDAR scanning by performing geometric transformations. Iterative error Kalman filtering is a recursive algorithm that uses the system's dynamic model and observation model to estimate the optimal value of the state variables through prediction and update steps, thereby achieving smoothing of the point cloud data.

[0031] Understandably, time synchronization is achieved through timestamp matching, which specifically involves matching the timestamps of the LiDAR 3D point cloud data and the IMU motion information, aligning data with similar timestamps, ensuring that each data point has an accurate timestamp, thereby obtaining time-synchronized data and ensuring the consistency of LiDAR data and IMU data in time.

[0032] Spatial registration criterion, based on time synchronization, aligns the 3D point cloud data of the LiDAR and the motion information of the IMU into the same spatial reference frame through coordinate transformation and calibration processes. This eliminates deviations and errors between different sensors, thereby obtaining registration data. The spatial registration criterion includes two steps: coarse registration and fine registration, gradually reducing the deviation between data from different sensors to ultimately obtain high-precision registration data.

[0033] Coarse registration based on initial pose estimation can quickly reduce the spatial deviation between data from different sensors, while fine registration based on the iterative nearest-point algorithm can further precisely adjust the position of the data to achieve high-precision spatial alignment. Specifically, this involves: using a transformation matrix based on initial pose estimation to roughly align the LiDAR 3D point cloud data and IMU motion information; subsequently, employing the iterative nearest-point algorithm to continuously optimize the transformation matrix, minimizing the spatial deviation between the two datasets until a preset convergence condition is met or the maximum number of iterations is reached, thus completing the fine registration process.

[0034] Step S20: Based on the registration data, perform landmark extraction and data association to obtain the data association results.

[0035] It's important to note that landmark extraction refers to identifying environmental elements with salient features from registration data, such as road signs and building edges. These elements play a crucial role in map construction, providing essential positioning references. Data association involves matching these landmarks with previous map or sensor data to establish temporal continuity and spatial consistency, ensuring the accuracy and coherence of the map. Through landmark extraction and data association, a preliminary estimate of vehicle pose can be achieved, providing a foundation for subsequent map optimization and construction.

[0036] In one feasible implementation, step S20 may include: extracting landmark features based on the registration data, wherein the landmark features include at least geometric features and semantic features; performing feature description on the extracted landmark features to obtain feature descriptors; performing semantic similarity matching, geometric position similarity matching, and target distance similarity matching based on the feature descriptors to obtain semantic similarity, geometric position similarity, and target distance similarity; performing weighted fusion based on the semantic similarity, geometric position similarity, and target distance similarity to obtain a comprehensive matching score; and performing landmark filtering and data association based on the comprehensive matching score to obtain data association results, wherein the data association results include at least the association relationship and association confidence between landmarks.

[0037] It should be noted that landmark features refer to information that can uniquely identify or significantly distinguish different landmarks, including geometric features and semantic features. Geometric features describe the spatial attributes of landmarks, such as shape, size, and orientation. Landmarks with stable geometric structures are extracted from the registered point cloud data, including corner points, line segments, planar regions, and cylinders. Line segments include streetlights and traffic sign poles, planar regions include building facades and walls, and cylinders include lampposts and utility poles. The extracted geometric features include: position coordinates (x, y, z), normal vector, curvature, dimensions (length, width, and height), and principal orientation.

[0038] Semantic features express high-level information such as the category and function of landmarks. Using pre-trained 3D semantic segmentation networks, such as PointNet++, PVCNN, and MinkowskiNet, the registered data is classified, and each point or clustered object is assigned a semantic label. Landmark semantic categories include: "traffic lights," "road signs," "streetlights," "utility poles," and "building corners," etc. Each landmark object receives a semantic label and its confidence score.

[0039] Feature description is the process of transforming landmark features into mathematical representations, that is, constructing a joint feature descriptor based on the extracted landmark features, in the form of a vector, as shown below: F=[Fgeo,Fsem] Where F is the feature descriptor, Fgeo is the geometric descriptor, such as SHOT, FPFH, or custom geometric parameter encoding, and Fsem is the semantic encoding, such as one-hot vector + confidence.

[0040] Semantic similarity matching focuses on the consistency of landmarks in terms of category and function, geometric location similarity matching considers the spatial proximity of landmarks, and target distance similarity matching further considers the relative distance relationship between landmarks. By weighted fusion of these three similarities, a comprehensive matching score can be obtained, which reflects the accuracy and reliability of landmark matching. Landmark filtering and data association based on the comprehensive matching score can eliminate mismatches and low-confidence landmarks, retaining high-quality data association results and providing accurate and reliable input for subsequent graph optimization and map construction.

[0041] The purpose of semantic similarity matching is to quickly eliminate obviously mismatched objects, narrowing the search space, by calculating the similarity between two landmarks L. i and L j Semantic similarity: ; in, Landmark L i and L j The semantic similarity is given by α∈(0,1), which is the preset approximate category similarity weight.

[0042] The purpose of geometric location similarity matching is to determine whether two landmarks are spatially close, measured by calculating the Euclidean distance or Manhattan distance between them. Calculating the Euclidean distance between the center coordinates of two landmarks: ; in, Landmark L i and L j Euclidean distance between them , Landmark L i and L j The position vector.

[0043] like If the distance is less than a preset threshold, the locations are considered close and have high similarity. This can be normalized to a similarity score, as shown in the following formula: ; in, Landmark L i and L j Geometric similarity, To control the decay rate.

[0044] Target spacing similarity matching, also known as topological matching, aims to improve matching robustness by utilizing the relative spatial relationships or configurations between landmarks. For each landmark L... i Construct its neighborhood graph: Select the k nearest neighbor landmarks sorted by distance, and determine the relative distance d of each neighbor. ij Relative azimuth angle θ ij And the semantic type of the neighbors; define local topological descriptors: such as "with a street lamp as the center, there is a traffic sign 1.5m to the east and a utility pole 2.3m to the north"; during matching, compare the neighborhood structure of the two landmarks and use Hausdorff distance or graph matching algorithms to calculate topological similarity, that is, target distance similarity. .

[0045] semantic similarity Geometric similarity , Target distance similarity A weighted fusion is performed to obtain a comprehensive matching score. : ; in, , , Semantic similarity Geometric similarity , Target distance similarity The weighting coefficients, , , satisfy + + =1.

[0046] Based on comprehensive matching score Perform landmark filtering if If the score exceeds the preset threshold, then landmark L is determined. i and L j For the same physical entity, complete the data association and output the associated landmarks and their association confidence as the data association result.

[0047] Step S30: Construct a hybrid factor graph model based on the data association results, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights.

[0048] It should be noted that the hybrid factor graphical model is a type of graphical model used to represent variables and their relationships. In autonomous driving map construction, the hybrid factor graphical model can integrate data from different sensors, including LiDAR and IMU, as well as landmark features and data association results, thereby achieving high-precision estimation of vehicle pose and map features.

[0049] In the hybrid factor graph model, loosely coupled and tightly coupled factor graphs are balanced by weights to adapt to different application scenarios and data characteristics. The weights can be adjusted based on factors such as data quality, sensor accuracy, and the reliability of landmark features to achieve optimal map building results.

[0050] By constructing a hybrid factor graph model and solving it using graph optimization algorithms, high-precision estimation of vehicle pose and map features can be achieved. This helps autonomous driving systems achieve stable navigation and positioning in complex environments, improving the safety and reliability of autonomous driving.

[0051] Step S40: Based on the hybrid factor graph model, jointly estimate the vehicle pose information, and construct a local map using the sparse incremental voxel method according to the vehicle pose information to obtain a local point cloud map containing dynamic objects.

[0052] It should be noted that vehicle pose information refers to the vehicle's position and orientation in three-dimensional space, including the vehicle's coordinates (x, y, z) and attitude, such as pitch, yaw, and roll angles. Jointly estimating vehicle pose information involves comprehensively considering information from LiDAR, IMU, landmark features, and data association results, and using graph optimization algorithms to accurately estimate the vehicle pose.

[0053] The sparse incremental voxel method is an efficient point cloud map construction approach. By voxelizing the point cloud data, the space is divided into a series of small cubes, or voxels. Only representative or feature points within each voxel are retained, thus significantly reducing the amount of data and computational complexity. Simultaneously, the map is updated incrementally, adding only newly observed point cloud data, maintaining the sparsity and real-time nature of the map.

[0054] When constructing local point cloud maps, dynamic objects in the environment, such as moving vehicles and pedestrians, must be considered. These dynamic objects can affect the accuracy and stability of the map. Therefore, effective methods are needed to handle and remove these dynamic objects during the construction of local point cloud maps. For example, dynamic objects can be identified and marked by analyzing the motion characteristics and temporal continuity of the point cloud data. In the subsequent map construction process, these marked dynamic objects can be removed from the point cloud data to reduce their impact on the map. In this way, more accurate and stable local point cloud maps can be constructed, providing more reliable environmental information for the navigation and positioning of autonomous driving systems.

[0055] In one feasible implementation, step S40 may include: jointly estimating vehicle pose information using a factor graph optimization algorithm based on the hybrid factor graph model, wherein the factor graph optimization algorithm includes at least the Gauss-Newton algorithm and the Levenberg-Marquardt algorithm; segmenting the point cloud data during vehicle movement according to time sequence based on the vehicle pose information to obtain a series of point cloud frames; performing voxelization processing on each frame of point cloud data to generate a voxel grid, wherein each voxel in the voxel grid represents a cubic region in space; filtering the generated voxel grid using the sparse incremental voxel method to retain non-empty voxels containing point cloud data to obtain a sparse voxel grid; registering and fusing adjacent point cloud frames based on the sparse voxel grid according to the vehicle pose information to obtain a local point cloud map; and using a machine learning-based dynamic object recognition algorithm to identify and mark dynamic objects in the local point cloud map to obtain a local point cloud map containing dynamic objects.

[0056] It should be noted that the hybrid factor graph model consists of edges and nodes. Nodes represent variables such as landmarks and vehicle poses in the map, while edges represent the relationships or constraints between these variables, including temporal continuity constraints between vehicle poses, spatial constraints between landmark feature points, and observation constraints between vehicle poses and landmark feature points. Loosely coupled factor graphs are mainly used to represent temporal continuity constraints between vehicle poses and spatial constraints between landmark feature points; these constraints are relatively relaxed, allowing for a certain error range. Tightly coupled factor graphs, on the other hand, are used to represent observation constraints between vehicle poses and landmark feature points; these constraints are more stringent, requiring precise matching.

[0057] Based on the hybrid factor graph model, the factor graph optimization algorithm is used to jointly estimate vehicle pose information. During this process, the algorithm iteratively adjusts the vehicle pose and the positions of landmark feature points to minimize the overall error function. This error function typically includes multiple components such as observation error, temporal continuity error, and spatial constraint error. By weighted summing of these errors, a comprehensive error index can be obtained. The goal of the factor graph optimization algorithm is to find the vehicle pose and landmark feature point positions that minimize this comprehensive error.

[0058] In the Gauss-Newton algorithm, the error function is approximated as a quadratic function, and the vehicle pose and the positions of landmark feature points are updated by solving a system of linear equations. The Levenberg-Marquardt algorithm adds a trust region to the Gauss-Newton algorithm to control the step size of the iterations, preventing the algorithm from diverging or getting trapped in local optima.

[0059] By jointly estimating vehicle pose information, the precise trajectory of the vehicle during its journey can be obtained. Furthermore, the factor graph optimization algorithm based on the hybrid factor graph model can effectively handle sensor noise and data uncertainty, improving the robustness and accuracy of map construction.

[0060] After obtaining the vehicle's pose information, the next step is to segment the point cloud data during the vehicle's movement in chronological order, resulting in a series of point cloud frames. Each frame of point cloud data contains environmental information observed by the vehicle at a certain moment, forming the basis for constructing a local point cloud map.

[0061] Voxelization of each frame of point cloud data divides it into a series of small cubic regions, or voxels, with each voxel containing only one representative or feature point. This significantly reduces the amount of point cloud data and computational complexity, improving the efficiency and real-time performance of map construction. Specifically, this includes defining the voxel size v. s ×v s ×v s For example, a 0.1m × 0.1m × 0.1m grid is used to divide the 3D space into a regular grid. Each cubic region is called a voxel. For each point pk in each frame of the point cloud Pt, the voxel index to which it belongs is calculated: ; in, , , This represents the 3D coordinates of the k-th point in the point cloud. This indicates the side length of a voxel.

[0062] Building a hash table or a three-dimensional array to store all non-empty voxels can avoid storing a large number of empty voxels, thus improving memory efficiency and processing speed.

[0063] In the sparse incremental voxel method, only non-empty voxels containing point cloud data are retained to generate a sparse voxel mesh. Then, based on the sparse voxel mesh and vehicle pose information, adjacent point cloud frames are registered and fused to obtain a local point cloud map. The registration process involves calculating the transformation relationship between adjacent point cloud frames and aligning them to the same coordinate system. The fusion process involves merging the registered point cloud data to generate a continuous local point cloud map.

[0064] However, when constructing local point cloud maps, dynamic objects in the environment can affect the accuracy and stability of the map. Therefore, effective methods are needed to identify and label these dynamic objects. Machine learning-based dynamic object recognition algorithms can automatically identify and label dynamic objects by analyzing information such as the shape, texture, and motion characteristics of point cloud data. In the subsequent map construction process, removing these labeled dynamic objects from the point cloud data can reduce their impact on the map and improve its accuracy and stability.

[0065] By combining the sparse incremental voxel method with dynamic object recognition algorithms, a local point cloud map containing dynamic objects can be constructed. This map not only includes static environmental information but also the position and motion information of dynamic objects, providing richer and more reliable environmental perception information for the navigation and positioning of autonomous driving systems.

[0066] Step S50: Perform dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0067] It should be noted that the global static point cloud map is an important environmental model required by autonomous driving systems. It provides static environmental information during vehicle operation, including roads, buildings, trees, traffic cones, etc. When constructing the global static point cloud map, it is necessary to further process the local point cloud maps containing dynamic objects to remove the dynamic objects and stitch multiple local maps together into a complete global map.

[0068] Dynamic object filtering is based on already identified and labeled dynamic objects. After obtaining a local point cloud map containing dynamic objects, the labeled dynamic objects can be directly removed from the point cloud data, retaining only static environmental information. This reduces the impact of dynamic objects on the map, improving its accuracy and stability.

[0069] Map stitching involves combining multiple local point cloud maps according to the vehicle's travel trajectory to obtain a complete global static point cloud map. During the stitching process, overlapping areas between adjacent local maps need to be considered. By calculating the transformation relationships between point cloud data within the overlapping areas, they are aligned to the same coordinate system. Simultaneously, the stitched point cloud data needs to be smoothed to reduce stitching gaps and noise, improving the map's continuity and smoothness.

[0070] In one feasible implementation, step S50 may include: identifying and classifying the marked dynamic objects in the local point cloud map containing dynamic objects; removing dynamic objects based on their category and motion characteristics using appropriate filtering strategies to obtain a local point cloud map without dynamic objects; extracting feature points from the local point cloud map without dynamic objects using a feature matching-based stitching algorithm and matching them to establish a relative pose relationship between different local point cloud maps without dynamic objects; optimizing the relative pose relationship obtained by feature matching using an ICP-based stitching algorithm through an iterative nearest-point algorithm to obtain an optimized relative pose relationship; and stitching the local point cloud maps without dynamic objects based on the optimized relative pose relationship to obtain a global static point cloud map without dynamic objects.

[0071] It's important to note that identifying and classifying dynamic objects is a crucial step in the dynamic object filtering process. Machine learning-based dynamic object recognition algorithms can automatically analyze the features of point cloud data, distinguishing dynamic objects from static environmental information and classifying them accurately. For example, dynamic objects can be categorized into different types such as vehicles, pedestrians, and animals, or further classified based on their motion characteristics, such as speed and acceleration. Different filtering strategies can be employed to remove different categories of dynamic objects. For instance, moving vehicles can be removed from the point cloud data based on their trajectory and speed information; pedestrians can be identified and removed by analyzing their shape and texture features. This approach ensures that only static environmental information is retained, improving the accuracy and stability of the map.

[0072] Feature matching is a crucial step in map stitching. When extracting feature points, methods based on geometric features such as curvature and normals, or machine learning-based feature extraction algorithms, can be used to extract representative and stable feature points from local point cloud maps. Then, feature matching algorithms, such as nearest neighbor search and RANSAC, are used to match feature points from different local point cloud maps, establishing their relative pose relationships. This process needs to consider the matching accuracy and robustness of the feature points to ensure the accuracy and continuity of the stitched global static point cloud map.

[0073] The Intermediate Point Collision (ICP) algorithm is a point cloud stitching algorithm that continuously optimizes the relative pose relationships between adjacent local point cloud maps through an iterative nearest-point algorithm until convergence. During optimization, the ICP algorithm calculates the transformation relationships between point cloud data within overlapping regions, aligning them to the same coordinate system. Simultaneously, it smooths the stitched point cloud data to reduce stitching gaps and noise, improving the map's continuity and smoothness. In this way, multiple local point cloud maps can be stitched into a complete global static point cloud map, providing a reliable environmental model for navigation and localization in autonomous driving systems.

[0074] This embodiment provides a static map construction method based on autonomous driving. It acquires 3D point cloud data from a LiDAR scanner and IMU motion information, and performs time synchronization and spatial registration between the 3D point cloud data and the IMU motion information to obtain registration data. Based on the registration data, landmark extraction and data association are performed to obtain data association results. A hybrid factor graph model is constructed based on the data association results, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights. Based on the hybrid factor graph model, vehicle pose information is jointly estimated, and a local map is constructed using the sparse incremental voxel method according to the vehicle pose information, resulting in a local point cloud map containing dynamic objects. Dynamic object filtering and map stitching are then performed on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects. Through the above method, by introducing factor graph optimization technology for pose estimation, a local point cloud map containing dynamic objects is accurately constructed. Furthermore, by combining dynamic object filtering algorithms and map stitching algorithms, the accuracy and robustness of static map construction are effectively improved, while enhancing adaptability to dynamic environments.

[0075] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S30 includes steps S301 to S303: Step S301: Construct an initial hybrid factor graph based on the data association results, wherein the initial hybrid factor graph consists of a loosely coupled factor graph and a tightly coupled factor graph with preset weights.

[0076] It should be noted that the weights of the loosely coupled and tightly coupled factor maps in the initial mixture factor map are preset and can be adjusted according to the actual situation. In this embodiment, to balance computational efficiency and estimation accuracy, different weights can be assigned to the loosely coupled and tightly coupled factor maps. For example, in road sections where vehicle movement is relatively smooth, the weight of the tightly coupled factor map can be increased to improve the accuracy of pose estimation; while in complex dynamic environments, the weight of the loosely coupled factor map can be increased to enhance adaptability to dynamic environments.

[0077] Understandably, the data association results provide correlation information between point cloud data at different times. When constructing the initial hybrid factor graph, the correlation information from the data association results can be used as edges of the factor graph, and the associated point cloud data or feature points can be used as nodes of the factor graph, thus forming a complex network structure containing multiple nodes and edges. Loosely coupled and tightly coupled factor graphs correspond to different correlation information and processing methods, respectively. By assigning them different weights, the accuracy and robustness of pose estimation can be improved while ensuring computational efficiency.

[0078] In one feasible implementation, step S301 may include: constructing a framework for a loosely coupled factor map and a tightly coupled factor map based on the association relationships and association confidence between landmarks; adding pose nodes to the frameworks of the loosely coupled factor map and the tightly coupled factor map respectively according to the current pose of the vehicle, to obtain an initial loosely coupled factor map and an initial tightly coupled factor map containing pose information, wherein each pose node represents the position and attitude of the vehicle at a certain moment; adding loosely coupled factors to the initial loosely coupled factor map to obtain a loosely coupled factor map, wherein the loosely coupled factors include at least semantic landmark observation factors, loop closure detection factors, and GNSS prior factors; adding tightly coupled factors to the initial tightly coupled factor map to obtain a tightly coupled factor map, wherein the tightly coupled factors include at least IMU pre-integration factors, laser odometry factors, and point cloud registration factors; combining the loosely coupled factor map and the tightly coupled factor map according to preset weights to obtain an initial hybrid factor map model, wherein the preset weights are set based on the association confidence between landmarks, the importance of landmark features, and the complexity of the vehicle's driving environment.

[0079] It should be noted that the data association results include the association relationships and association confidence levels between landmarks. The landmark set in the data association results is L={L1,L2,...,L...} N Each landmark contains its geometric location, semantic category, and feature descriptor. Relationships Indicates the current frame landmark L i With historical landmark L j Match successful. Association confidence level c ij∈[0,1], obtained by fusing semantic similarity, geometric consistency and topological matching scores.

[0080] Understandably, a unified graph structure framework is defined using a graph optimization library based on the relationships and confidence levels between landmarks, and frameworks for loosely coupled factor graphs and tightly coupled factor graphs are constructed respectively.

[0081] Based on the vehicle's current pose, pose nodes are added to both the loosely coupled factor graph and the tightly coupled factor graph. These pose nodes are arranged chronologically to form a pose chain representing the vehicle's trajectory, with each node representing the vehicle's position and attitude at a given moment. Simultaneously, corresponding factors are added between the pose nodes based on the relationships between landmarks to form a complete factor graph structure.

[0082] The loosely coupled factor graph primarily adds factors related to semantic landmark observation, loop closure detection, and GNSS prior. These factors are connected to corresponding landmarks through association confidence levels, thereby constructing a constraint relationship between vehicle pose and landmarks. Specifically, the semantic landmark observation factor matches landmarks based on their semantic categories and feature descriptors; the loop closure detection factor provides additional constraints by detecting whether the vehicle returns to previously visited locations; and the GNSS prior factor utilizes preliminary pose information provided by GNSS as prior knowledge to further improve the accuracy of pose estimation.

[0083] The tightly coupled factor diagram primarily incorporates factors related to IMU pre-integration, laser odometry, and point cloud registration. These factors provide more accurate pose estimation by directly processing point cloud data. Specifically, the IMU pre-integration factor utilizes the temporal continuity of IMU data to predict and integrate the vehicle's pose; the laser odometry factor calculates the vehicle's relative pose change by matching point cloud data from consecutive frames; and the point cloud registration factor further refines the vehicle's pose by optimizing the matching error between point clouds. These tightly coupled factors work together to achieve high-precision vehicle pose estimation, significantly improving the robustness of pose estimation, especially in complex dynamic environments.

[0084] The loosely coupled factor map and the tightly coupled factor map are combined according to preset weights to obtain the initial mixed factor map model. The setting of the preset weights needs to comprehensively consider multiple factors, including the confidence level of association between landmarks, the importance of landmark features, and the complexity of the vehicle's driving environment. By setting the weights appropriately, the initial mixed factor map model can exhibit good pose estimation performance under different driving environments.

[0085] Step S302: Adopt the factor graph weight adaptive algorithm based on the gradient of the dual-delay deep deterministic strategy, and dynamically adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph according to the real-time scene information during the vehicle driving process, so as to obtain the adjusted initial mixed factor graph model.

[0086] It should be noted that during vehicle operation, real-time scene information includes motion information such as vehicle speed, acceleration, and steering angle, as well as environmental perception information acquired through sensors such as LiDAR and cameras. This information reflects the vehicle's current driving status and environmental changes, serving as a crucial basis for adaptive adjustment of factor graph weights.

[0087] The dual-delay deep deterministic policy gradient algorithm is a deep learning-based reinforcement learning algorithm that optimizes policies in a continuous action space, making it suitable for handling complex dynamic environments. In this embodiment, the dual-delay deep deterministic policy gradient algorithm is applied to adaptive adjustment of factor graph weights. An actor network is trained to dynamically adjust the weights of loosely coupled and tightly coupled factor graphs.

[0088] Specifically, the actor network's input includes real-time scene information and the weights of the current mixed factor graph model, and the output is the adjusted weights. During training, a reward function is defined to evaluate the effectiveness of the weight adjustment. The reward function can be designed based on multiple metrics such as the accuracy of pose estimation, computational efficiency, and robustness to guide the actor network to learn the optimal weight adjustment strategy.

[0089] Through continuous iterative training, the actor network can gradually learn how to dynamically adjust the weights of the factor map under different driving environments to adapt to environmental changes. During actual vehicle operation, the actor network can dynamically adjust the weights based on real-time scene information, thereby obtaining more accurate pose estimation results and improving the accuracy and robustness of static map construction.

[0090] In one feasible implementation, step S302 may include: constructing a dual-delay deep deterministic policy gradient algorithm model, wherein the dual-delay deep deterministic policy gradient algorithm model includes an actor network and a critic network, the actor network being used to select actions according to the current policy, and the critic network being used to evaluate the value of the selected actions; using the weights of the loosely coupled factor map and the tightly coupled factor map in the initial mixture factor map as the action space of the dual-delay deep deterministic policy gradient algorithm model, and using the real-time scene information during vehicle movement as the state space of the dual-delay deep deterministic policy gradient algorithm model; and using the actor network to select weights to adjust actions in the action space; The critic network evaluates the value of the selected weight adjustment action in the state space to obtain a value evaluation result. Based on the value evaluation result, the policy parameters of the actor network are updated using a policy gradient update algorithm to obtain an updated actor network. A new weight adjustment action is selected again in the action space based on the updated actor network, and the critic network continuously evaluates and optimizes the value of the selected weight adjustment action in the state space until a preset convergence condition is reached to obtain the optimal weight adjustment strategy. The weights of the loosely coupled and tightly coupled factor graphs in the initial mixture factor graph are dynamically adjusted according to the optimal weight adjustment strategy to obtain the adjusted mixture factor graph model.

[0091] It should be noted that when constructing the dual-delay deep deterministic policy gradient algorithm model, the actor network is responsible for adjusting the action weights according to the current policy, that is, deciding when and how to adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph. The critic network, on the other hand, is responsible for evaluating the value of the actions selected by the actor network, guiding the actor network to optimize its policy through feedback reward signals.

[0092] The action space and state space of the dual-delay deep deterministic policy gradient algorithm model correspond to the weight adjustment ranges of the loosely coupled factor graph and the tightly coupled factor graph, respectively, as well as various scene information that the vehicle may encounter during its journey. The action space needs to be adjusted according to the actual situation to ensure that the actor network can select weight adjustment actions within a reasonable range. The state space needs to cover various scene information that the vehicle may encounter during its journey, including different road types, traffic conditions, weather conditions, etc., to ensure that the actor network can make accurate weight adjustment decisions based on real-time scene information.

[0093] During training, by iteratively allowing the actor network to select actions in the state space and having the critic network evaluate them, the network can gradually learn how to optimally adjust the factor map weights under different driving environments. This deep learning-based reinforcement learning method can adapt to complex dynamic environments and achieve high-precision estimation of vehicle pose. By employing a dual-delay deep deterministic policy gradient algorithm, the weights of the factor map can be dynamically adjusted based on real-time scene information, thereby improving the accuracy and robustness of pose estimation while ensuring computational efficiency. This adaptive adjustment strategy allows the static map construction method to better adapt to different driving environments and scene requirements.

[0094] The value assessment results reflect the contribution of the selected weight adjustment actions to improving the accuracy and robustness of pose estimation. The reward function is designed based on the value assessment results to provide positive or negative feedback to the actor network, guiding its policy optimization. For example, when the selected weight adjustment action significantly improves pose estimation accuracy, the reward function can award a higher reward; conversely, when the selected action is ineffective, a lower reward or penalty is given. This reward mechanism incentivizes the actor network to continuously learn and optimize its weight adjustment strategy.

[0095] Through continuous iterative training and adjustments, the actor network eventually learns how to optimally adjust the weights of the loosely coupled and tightly coupled factor maps under different driving environments. This adaptive adjustment capability enables the static map construction method to maintain high pose estimation performance in complex and ever-changing dynamic environments, thereby improving the accuracy and robustness of static map construction.

[0096] Step S303: Using a scene-aware, loosely coupled adaptive factor graph combination algorithm, different loosely coupled factors are dynamically selected and combined based on real-time scene information during vehicle operation to expand and optimize the adjusted initial hybrid factor graph model, thereby obtaining a hybrid factor graph model.

[0097] It should be noted that during vehicle operation, real-time scene information not only affects the weight adjustment of the factor graph but also determines which loosely coupled factors should be selected and combined into the hybrid factor graph model. Different driving environments and scene requirements have different demands on the accuracy and robustness of pose estimation. Therefore, it is necessary to dynamically select and combine appropriate loosely coupled factors based on real-time scene information to extend and optimize the hybrid factor graph model.

[0098] The scene-aware, loosely coupled adaptive factor graph combination algorithm first requires the parsing and classification of real-time scene information. This can be achieved through environmental perception information acquired by sensors such as LiDAR and cameras, including road type, traffic conditions, and weather conditions. This information reflects the vehicle's current driving environment and potential challenges.

[0099] Based on the analysis and classification of real-time scene information, the algorithm selects appropriate loosely coupled factors according to preset rules or strategies. For example, in complex dynamic environments, it may be necessary to rely more on loosely coupled factors, such as semantic landmark observation factors and loop closure detection factors, to enhance the adaptability to dynamic environments; while in stable road sections, it is possible to make more use of tightly coupled factors, such as IMU pre-integration factors and laser odometry factors, to improve the accuracy of pose estimation.

[0100] After selecting appropriate loose-coupling factors, the algorithm combines these factors into the adjusted initial mixed factor graph model to form a new mixed factor graph model. During this process, the correlation and constraints between factors need to be considered to ensure that the newly added factors can work collaboratively with the existing factors to improve the accuracy and robustness of pose estimation.

[0101] By employing a scene-aware, loosely coupled adaptive factor graph combination algorithm, appropriate loosely coupled factors can be dynamically selected and combined based on real-time scene information, thereby extending and optimizing the hybrid factor graph model. This adaptive adjustment capability enables static map construction methods to flexibly adjust and optimize the factor graph model according to different driving environments and scene requirements, thus improving the accuracy and robustness of pose estimation.

[0102] In one feasible implementation, step S303 may include: acquiring real-time scene information during vehicle operation; dynamically selecting loosely coupled and tightly coupled factors based on the real-time scene information using a rule-based decision tree algorithm, wherein each node of the decision tree algorithm represents a scene feature, each branch represents a decision path based on the scene feature, and each leaf node represents the selected loosely coupled or tightly coupled factor; constructing a temporary factor graph based on the selected loosely coupled and tightly coupled factors, wherein the temporary factor graph is composed of the selected loosely coupled and tightly coupled factors; performing consistency checks and optimizations on the temporary factor graph to obtain an optimized temporary factor graph; and fusing the optimized temporary factor graph with the adjusted initial hybrid factor graph model to obtain a hybrid factor graph model, wherein the fusion process includes merging factor graphs, updating pose nodes, and re-establishing relationships between factors.

[0103] It's important to note that the selection of scene features is crucial in rule-based decision tree algorithms. These features need to accurately reflect the vehicle's current driving environment and potential challenges so that the algorithm can make reasonable decisions. Common scene features include road type, traffic flow, driving speed, and weather conditions. By comprehensively considering these features, decision tree algorithms can dynamically select the most suitable loose coupling factor to adapt to different driving environments and scene requirements.

[0104] The construction of decision trees relies heavily on extensive real-world data and expert experience. First, vehicle driving data under various driving environments and scenarios needs to be collected, including pose estimation results and sensor data. Then, this data is used to train and optimize the decision tree, enabling it to accurately select appropriate loose-coupling factors based on real-time scene information. During training, methods such as cross-validation can be employed to evaluate the performance of the decision tree, ensuring its accuracy and reliability in practical applications.

[0105] After constructing the decision tree, it can be applied to real-time scene information analysis and factor selection during vehicle operation. As the vehicle travels in different environments, the decision tree algorithm traverses the decision path based on the current scene characteristics, eventually reaching a leaf node. This leaf node represents either the loosely coupled or tightly coupled factor that should be selected in the current scene.

[0106] After selecting suitable factors, a temporary factor graph needs to be constructed. The temporary factor graph consists of the selected loosely coupled and tightly coupled factors, and is used to temporarily store and represent the relationships between these factors. When constructing the temporary factor graph, the correlations and constraints between factors need to be considered to ensure that newly added factors can work collaboratively with existing factors.

[0107] After constructing the temporary factor graph, it needs to undergo consistency checks and optimization. This process primarily ensures that the factors and relationships in the temporary factor graph satisfy certain constraints, such as geometric consistency and topological matching. Optimization can further improve the accuracy and robustness of the temporary factor graph.

[0108] Finally, the optimized temporary factor graph is fused with the adjusted initial hybrid factor graph model to obtain the final hybrid factor graph model. The fusion process includes merging factor graphs, updating pose nodes, and re-establishing relationships between factors. Through fusion, new factors and relationships can be introduced into the hybrid factor graph model, thereby expanding and optimizing the model. This adaptive adjustment capability allows the static map construction method to better adapt to different driving environments and scenario requirements, improving the accuracy and robustness of pose estimation.

[0109] In this embodiment, the adaptive adjustment of factor graph weights is achieved by introducing a dual-delay depth deterministic strategy gradient algorithm. The scene-aware loosely coupled adaptive factor graph combination algorithm can dynamically select and combine appropriate loosely coupled factors according to real-time scene information, thereby expanding and optimizing the hybrid factor graph model. It can improve the accuracy and robustness of pose estimation according to different driving environments and scene requirements, and further improve the accuracy and robustness of static map construction.

[0110] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the static map construction method based on autonomous driving in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0111] This application also provides a static map building device based on autonomous driving, please refer to... Figure 3 The static map building device based on autonomous driving includes: The registration module 10 is used to acquire three-dimensional point cloud data of the lidar and IMU motion information, and to perform time synchronization and spatial registration of the three-dimensional point cloud data and IMU motion information to obtain registration data.

[0112] The association module 20 is used to extract landmarks and associate data based on the registration data to obtain the data association results.

[0113] The construction module 30 is used to construct a hybrid factor graph model based on the data association results, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights.

[0114] The estimation module 40 is used to jointly estimate vehicle pose information based on the hybrid factor graph model, and to construct a local map using the sparse incremental voxel method based on the vehicle pose information, thereby obtaining a local point cloud map containing dynamic objects.

[0115] The stitching module 50 is used to perform dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0116] The static map building apparatus based on autonomous driving provided in this application, employing the static map building method based on autonomous driving in the above embodiments, can solve the technical problem that existing technologies relying solely on LiDAR data to build environmental maps cannot effectively cope with interference from dynamic objects, resulting in poor accuracy of the obtained map information. Compared with the prior art, the beneficial effects of the static map building apparatus based on autonomous driving provided in this application are the same as those of the static map building method based on autonomous driving provided in the above embodiments, and other technical features in the static map building apparatus based on autonomous driving are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0117] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for constructing static maps based on autonomous driving, characterized in that, The method includes: Acquire three-dimensional point cloud data from a lidar radar and motion information from an IMU, and then synchronize and spatially register the three-dimensional point cloud data with the IMU motion information to obtain registration data. Based on the registration data, landmark extraction and data association are performed to obtain the data association results; Based on the data association results, a hybrid factor graph model is constructed, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights; Based on the hybrid factor graph model, the vehicle pose information is jointly estimated, and the local map is constructed using the sparse incremental voxel method according to the vehicle pose information, to obtain a local point cloud map containing dynamic objects. Dynamic object filtering and map stitching are performed on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

2. The method as described in claim 1, characterized in that, The process of acquiring 3D point cloud data from the lidar and motion information from the IMU, and then performing time synchronization and spatial registration between the 3D point cloud data and the IMU motion information to obtain registration data includes: Acquire 3D point cloud data from LiDAR and motion information from IMU; The three-dimensional point cloud data of the lidar is preprocessed to obtain preprocessed three-dimensional point cloud data. The point cloud preprocessing includes at least point cloud distortion correction, iterative error Kalman filtering, and noise point removal. The preprocessed 3D point cloud data is synchronized with the IMU motion information in time to obtain time synchronization data. Spatial registration is performed on the time synchronization data to obtain registered data, wherein spatial registration includes at least coarse registration based on initial pose estimation and fine registration based on iterative nearest point algorithm.

3. The method as described in claim 1, characterized in that, The process of extracting landmarks and associating data based on the registration data to obtain data association results includes: Landmark features are extracted based on the registration data, wherein the landmark features include at least geometric features and semantic features; The extracted landmark features are described to obtain feature descriptors; Based on the feature descriptors, semantic similarity matching, geometric position similarity matching, and target distance similarity matching are performed to obtain semantic similarity, geometric position similarity, and target distance similarity. A comprehensive matching score is obtained by weighting and fusing the semantic similarity, geometric position similarity, and target distance similarity. Based on the comprehensive matching score, landmark filtering and data association are performed to obtain data association results, wherein the data association results include at least the association relationship and association confidence between landmarks.

4. The method as described in claim 1, characterized in that, The construction of the mixed factor graph model based on the data association results includes: An initial hybrid factor graph is constructed based on the data association results, wherein the initial hybrid factor graph consists of a loosely coupled factor graph and a tightly coupled factor graph with preset weights; An adaptive factor graph weight algorithm based on dual-delay deep deterministic policy gradient is adopted to dynamically adjust the weights of loosely coupled and tightly coupled factor graphs in the initial mixed factor graph according to the real-time scene information during vehicle driving, so as to obtain the adjusted initial mixed factor graph model. A scene-aware, loosely coupled adaptive factor graph combination algorithm is adopted to dynamically select and combine different loosely coupled factors based on real-time scene information during vehicle operation to expand and optimize the adjusted initial hybrid factor graph model, thereby obtaining a hybrid factor graph model.

5. The method as described in claim 4, characterized in that, The construction of the initial mixture factor graph based on the data association results includes: Frameworks for constructing loosely coupled factor graphs and tightly coupled factor graphs based on the association relationships and association confidence between landmarks; Based on the vehicle's current pose, pose nodes are added to the frames of the loosely coupled factor graph and the tightly coupled factor graph respectively to obtain the initial loosely coupled factor graph and the initial tightly coupled factor graph containing pose information. Each pose node represents the vehicle's position and attitude at a certain moment. Add loose coupling factors to the initial loose coupling factor map to obtain a loose coupling factor map, wherein the loose coupling factors include at least semantic landmark observation factors, loop closure detection factors, and GNSS prior factors; Add a tight coupling factor to the initial tight coupling factor map to obtain a tight coupling factor map, wherein the tight coupling factor includes at least an IMU pre-integration factor, a laser odometry factor, and a point cloud registration factor; The loosely coupled factor map and the tightly coupled factor map are combined according to preset weights to obtain an initial hybrid factor map model. The preset weights are set based on the confidence of the association between landmarks, the importance of landmark features, and the complexity of the vehicle driving environment.

6. The method as described in claim 4, characterized in that, The method employs a factor graph weight adaptive algorithm based on a dual-delay deep deterministic policy gradient. This algorithm dynamically adjusts the weights of the loosely coupled and tightly coupled factor graphs in the initial mixed factor graph according to real-time scene information during vehicle operation, resulting in an adjusted initial mixed factor graph model. The algorithm includes: A dual-delay deep deterministic policy gradient algorithm model is constructed, wherein the dual-delay deep deterministic policy gradient algorithm model includes an actor network and a critic network, the actor network is used to select actions according to the current policy, and the critic network is used to evaluate the value of the selected actions; The weights of the loosely coupled and tightly coupled factor maps in the initial mixed factor map are used as the action space of the dual-delay deep deterministic policy gradient algorithm model, and the real-time scene information during vehicle driving is used as the state space of the dual-delay deep deterministic policy gradient algorithm model. The actor network selects weighted adjustments to actions within the motion space; The value assessment result is obtained by evaluating the value of the selected weight adjustment action in the state space based on the critic network. Based on the value assessment results, the policy parameters of the actor network are updated using the policy gradient update algorithm to obtain the updated actor network. Based on the updated actor network, a new weight adjustment action is selected again in the action space, and the value of the selected weight adjustment action is continuously evaluated and optimized in the state space by the critic network until the preset convergence condition is reached, so as to obtain the optimal weight adjustment strategy. The weights of the loosely coupled and tightly coupled factor graphs in the initial mixed factor graph are dynamically adjusted according to the optimal weight adjustment strategy to obtain the adjusted mixed factor graph model.

7. The method as described in claim 4, characterized in that, The algorithm employs a scene-aware, loosely coupled adaptive factor graph combination method. Based on real-time scene information during vehicle operation, it dynamically selects and combines different loosely coupled factors to expand and optimize the adjusted initial hybrid factor graph model, resulting in a hybrid factor graph model, including: Acquire real-time scene information during vehicle operation; Based on the real-time scene information, a rule-based decision tree algorithm is used to dynamically select loose coupling factors and tight coupling factors. Each node of the decision tree algorithm represents a scene feature, each branch represents a decision path based on the scene feature, and each leaf node represents the selected loose coupling factor or tight coupling factor. A temporary factor graph is constructed based on the selected loose coupling factor and tight coupling factor, wherein the temporary factor graph is composed of the selected loose coupling factor and tight coupling factor; The temporary factor graph is subjected to consistency checks and optimization to obtain an optimized temporary factor graph; The optimized temporary factor graph is fused with the adjusted initial hybrid factor graph model to obtain a hybrid factor graph model. The fusion process includes merging factor graphs, updating pose nodes, and re-establishing relationships between factors.

8. The method as described in claim 1, characterized in that, The process of jointly estimating vehicle pose information based on the hybrid factor graph model, and constructing a local map using the sparse incremental voxel method based on the vehicle pose information to obtain a local point cloud map containing dynamic objects, includes: Based on the hybrid factor graph model, the vehicle pose information is jointly estimated using a factor graph optimization algorithm, wherein the factor graph optimization algorithm includes at least the Gauss-Newton algorithm and the Levenberg-Marquardt algorithm. Based on the vehicle pose information, the point cloud data during the vehicle's driving process is segmented in chronological order to obtain a series of point cloud frames. Each frame of point cloud data is voxelized to generate a voxel mesh, where each voxel in the voxel mesh represents a cubic region in space. The generated voxel grid is filtered using the sparse incremental voxel method, retaining non-empty voxels containing point cloud data to obtain a sparse voxel grid. Based on the sparse voxel mesh, adjacent point cloud frames are registered and fused according to the vehicle pose information to obtain a local point cloud map. The local point cloud map is used to identify and mark dynamic objects using a machine learning-based dynamic object recognition algorithm, resulting in a local point cloud map containing dynamic objects.

9. The method as described in claim 1, characterized in that, The step of filtering and stitching the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects includes: The dynamic objects marked in the local point cloud map containing dynamic objects are identified and classified. Based on the category and motion characteristics of the dynamic objects, the corresponding filtering strategy is used to remove the dynamic objects, so as to obtain a local point cloud map without dynamic objects. Feature points in the local point cloud map without dynamic objects are extracted and matched using a feature matching-based stitching algorithm to establish the relative pose relationship between different local point cloud maps without dynamic objects. An ICP-based stitching algorithm is used to optimize the relative pose relationship obtained by feature matching through an iterative nearest-point algorithm, resulting in an optimized relative pose relationship. Based on the optimized relative pose relationship, the local point cloud maps without dynamic objects are stitched together to obtain a global static point cloud map without dynamic objects.

10. A static map building device based on autonomous driving, characterized in that, The static map building device based on autonomous driving includes: The registration module is used to acquire three-dimensional point cloud data of the lidar and IMU motion information, and to perform time synchronization and spatial registration of the three-dimensional point cloud data and IMU motion information to obtain registration data. The association module is used to extract landmarks and associate data based on the registration data to obtain the data association results; A construction module is used to construct a hybrid factor graph model based on the data association results, wherein the hybrid factor graph model consists of loosely coupled factor graphs and tightly coupled factor graphs with different weights; The estimation module is used to jointly estimate vehicle pose information based on the hybrid factor graph model, and to construct a local map using the sparse incremental voxel method based on the vehicle pose information to obtain a local point cloud map containing dynamic objects. The stitching module is used to perform dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

Images