Static map construction method and device based on autonomous driving

Abstract

The application discloses a kind of static map construction method and device based on automatic driving, comprising: laser radar three-dimensional point cloud data and IMU motion information are time synchronized and space registration, obtain registration data;Landmark extraction and data association are carried out based on registration data, obtain data association result;Mixed factor graph model is constructed based on data association result;Vehicle pose information is estimated based on mixed factor graph model and local map is constructed using sparse incremental voxel method, and local point cloud map containing dynamic object is obtained;Dynamic object filtering and map stitching are carried out to local point cloud map, and global static point cloud map without dynamic object is obtained.The application estimates pose by introducing factor graph optimization technology, accurately constructs local point cloud map containing dynamic object, and then combines dynamic object filtering and map stitching algorithm, effectively improves the accuracy and robustness of static map construction, and enhances the adaptability to dynamic environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of map construction, and particularly relates to a static map construction method and device based on automatic driving. BACKGROUND

[0002] In the field of automatic driving, environment perception and map construction are key tasks to achieve efficient navigation and positioning. With the advancement of lidar technology, three-dimensional point cloud data is increasingly applied to environment modeling. However, relying solely on lidar data to construct an environment map may not effectively cope with the interference of dynamic objects, resulting in obtained map information having strong noise and inaccuracy. SUMMARY

[0003] The main purpose of the present application is to provide a static map construction method and device based on automatic driving, aiming to solve the technical problem that the prior art of relying solely on lidar data to construct an environment map cannot effectively cope with the interference of dynamic objects, resulting in poor accuracy of obtained map information.

[0004] To achieve the above-mentioned purpose, the present application provides a static map construction method based on automatic driving, which comprises the following steps:

[0005] acquiring lidar three-dimensional point cloud data and IMU motion information, and time synchronizing and spatially registering the three-dimensional point cloud data and the IMU motion information to obtain registration data;

[0006] performing landmark extraction and data association based on the registration data to obtain a data association result;

[0007] constructing a hybrid factor graph model based on the data association result, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights;

[0008] jointly estimating vehicle pose information based on the hybrid factor graph model, and performing local map construction using a sparse incremental voxel method according to the vehicle pose information to obtain a local point cloud map containing dynamic objects;

[0009] performing dynamic object filtering and map stitching on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0010] In an embodiment, the acquiring lidar three-dimensional point cloud data and IMU motion information, and time synchronizing and spatially registering the three-dimensional point cloud data and the IMU motion information to obtain registration data comprises:

[0011] acquiring lidar three-dimensional point cloud data and IMU motion information;

[0012] performing point cloud preprocessing on the laser radar three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data, wherein the point cloud preprocessing at least includes point cloud distortion correction, iterative error Kalman filtering and noise point removal;

[0013] performing time synchronization on the preprocessed three-dimensional point cloud data and IMU motion information to obtain time synchronization data;

[0014] performing spatial registration on the time synchronization data to obtain registration data, wherein the spatial registration at least includes coarse registration based on initial pose estimation and fine registration based on an iterative closest point algorithm.

[0015] In an embodiment, the landmark extraction and data association based on the registration data are performed to obtain a data association result, including:

[0016] extracting landmark features based on the registration data, wherein the landmark features at least include geometric features and semantic features;

[0017] performing feature description on the extracted landmark features to obtain feature descriptors;

[0018] performing semantic similarity matching, geometric position similarity matching and target distance similarity matching according to the feature descriptors to obtain semantic similarity, geometric position similarity and target distance similarity;

[0019] performing weighted fusion according to the semantic similarity, geometric position similarity and target distance similarity to obtain a comprehensive matching score;

[0020] performing landmark screening and data association based on the comprehensive matching score to obtain a data association result, wherein the data association result at least includes an association relationship and an association confidence between landmarks.

[0021] In an embodiment, the mixed factor graph model is constructed based on the data association result, including:

[0022] constructing an initial mixed factor graph based on the data association result, wherein the initial mixed factor graph is composed of a loosely coupled factor graph and a tightly coupled factor graph with a preset weight;

[0023] adopting a factor graph weight self-adaptive algorithm based on a double-delay deep deterministic policy gradient to dynamically adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph according to real-time scene information in the vehicle driving process to obtain an adjusted initial mixed factor graph model;

[0024] The combination algorithm of the scene-aware loose-coupling adaptive factor graph is adopted, and the initial mixed factor graph model is expanded and optimized by dynamically selecting and combining different loose-coupling factors according to real-time scene information during vehicle driving to obtain a mixed factor graph model.

[0025] In an embodiment, the constructing the initial mixed factor graph based on the data association result comprises:

[0026] Constructing a framework of a loose-coupling factor graph and a tight-coupling factor graph based on the association relationship and the association confidence between landmarks;

[0027] Adding a pose node in the framework of the loose-coupling factor graph and the framework of the tight-coupling factor graph according to the current pose of the vehicle to obtain an initial loose-coupling factor graph and an initial tight-coupling factor graph containing pose information, wherein each pose node represents the position and attitude of the vehicle at a certain time;

[0028] Adding a loose-coupling factor in the initial loose-coupling factor graph to obtain a loose-coupling factor graph, wherein the loose-coupling factor at least includes a semantic landmark observation factor, a loop closure detection factor and a GNSS prior factor;

[0029] Adding a tight-coupling factor in the initial tight-coupling factor graph to obtain a tight-coupling factor graph, wherein the tight-coupling factor at least includes an IMU pre-integration factor, a laser odometry factor and a point cloud registration factor;

[0030] Combining the loose-coupling factor graph and the tight-coupling factor graph according to a preset weight to obtain an initial mixed factor graph model, wherein the preset weight is set according to the association confidence between landmarks, the importance of landmark features and the complexity of the vehicle driving environment.

[0031] In an embodiment, the factor graph weight adaptive algorithm based on the double-delay deep deterministic policy gradient dynamically adjusts the weights of the loose-coupling factor graph and the tight-coupling factor graph in the initial mixed factor graph according to real-time scene information during vehicle driving to obtain an adjusted initial mixed factor graph model, comprising:

[0032] Constructing a double-delay deep deterministic policy gradient algorithm model, wherein the double-delay deep deterministic policy gradient algorithm model includes an actor network and a critic network, the actor network is used to select an action according to a current policy, and the critic network is used to evaluate the value of the selected action;

[0033] Taking the weights of the loose-coupling factor graph and the tight-coupling factor graph in the initial mixed factor graph as the action space of the double-delay deep deterministic policy gradient algorithm model, and taking real-time scene information during vehicle driving as the state space of the double-delay deep deterministic policy gradient algorithm model;

[0034] select a weight adjustment action in the action space through the actor network;

[0035] evaluate a value of the selected weight adjustment action in the state space according to the critic network, to obtain a value evaluation result;

[0036] update policy parameters of the actor network based on the value evaluation result using a policy gradient update algorithm, to obtain an updated actor network;

[0037] select a new weight adjustment action in the action space according to the updated actor network, and continuously evaluate and optimize the value of the selected weight adjustment action in the state space through the critic network until a preset convergence condition is reached, to obtain an optimal weight adjustment strategy;

[0038] adjust weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph model according to the optimal weight adjustment strategy, to obtain an adjusted mixed factor graph model.

[0039] In an embodiment, the combination algorithm of the scene-aware loosely-tightly coupled adaptive factor graph dynamically selects and combines different loosely-tightly coupled factors according to real-time scene information during vehicle driving to expand and optimize the adjusted initial mixed factor graph model, to obtain a mixed factor graph model, including:

[0040] obtain real-time scene information during vehicle driving;

[0041] dynamically select a loosely coupled factor and a tightly coupled factor using a rule-based decision tree algorithm according to the real-time scene information, wherein each node of the decision tree algorithm represents a scene feature, each branch represents a decision path based on the scene feature, and each leaf node represents the selected loosely coupled factor or tightly coupled factor;

[0042] construct a temporary factor graph according to the selected loosely coupled factor and tightly coupled factor, wherein the temporary factor graph is composed of the selected loosely coupled factor and tightly coupled factor;

[0043] perform consistency checking and optimization on the temporary factor graph, to obtain an optimized temporary factor graph;

[0044] fuse the optimized temporary factor graph with the adjusted initial mixed factor graph model, to obtain a mixed factor graph model, wherein the fusion process includes merging of factor graphs, updating of pose nodes, and re-establishment of relationships between factors.

[0045] In an embodiment, the vehicle pose information is jointly estimated based on the hybrid factor graph model, and a local map containing dynamic objects is constructed based on the vehicle pose information using a sparse incremental voxel method, including:

[0046] The vehicle pose information is jointly estimated based on the hybrid factor graph model using a factor graph optimization algorithm, wherein the factor graph optimization algorithm at least includes a Gauss-Newton algorithm and a Levenberg-Marquardt algorithm.

[0047] The point cloud data during the vehicle driving process is segmented in time sequence according to the vehicle pose information, to obtain a series of point cloud frames.

[0048] The voxelization processing is performed on each frame of point cloud data to generate a voxel grid, wherein each voxel of the voxel grid represents a cubic region in space.

[0049] The generated voxel grid is screened using a sparse incremental voxel method to retain non-empty voxels containing point cloud data, to obtain a sparse voxel grid.

[0050] Based on the sparse voxel grid, the adjacent point cloud frames are registered and fused according to the vehicle pose information, to obtain a local point cloud map.

[0051] The dynamic object recognition algorithm based on machine learning is used on the local point cloud map to identify and mark the dynamic objects, to obtain a local point cloud map containing dynamic objects.

[0052] In an embodiment, the local point cloud map containing dynamic objects is filtered and spliced to obtain a global static point cloud map without dynamic objects, including:

[0053] The dynamic objects marked in the local point cloud map containing dynamic objects are identified and classified, and the dynamic objects are removed using a corresponding filtering strategy based on the category and motion characteristics of the dynamic objects, to obtain a local point cloud map without dynamic objects.

[0054] The feature points in the local point cloud map without dynamic objects are extracted and matched using a feature matching-based splicing algorithm, to establish the relative pose relationship between different local point cloud maps without dynamic objects.

[0055] The relative pose relationship obtained by feature matching is optimized by an iterative closest point (ICP) algorithm-based splicing algorithm, to obtain an optimized relative pose relationship.

[0056] The local point cloud maps without dynamic objects are spliced based on the optimized relative pose relationship, to obtain a global static point cloud map without dynamic objects.

[0057] In addition, to achieve the above object, the application further provides a static map construction device based on automatic driving, which comprises:

[0058] a registration module, configured to acquire laser radar three-dimensional point cloud data and IMU motion information, and perform time synchronization and spatial registration on the three-dimensional point cloud data and the IMU motion information to obtain registration data;

[0059] an association module, configured to perform landmark extraction and data association based on the registration data to obtain a data association result;

[0060] a construction module, configured to construct a hybrid factor graph model based on the data association result, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights;

[0061] an estimation module, configured to jointly estimate vehicle pose information based on the hybrid factor graph model, and perform local map construction by using a sparse incremental voxel method according to the vehicle pose information to obtain a local point cloud map containing dynamic objects;

[0062] a splicing module, configured to perform dynamic object filtering and map splicing on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0063] The one or more technical solutions provided by the application acquire laser radar three-dimensional point cloud data and IMU motion information, and perform time synchronization and spatial registration on the three-dimensional point cloud data and the IMU motion information to obtain registration data; perform landmark extraction and data association based on the registration data to obtain a data association result; construct a hybrid factor graph model based on the data association result, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights; jointly estimate vehicle pose information based on the hybrid factor graph model, and perform local map construction by using a sparse incremental voxel method according to the vehicle pose information to obtain a local point cloud map containing dynamic objects; and perform dynamic object filtering and map splicing on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects. In this way, by introducing a factor graph optimization technique for pose estimation, a local point cloud map containing dynamic objects is accurately constructed, and then a dynamic object filtering algorithm and a map splicing algorithm are combined, so that the accuracy and robustness of static map construction are effectively improved, and the adaptability to dynamic environments is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, the other drawings can also be obtained based on these drawings without any creative effort.

[0066] Figure 1 A flowchart provided by the present application for the first embodiment of the method for constructing a static map based on autonomous driving;

[0067] Figure 2 A flowchart provided by the present application for the second embodiment of the method for constructing a static map based on autonomous driving;

[0068] Figure 3 A module structure diagram of the device for constructing a static map based on autonomous driving according to the embodiments of the present application.

[0069] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0070] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0071] In order to better understand the technical solutions of the present application, the following will be described in detail with reference to the drawings and specific embodiments in the specification.

[0072] The main solution of the embodiments of the present application is: obtaining laser radar three-dimensional point cloud data and IMU motion information, and time synchronizing and spatially registering the three-dimensional point cloud data and the IMU motion information to obtain registration data; performing landmark extraction and data association based on the registration data to obtain a data association result; constructing a hybrid factor graph model based on the data association result, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights; jointly estimating vehicle pose information based on the hybrid factor graph model, and constructing a local map containing dynamic objects by using a sparse incremental voxel method according to the vehicle pose information; filtering dynamic objects from the local point cloud map containing dynamic objects and splicing the map to obtain a global static point cloud map without dynamic objects.

[0073] Relying only on laser radar data to construct an environment map may not effectively deal with the interference of dynamic objects, resulting in strong noise and inaccuracy of the obtained map information.

[0074] The application provides a solution for pose estimation by introducing factor graph optimization technology, accurately constructing a local point cloud map containing dynamic objects, and then combining a dynamic object filtering algorithm and a map stitching algorithm to effectively improve the accuracy and robustness of static map construction while enhancing the adaptability to dynamic environments.

[0075] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a static map construction device based on autonomous driving, etc. In the following, the embodiment and the following embodiments are described by taking the static map construction device based on autonomous driving as an example.

[0076] Based on this, the embodiment of the application provides a static map construction method based on autonomous driving, with reference to Figure 1 , Figure 1 The flowchart of the first embodiment of the static map construction method based on autonomous driving of the application is shown in the figure.

[0077] In the embodiment, the static map construction method based on autonomous driving includes steps S10-S50:

[0078] Step S10: Obtain laser radar three-dimensional point cloud data and IMU motion information, and time synchronize and spatially register the three-dimensional point cloud data and the IMU motion information to obtain registration data.

[0079] It should be noted that the laser radar three-dimensional point cloud data is obtained by scanning the surrounding environment by a laser radar, and contains rich spatial information such as the position and shape of an object. The IMU motion information is obtained by an inertial measurement unit, and contains dynamic information such as the acceleration and angular velocity of a vehicle. Time synchronization and spatial registration are to align the two kinds of data to the same time reference system and spatial reference system to ensure the accuracy and consistency of the data.

[0080] It can be understood that before time synchronizing and spatially registering the three-dimensional point cloud data and the IMU motion information, the three-dimensional point cloud data needs to be preprocessed, such as denoising, filtering, etc., to improve the quality of the data.

[0081] It is worth noting that time synchronization is usually achieved by timestamp matching to ensure that each data point has an accurate time label. Spatial registration involves complex coordinate transformation and calibration processes to eliminate the deviations and errors between different sensors.

[0082] In an implementable embodiment, step S10 can include: acquiring laser radar three-dimensional point cloud data and IMU motion information; performing point cloud preprocessing on the laser radar three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data, wherein the point cloud preprocessing at least includes point cloud distortion correction, iterative error Kalman filtering, and noise point removal; time synchronizing the preprocessed three-dimensional point cloud data and the IMU motion information to obtain time synchronization data; and performing spatial registration on the time synchronization data to obtain registration data, wherein the spatial registration at least includes coarse registration based on initial pose estimation and fine registration based on an iterative closest point algorithm

[0083] It should be noted that in the present embodiment, the point cloud preprocessing includes point cloud distortion correction, iterative error Kalman filtering, and noise point removal. The point cloud distortion correction can eliminate the distortion of the point cloud data caused by uneven laser radar scanning speed or vehicle motion; the iterative error Kalman filtering can smooth the point cloud data and reduce noise interference; and the noise point removal can further clean up the data and improve the accuracy of subsequent processing. The point cloud distortion correction can be performed by geometric transformation on the original point cloud data obtained by laser radar scanning to restore its true spatial position. The iterative error Kalman filtering is a recursive algorithm that estimates the optimal value of the state variable by predicting and updating the system dynamic model and the observation model, thereby achieving smoothing processing of the point cloud data.

[0084] It can be understood that the time synchronization is achieved by timestamp matching, specifically including: matching the timestamps of the laser radar three-dimensional point cloud data and the timestamps of the IMU motion information, aligning the data with similar timestamps, ensuring that each data point has an accurate time label, thereby obtaining time synchronization data, and ensuring the consistency of the laser radar data and the IMU data in time.

[0085] The spatial registration is to align the laser radar three-dimensional point cloud data and the IMU motion information to the same spatial reference frame through coordinate transformation and calibration process on the basis of time synchronization, so as to eliminate the deviation and error between different sensors, thereby obtaining registration data. The spatial registration includes two steps of coarse registration and fine registration, which gradually reduces the deviation between the data of different sensors, and finally obtains high-precision registration data.

[0086] The coarse registration based on the initial pose estimation can quickly narrow down the spatial deviation between different sensor data, and the fine registration based on the iterative closest point algorithm can further accurately adjust the position of the data to achieve high-precision spatial alignment. Specifically, it includes: based on the initial pose estimation, the three-dimensional point cloud data of the laser radar and the IMU motion information are roughly aligned by using the transformation matrix; then, the iterative closest point algorithm is adopted, the transformation matrix is continuously iteratively optimized, the spatial deviation between the two kinds of data is minimized, until the preset convergence condition is met or the maximum number of iterations is reached, thereby completing the fine registration process.

[0087] Step S20: landmark extraction and data association based on the registration data, to obtain a data association result.

[0088] It should be noted that landmark extraction refers to identifying environment elements with distinctive features from the registration data, such as road signs, building edges, etc., which play a key role in positioning reference in map construction. Data association is to match these landmarks with previous map data or sensor data to establish temporal continuity and spatial consistency, ensuring the accuracy and continuity of map construction. Through landmark extraction and data association, a preliminary estimation of the vehicle pose can be achieved, providing a basis for subsequent graph optimization and map construction.

[0089] In a feasible implementation, step S20 can include: extracting landmark features based on the registration data, wherein the landmark features at least include geometric features and semantic features; performing feature description on the extracted landmark features to obtain feature descriptors; performing semantic similarity matching, geometric position similarity matching and target distance similarity matching according to the feature descriptors to obtain semantic similarity, geometric position similarity and target distance similarity; performing weighted fusion according to the semantic similarity, geometric position similarity and target distance similarity to obtain a comprehensive matching score; performing landmark screening and data association based on the comprehensive matching score to obtain a data association result, wherein the data association result at least includes the association relationship and the association confidence between landmarks.

[0090] It should be noted that landmark features refer to information that can uniquely identify or significantly distinguish different landmarks, including geometric features and semantic features. Geometric features describe the shape, size, direction and other spatial attributes of landmarks, and are extracted from the point cloud data after registration, including corner points, line segments, planar regions, cylindrical bodies, etc., wherein line segments such as street lamp poles, traffic sign poles, planar regions such as building facades, walls, cylindrical bodies such as lamp poles, power poles, etc. The extracted geometric features include: position coordinates (x, y, z), normal vector, curvature, size (length, width, height), main direction, etc.

[0091] Semantic features express the high-level information of landmarks, such as categories and functions. The registered data are classified by a trained 3D semantic segmentation network, such as PointNet++, PVCNN, MinkowskiNet, etc., and each point or clustered object is labeled with a semantic label. The semantic categories of landmarks include "traffic light", "road sign", "street lamp", "electric pole", "building corner", etc. Each landmark object obtains a semantic label and its confidence.

[0092] Feature description is the process of converting landmark features into mathematical representations, i.e., constructing a joint feature descriptor based on the extracted landmark features, in the form of a vector, as follows:

[0093] F = [Fgeo, Fsem]

[0094] where F is the feature descriptor, Fgeo is the geometric descriptor, such as SHOT, FPFH, or custom geometric parameter encoding, and Fsem is the semantic encoding, such as one-hot vector + confidence.

[0095] Semantic similarity matching focuses on the consistency of landmarks in categories and functions, geometric position similarity matching considers the proximity of landmarks in spatial position, and target distance similarity matching further considers the relative distance relationship between landmarks. By weighted fusion of the three similarities, a comprehensive matching score can be obtained, which reflects the accuracy and reliability of the matching between landmarks. Based on the comprehensive matching score, landmark screening and data association can be performed to eliminate mis-matches and low-confidence landmarks, and to retain high-quality data association results, providing accurate and reliable input for subsequent graph optimization and map construction.

[0096] The purpose of semantic similarity matching is to quickly exclude obviously mismatched objects and narrow the search space. The semantic similarity between two landmarks L i and L j is calculated as follows:

[0097] ;

[0098] where, is the semantic similarity between landmarks L i and L j , and a ∈ (0, 1) is a pre-set approximate category similarity weight.

[0099] The purpose of geometric position similarity matching is to determine whether two landmarks are close in spatial position. The Euclidean distance or Manhattan distance between landmarks is calculated to measure the proximity. The Euclidean distance between the center coordinates of two landmarks is calculated as follows:

[0100] ;

[0101] where, L i and L j , , L i and L j .

[0102] If is less than a preset distance threshold, it is considered that the positions are similar and the position similarity is high. It can be normalized to a similarity score as follows:

[0103] ;

[0104] wherein is the geometric position similarity of L i and L j , is a control decay rate.

[0105] The target distance similarity matching, i.e. the topological structure matching, aims to improve the matching robustness by using the relative spatial relationship or configuration between landmarks. For each landmark L i , its neighborhood graph is constructed: the k nearest neighbors in the surrounding are selected according to the distance ranking, the relative distance d ij , the relative azimuth angle θ ij of each neighbor and the semantic type of the neighbor are determined; the local topological descriptor is defined: for example, "there is a traffic sign 1.5 m east of a certain street lamp and a power pole 2.3 m north of the street lamp"; when matching, the neighborhood structures of the two landmarks are compared, and the Hausdorff distance or graph matching algorithm is used to calculate the topological similarity, i.e. the target distance similarity .

[0106] The semantic similarity , the geometric position similarity , the target distance similarity are weighted and fused to obtain a comprehensive matching score :

[0107] ;

[0108] wherein , , are the weight coefficients of the semantic similarity , the geometric position similarity , the target distance similarity , , , satisfy + + =1.

[0109] based on the comprehensive matching score , landmark screening is performed, if is greater than a preset score threshold, it is determined that the landmarks L i and L j are the same physical entity, data association is completed, and the associated landmarks and their association confidence are output as data association results.

[0110] Step S30: constructing a hybrid factor graph model based on the data association results, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights.

[0111] It should be noted that the hybrid factor graph model is a graph model used to represent variables and their mutual relationships. In the map construction of autonomous driving, the hybrid factor graph model can integrate data from different sensors, including lidar and IMU, as well as landmark features and data association results, thereby achieving high-precision estimation of vehicle pose and map features.

[0112] In the hybrid factor graph model, the loosely coupled factor graph and the tightly coupled factor graph are balanced through weights to adapt to different application scenarios and data characteristics. The weights can be adjusted according to factors such as data quality, sensor accuracy, and reliability of landmark features to achieve the best map construction effect.

[0113] By constructing a hybrid factor graph model and using a graph optimization algorithm to solve it, high-precision estimation of vehicle pose and map features can be achieved. This helps the autonomous driving system to achieve stable navigation and positioning in complex environments, improving the safety and reliability of autonomous driving.

[0114] Step S40: jointly estimating vehicle pose information based on the hybrid factor graph model, and constructing a local map containing dynamic objects using a sparse incremental voxel method based on the vehicle pose information.

[0115] It should be noted that vehicle pose information refers to the position and orientation of a vehicle in three-dimensional space, including the coordinates (x, y, z) and attitude of the vehicle, such as pitch angle, yaw angle, and roll angle. Jointly estimating vehicle pose information means considering information from lidar, IMU, and landmark features and data association results to accurately estimate vehicle pose through a graph optimization algorithm.

[0116] Sparse incremental voxel method is an efficient point cloud map construction method. By voxelizing the point cloud data, the space is divided into a series of small cubes, i.e. voxels. Only the representative points or feature points in each voxel are retained, thereby greatly reducing the data volume and computational complexity. At the same time, the map is incrementally updated, only adding newly observed point cloud data, maintaining the sparsity and real-time nature of the map.

[0117] When constructing a local point cloud map, considering that there may be dynamic objects in the environment, such as moving vehicles, pedestrians, etc., these dynamic objects will affect the accuracy and stability of the map. Therefore, when constructing a local point cloud map, an effective method needs to be adopted to process and eliminate these dynamic objects. For example, by analyzing the motion characteristics and time continuity of the point cloud data, dynamic objects can be identified and labeled. In the subsequent map construction process, these labeled dynamic objects are removed from the point cloud data to reduce their impact on the map. In this way, a more accurate and stable local point cloud map can be constructed, providing more reliable environmental information for the navigation and positioning of autonomous driving systems.

[0118] In a feasible implementation, step S40 can include: based on the hybrid factor graph model, jointly estimating vehicle pose information using a factor graph optimization algorithm, wherein the factor graph optimization algorithm at least includes Gauss-Newton algorithm and Levenberg-Marquardt algorithm; according to the vehicle pose information, segmenting the point cloud data in the vehicle driving process in time sequence to obtain a series of point cloud frames; voxelizing each frame of point cloud data to generate a voxel grid, wherein each voxel of the voxel grid represents a cubic region in space; using a sparse incremental voxel method to screen the generated voxel grid, retaining non-empty voxels containing point cloud data to obtain a sparse voxel grid; based on the sparse voxel grid, registering and fusing adjacent point cloud frames according to the vehicle pose information to obtain a local point cloud map; using a machine learning-based dynamic object recognition algorithm to identify and label dynamic objects in the local point cloud map to obtain a local point cloud map containing dynamic objects.

[0119] It should be noted that the hybrid factor graph model is composed of edges and nodes, wherein the nodes represent landmarks, vehicle poses, etc. in the map, and the edges represent the mutual relationship or constraint between these variables, including the time continuity constraint between vehicle poses, the spatial constraint between landmark feature points, and the observation constraint between vehicle poses and landmark feature points. The loosely coupled factor graph is mainly used to represent the time continuity constraint between vehicle poses and the spatial constraint between landmark feature points. These constraints are relatively loose and allow a certain error range. The tightly coupled factor graph is used to represent the observation constraint between vehicle poses and landmark feature points. These constraints are relatively strict and require accurate matching.

[0120] In the process of jointly estimating vehicle pose information based on the hybrid factor graph model, the factor graph optimization algorithm continuously iteratively adjusts the positions of vehicle pose and landmark feature points to minimize the overall error function. The error function usually includes observation error, time continuity error, and spatial constraint error, etc. By weighted sum of these errors, a comprehensive error index can be obtained. The goal of the factor graph optimization algorithm is to find the vehicle pose and landmark feature point positions that minimize this comprehensive error.

[0121] In the Gauss-Newton algorithm, the error function is approximated as a quadratic function, and then the positions of vehicle pose and landmark feature points are updated by solving a linear equation set. The Levenberg-Marquardt algorithm adds a trust region to the Gauss-Newton algorithm to control the step size of iteration, avoiding algorithm divergence or falling into local optimal solution.

[0122] By jointly estimating vehicle pose information, the accurate trajectory of the vehicle during driving can be obtained. At the same time, the factor graph optimization algorithm based on the hybrid factor graph model can also effectively handle sensor noise and data uncertainty, improving the robustness and accuracy of map construction.

[0123] After obtaining the vehicle pose information, the next step is to segment the point cloud data of the vehicle driving process in chronological order to obtain a series of point cloud frames. Each frame of point cloud data contains the environmental information observed by the vehicle at a certain time, which is the basis for constructing a local point cloud map.

[0124] By voxelizing each frame of point cloud data, it can be divided into a series of small cubic regions, i.e. voxels, and only one representative point or feature point is retained in each voxel. This can greatly reduce the data volume and computational complexity of point cloud data, improving the efficiency and real-time performance of map construction. Specifically, define the voxel size v s ×v s ×v s , such as 0.1m×0.1m×0.1m, divide the three-dimensional space into regular grids, and each cubic region is called a voxel Voxel. For each point pk in each frame of point cloud Pt, calculate its voxel index:

[0125] ;

[0126] where, 、 、 represents the three-dimensional coordinates of the kth point in the point cloud, indicates the size of the voxel edge length.

[0127] Constructing a hash table or a three-dimensional array to store all non-empty voxels can avoid storing a large number of empty voxels, thereby improving memory efficiency and processing speed.

[0128] In the sparse incremental voxel method, only non-empty voxels containing point cloud data are retained to generate a sparse voxel grid. Then, based on the sparse voxel grid and vehicle pose information, adjacent point cloud frames are registered and fused to obtain a local point cloud map. The registration process is to align the adjacent point cloud frames to the same coordinate system by calculating the transformation relationship between them. The fusion process is to merge the registered point cloud data to generate a continuous local point cloud map.

[0129] However, when constructing a local point cloud map, dynamic objects that may exist in the environment can affect the accuracy and stability of the map. Therefore, an effective method is needed to identify and mark these dynamic objects. A dynamic object identification algorithm based on machine learning can automatically identify and mark dynamic objects by analyzing the shape, texture, motion characteristics, and other information of the point cloud data. In the subsequent map construction process, these marked dynamic objects are excluded from the point cloud data, which can reduce their impact on the map and improve the accuracy and stability of the map.

[0130] By combining the sparse incremental voxel method and the dynamic object identification algorithm, a local point cloud map containing dynamic objects can be constructed. This map not only contains static environmental information but also contains the position and motion information of dynamic objects, providing more rich and reliable environmental perception information for the navigation and positioning of autonomous driving systems.

[0131] Step S50: Dynamic object filtering and map stitching are performed on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0132] It should be noted that the global static point cloud map is an important environmental model required by autonomous driving systems, which provides static environmental information during vehicle travel, including roads, buildings, trees, ice cream cones, etc. When constructing a global static point cloud map, the local point cloud map containing dynamic objects needs to be further processed to exclude dynamic objects and stitch multiple local maps into a complete global map.

[0133] Dynamic object filtering is based on the identified and marked dynamic objects. After obtaining the local point cloud map containing dynamic objects, the marked dynamic objects can be directly excluded from the point cloud data, leaving only static environmental information. This can reduce the impact of dynamic objects on the map and improve the accuracy and stability of the map.

[0134] Map stitching is the process of combining multiple local point cloud maps along the vehicle's trajectory to generate a complete global static point cloud map. During the stitching process, the overlapping regions between adjacent local maps need to be considered. By calculating the transformation relationship between the point cloud data in the overlapping region, they can be aligned in the same coordinate system. At the same time, the stitched point cloud data needs to be smoothed to reduce the stitching gap and noise, and to improve the continuity and smoothness of the map.

[0135] In a feasible implementation, step S50 can include: identifying and classifying the dynamic objects marked in the local point cloud map containing dynamic objects, removing dynamic objects based on the category and motion characteristics of dynamic objects using a corresponding filtering strategy to obtain a local point cloud map without dynamic objects; using a feature matching-based stitching algorithm to extract feature points in the local point cloud map without dynamic objects and perform matching to establish the relative pose relationship between different local point cloud maps without dynamic objects; using an ICP algorithm-based stitching algorithm to optimize the relative pose relationship obtained by feature matching through the iterative closest point algorithm to obtain an optimized relative pose relationship; and stitching the local point cloud maps without dynamic objects based on the optimized relative pose relationship to obtain a global static point cloud map without dynamic objects.

[0136] It should be noted that in the process of dynamic object filtering, identifying and classifying dynamic objects is a key step. The dynamic object recognition algorithm based on machine learning can automatically analyze the features of point cloud data, distinguish dynamic objects from static environmental information, and accurately classify them. For example, dynamic objects can be classified into different categories such as vehicles, pedestrians, and animals, or more detailed classification can be performed according to the motion characteristics of dynamic objects such as speed and acceleration. Different filtering strategies can be used to remove different categories of dynamic objects. For example, for vehicles in motion, their motion trajectory and speed information can be used to remove them from the point cloud data; for pedestrians, their shape and texture features can be analyzed to identify and remove them. In this way, only static environmental information is retained, improving the accuracy and stability of the map.

[0137] Feature matching is an important part of the map stitching process. When extracting feature points, methods based on curvature, normal, and other geometric features, or machine learning-based feature extraction algorithms can be used to extract representative and stable feature points from local point cloud maps. Then, through feature matching algorithms such as nearest neighbor search and RANSAC algorithm, the feature points in different local point cloud maps are matched to establish their relative pose relationship. This process needs to consider the matching accuracy and robustness of feature points to ensure the accuracy and continuity of the stitched global static point cloud map.

[0138] The ICP algorithm is a point cloud splicing algorithm. It continuously optimizes the relative pose relationship between adjacent local point cloud maps through the iterative closest point algorithm until convergence. During the optimization process, the ICP algorithm calculates the transformation relationship between the point cloud data in the overlapping area and aligns them to the same coordinate system. At the same time, the spliced point cloud data is also smoothed to reduce the splicing gap and noise and improve the continuity and smoothness of the map. In this way, multiple local point cloud maps can be spliced into a complete global static point cloud map, providing a reliable environment model for the navigation and positioning of the autonomous driving system.

[0139] The embodiment provides a static map construction method based on automatic driving. Three-dimensional point cloud data and IMU motion information of a laser radar are acquired, and the three-dimensional point cloud data and the IMU motion information are time-synchronized and space-registered to obtain registration data. Landmark extraction and data association are performed based on the registration data to obtain a data association result. A hybrid factor graph model is constructed based on the data association result, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights. Vehicle pose information is jointly estimated based on the hybrid factor graph model, and local map construction is performed based on the vehicle pose information using a sparse incremental voxel method to obtain a local point cloud map containing dynamic objects. Dynamic object filtering and map splicing are performed on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects. Through the above method, the factor graph optimization technology is introduced for pose estimation, the local point cloud map containing dynamic objects is accurately constructed, and then the dynamic object filtering algorithm and the map splicing algorithm are combined to effectively improve the accuracy and robustness of the static map construction while enhancing the adaptability to the dynamic environment.

[0140] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and the following will not be repeated. On this basis, please refer to Figure 2 , step S30 includes steps S301-S303:

[0141] Step S301: Construct an initial hybrid factor graph based on the data association result, wherein the initial hybrid factor graph is composed of loosely coupled factor graphs and tightly coupled factor graphs with preset weights.

[0142] It should be noted that the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph are pre-set and can be adjusted according to actual conditions. In this embodiment, in order to balance the calculation efficiency and the estimation accuracy, different weights can be given to the loosely coupled factor graph and the tightly coupled factor graph. For example, in a road section where the vehicle travels relatively smoothly, the weight of the tightly coupled factor graph can be increased to improve the accuracy of the pose estimation; and in a complex dynamic environment, the weight of the loosely coupled factor graph can be increased to enhance the adaptability to the dynamic environment.

[0143] It can be understood that the data association result provides association information between point cloud data at different time points. When the initial mixed factor graph is constructed, the association information in the data association result can be taken as an edge of the factor graph, and the associated point cloud data or feature points can be taken as nodes of the factor graph, so as to form a complex network structure including multiple nodes and edges. The loosely coupled factor graph and the tightly coupled factor graph correspond to different association information and processing modes respectively, and by giving them different weights, the accuracy and robustness of the pose estimation can be improved while ensuring the calculation efficiency.

[0144] In a feasible implementation, the step S301 can include: constructing frameworks of the loosely coupled factor graph and the tightly coupled factor graph based on the association relationship and the association confidence between landmarks; adding pose nodes in the frameworks of the loosely coupled factor graph and the tightly coupled factor graph respectively according to the current pose of the vehicle, to obtain an initial loosely coupled factor graph and an initial tightly coupled factor graph containing pose information, wherein each pose node represents the position and attitude of the vehicle at a time point; adding loosely coupled factors in the initial loosely coupled factor graph to obtain the loosely coupled factor graph, wherein the loosely coupled factors at least include semantic landmark observation factors, loop closure detection factors and GNSS prior factors; adding tightly coupled factors in the initial tightly coupled factor graph to obtain the tightly coupled factor graph, wherein the tightly coupled factors at least include IMU pre-integration factors, laser odometry factors and point cloud registration factors; and combining the loosely coupled factor graph and the tightly coupled factor graph according to a pre-set weight to obtain an initial mixed factor graph model, wherein the pre-set weight is set according to the association confidence between landmarks, the importance of landmark features and the complexity of the vehicle driving environment.

[0145] It should be noted that the data association result includes the association relationship and the association confidence between landmarks, and the landmark set in the data association result is L={L1, L2,..., L N} and each landmark contains a geometric position, a semantic category and a feature descriptor. The association relationship represents that the landmark L i in the current frame is successfully matched with the landmark L j in the history. The association confidence c ij∈[0, 1], which is obtained by fusing semantic similarity, geometric consistency and topological matching scores.

[0146] It can be understood that a unified graph structure framework is defined using a graph optimization library according to the association relationship and the association confidence between landmarks, and the frameworks of the loosely coupled factor graph and the tightly coupled factor graph are respectively constructed.

[0147] According to the current pose of the vehicle, pose nodes are added in the framework of the loosely coupled factor graph and the framework of the tightly coupled factor graph respectively, wherein the pose nodes are arranged in time sequence to form a pose chain representing a driving track of the vehicle, and each pose node represents a position and an attitude of the vehicle at a time. While adding the pose nodes, corresponding factors are also added between the pose nodes according to the association relationship between the landmarks to form a complete factor graph structure.

[0148] In the loosely coupled factor graph, mainly added are factors related to semantic landmark observation, loop closure detection and GNSS prior. These factors are connected with corresponding landmarks through association confidence, thereby constructing a constraint relationship between the vehicle pose and the landmarks. Among them, the semantic landmark observation factor is matched according to the semantic category and the feature descriptor of the landmark, the loop closure detection factor provides additional constraints by detecting whether the vehicle returns to a previously passed place, and the GNSS prior factor uses the preliminary pose information provided by the GNSS as prior knowledge to further improve the accuracy of pose estimation.

[0149] In the tightly coupled factor graph, mainly added are factors related to IMU pre-integration, laser odometry and point cloud registration. These factors provide more accurate pose estimation by directly processing point cloud data. Among them, the IMU pre-integration factor uses the continuity of IMU data in time to predict and integrate the pose of the vehicle, the laser odometry factor calculates the relative pose change of the vehicle by matching the point cloud data of consecutive frames, and the point cloud registration factor further accurately positions the pose of the vehicle by optimizing the matching error between point clouds. These tightly coupled factors jointly act to achieve high-precision estimation of the vehicle pose, especially in complex dynamic environments, which can significantly improve the robustness of pose estimation.

[0150] The loosely coupled factor graph and the tightly coupled factor graph are combined according to a preset weight to obtain an initial mixed factor graph model. The setting of the preset weight needs to consider various factors, including the association confidence between landmarks, the importance of landmark features and the complexity of the driving environment of the vehicle. By reasonably setting the weight, the initial mixed factor graph model can exhibit good pose estimation performance in different driving environments.

[0151] Step S302: using a double-delay deep deterministic policy gradient-based factor graph weight adaptive algorithm, dynamically adjusting the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph according to real-time scene information in the vehicle driving process, to obtain an adjusted initial mixed factor graph model.

[0152] It should be noted that, in the vehicle driving process, the real-time scene information includes motion information such as the driving speed, acceleration, and steering angle of the vehicle, as well as environmental perception information obtained through sensors such as lidar and camera. These information can reflect the current driving state and environmental changes of the vehicle, and are important basis for realizing adaptive adjustment of factor graph weights.

[0153] The double-delay deep deterministic policy gradient algorithm is a deep learning-based reinforcement learning algorithm that can optimize policies in continuous action space and is suitable for handling complex dynamic environments. In this embodiment, the double-delay deep deterministic policy gradient algorithm is applied to adaptive adjustment of factor graph weights, and an actor network is trained to dynamically adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph.

[0154] Specifically, the input of the actor network includes real-time scene information and the weights of the current mixed factor graph model, and the output is the adjusted weights. In the training process, a reward function is defined to evaluate the effect of weight adjustment. The reward function can be designed according to multiple indicators such as the accuracy of pose estimation, computational efficiency, and robustness, to guide the actor network to learn the optimal weight adjustment strategy.

[0155] Through continuous iterative training, the actor network can gradually learn how to dynamically adjust the weights of the factor graph in different driving environments to adapt to changes in the environment. In the actual driving process of the vehicle, the actor network can dynamically adjust the weights according to real-time scene information, thereby obtaining more accurate pose estimation results and improving the accuracy and robustness of static map construction.

[0156] In an implementable embodiment, step S302 can include: constructing a double-delay deep deterministic policy gradient algorithm model, wherein the double-delay deep deterministic policy gradient algorithm model includes an actor network and a critic network, the actor network is used to select an action according to a current policy, and the critic network is used to evaluate the value of the selected action; taking the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial hybrid factor graph as the action space of the double-delay deep deterministic policy gradient algorithm model, and taking the real-time scene information in the vehicle driving process as the state space of the double-delay deep deterministic policy gradient algorithm model; selecting a weight adjustment action in the action space through the actor network; evaluating the value of the selected weight adjustment action in the state space according to the critic network to obtain a value evaluation result; updating the policy parameter of the actor network based on the value evaluation result by using a policy gradient update algorithm to obtain an updated actor network; selecting a new weight adjustment action in the action space again according to the updated actor network, and continuously evaluating and optimizing the value of the selected weight adjustment action in the state space through the critic network until a preset convergence condition is reached to obtain an optimal weight adjustment strategy; and dynamically adjusting the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial hybrid factor graph according to the optimal weight adjustment strategy to obtain an adjusted hybrid factor graph model.

[0157] It should be noted that when the double-delay deep deterministic policy gradient algorithm model is constructed, the actor network is responsible for selecting a weight adjustment action according to a current policy, that is, deciding when and how to adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph. The critic network is responsible for evaluating the value of the action selected by the actor network, and guiding the actor network to optimize the policy through a feedback reward signal.

[0158] The action space and the state space of the double-delay deep deterministic policy gradient algorithm model correspond to the weight adjustment range of the loosely coupled factor graph and the tightly coupled factor graph, respectively, and various scene information that can be encountered in the vehicle driving process. The setting of the action space needs to be adjusted according to the actual situation to ensure that the actor network can select a weight adjustment action within a reasonable range. The setting of the state space needs to cover various scene information that can be encountered in the vehicle driving process, including different road types, traffic conditions, weather conditions, etc., to ensure that the actor network can make accurate weight adjustment decisions according to real-time scene information.

[0159] During the training process, the actor network is iteratively allowed to select actions in the state space, and the critic network evaluates them. This way, the optimal adjustment of factor graph weights in different driving environments can be gradually learned. This deep learning-based reinforcement learning method can adapt to complex dynamic environments and achieve high-precision vehicle pose estimation. By using the double-delay deep deterministic policy gradient algorithm, the weights of the factor graph can be dynamically adjusted based on real-time scene information, thereby further improving the accuracy and robustness of pose estimation while ensuring computational efficiency. This adaptive adjustment strategy enables the static map construction method to better adapt to different driving environments and scene requirements.

[0160] The value evaluation result reflects the contribution of the selected weight adjustment action to improving the accuracy and robustness of pose estimation. The reward function is designed based on the value evaluation result to give positive or negative feedback to the actor network to guide its policy optimization. For example, when the selected weight adjustment action significantly improves the accuracy of pose estimation, the reward function can give a higher reward; when the selected action is not effective, a lower reward or penalty is given. This reward mechanism can encourage the actor network to continuously learn and optimize its weight adjustment strategy.

[0161] Through continuous iterative training and adjustment, the actor network can ultimately learn how to optimally adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph in different driving environments. This adaptive adjustment capability enables the static map construction method to maintain high pose estimation performance in complex and dynamic environments, thereby improving the accuracy and robustness of static map construction.

[0162] Step S303: A combination algorithm of scene-aware adaptive loosely and tightly coupled factor graphs is used to dynamically select and combine different loosely and tightly coupled factors to expand and optimize the adjusted initial hybrid factor graph model based on real-time scene information during vehicle driving, obtaining a hybrid factor graph model.

[0163] It should be noted that during vehicle driving, real-time scene information not only affects the weight adjustment of the factor graph, but also determines which loosely and tightly coupled factors should be selected and combined into the hybrid factor graph model. Different driving environments and scene requirements have different requirements for the accuracy and robustness of pose estimation. Therefore, appropriate loosely and tightly coupled factors need to be dynamically selected and combined based on real-time scene information to achieve the expansion and optimization of the hybrid factor graph model.

[0164] The scene-aware loose-tight coupling adaptive factor graph combination algorithm first needs to analyze and classify real-time scene information. This can be achieved through environmental perception information obtained by sensors such as lidar and cameras, including road type, traffic conditions, weather conditions, etc. These information can reflect the current driving environment and possible challenges faced by the vehicle.

[0165] Based on the analysis and classification of real-time scene information, the algorithm will select the corresponding loose-tight coupling factor according to the preset rules or strategies. For example, in a complex dynamic environment, more reliance on loose coupling factors such as semantic landmark observation factors and loop closure detection factors may be needed to enhance the adaptability to dynamic environments; while on a smooth road segment, more tight coupling factors such as IMU pre-integration factors and laser odometry factors can be used to improve the accuracy of pose estimation.

[0166] After selecting the appropriate loose-tight coupling factor, the algorithm will combine these factors into the adjusted initial mixed factor graph model to form a new mixed factor graph model. In this process, the relevance and constraint relationship between factors need to be considered to ensure that the newly added factors can work together with the original factors to improve the accuracy and robustness of pose estimation.

[0167] By using the scene-aware loose-tight coupling adaptive factor graph combination algorithm, appropriate loose-tight coupling factors can be dynamically selected and combined according to real-time scene information, realizing the expansion and optimization of the mixed factor graph model. This adaptive adjustment capability enables the static map construction method to flexibly adjust and optimize the factor graph model according to different driving environments and scene requirements, thereby improving the accuracy and robustness of pose estimation.

[0168] In a feasible implementation, step S303 can include: obtaining real-time scene information during vehicle driving; dynamically selecting loose coupling factors and tight coupling factors based on a rule-based decision tree algorithm according to the real-time scene information, wherein each node of the decision tree algorithm represents a scene feature, each branch represents a decision path based on the scene feature, and each leaf node represents the selected loose coupling factor or tight coupling factor; constructing a temporary factor graph according to the selected loose coupling factors and tight coupling factors, wherein the temporary factor graph is composed of the selected loose coupling factors and tight coupling factors; performing consistency check and optimization on the temporary factor graph to obtain an optimized temporary factor graph; and fusing the optimized temporary factor graph with the adjusted initial mixed factor graph model to obtain a mixed factor graph model, wherein the fusion process includes factor graph merging, pose node updating, and inter-factor relationship re-establishment.

[0169] It is worth noting that in the rule-based decision tree algorithm, the selection of scene features is crucial. These features need to accurately reflect the current driving environment and possible challenges faced by the vehicle, so that the algorithm can make reasonable decisions. Common scene features include road type, traffic flow, driving speed, weather conditions, etc. By considering these features comprehensively, the decision tree algorithm can dynamically select the most appropriate loose-coupling factor to adapt to different driving environments and scene requirements.

[0170] The construction process of the decision tree needs to rely on a large amount of actual data and expert experience. First, vehicle driving data in different driving environments and scenes need to be collected, including pose estimation results, sensor data, etc. Then, these data are used to train and optimize the decision tree, so that it can accurately select the appropriate loose-coupling factor according to real-time scene information. In the training process, cross-validation and other methods can be used to evaluate the performance of the decision tree to ensure its accuracy and reliability in actual application.

[0171] After the decision tree is constructed, it can be applied to real-time scene information analysis and factor selection during vehicle driving. When the vehicle drives in different environments, the decision tree algorithm will traverse along the decision path according to the current scene features, and finally reach a leaf node. The leaf node represents the loose-coupling factor or tight-coupling factor that should be selected in the current scene.

[0172] After selecting the appropriate factor, a temporary factor graph needs to be constructed. The temporary factor graph is composed of the selected loose-coupling factor and tight-coupling factor, and is used to temporarily store and represent the relationship between these factors. When constructing the temporary factor graph, the relevance and constraint relationship between factors need to be considered to ensure that the newly added factors can work cooperatively with the original factors.

[0173] After the temporary factor graph is constructed, it needs to be checked for consistency and optimized. This process is mainly to ensure that the factors and relationships in the temporary factor graph meet certain constraint conditions, such as geometric consistency, topological matching, etc. Through optimization, the accuracy and robustness of the temporary factor graph can be further improved.

[0174] Finally, the optimized temporary factor graph is fused with the adjusted initial hybrid factor graph model to obtain the final hybrid factor graph model. The fusion process includes the merging of factor graphs, the updating of pose nodes, and the re-establishment of relationships between factors. Through fusion, new factors and relationships can be introduced into the hybrid factor graph model, thereby realizing the expansion and optimization of the model. This adaptive adjustment capability enables the static map construction method to better adapt to different driving environments and scene requirements, improving the accuracy and robustness of pose estimation.

[0175] In this embodiment, the adaptive adjustment of the factor graph weight is realized by introducing a double-delay deep deterministic policy gradient algorithm, and the loose-tight coupled adaptive factor graph combination algorithm based on scene perception can dynamically select and combine appropriate loose-tight coupled factors according to real-time scene information, realize the expansion and optimization of the hybrid factor graph model, and improve the accuracy and robustness of the pose estimation according to different driving environments and scene requirements, thereby further improving the accuracy and robustness of the static map construction.

[0176] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the static map construction method based on automatic driving of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0177] The present application also provides a static map construction device based on automatic driving, which is described in detail as follows. Figure 3 The static map construction device based on automatic driving comprises:

[0178] The registration module 10 is configured to acquire the laser radar three-dimensional point cloud data and the IMU motion information, and perform time synchronization and spatial registration on the three-dimensional point cloud data and the IMU motion information to obtain registration data.

[0179] The association module 20 is configured to perform landmark extraction and data association based on the registration data to obtain a data association result.

[0180] The construction module 30 is configured to construct a hybrid factor graph model based on the data association result, wherein the hybrid factor graph model is composed of loose-coupled factor graphs and tight-coupled factor graphs with different weights.

[0181] The estimation module 40 is configured to jointly estimate vehicle pose information based on the hybrid factor graph model, and perform local map construction by using a sparse incremental voxel method according to the vehicle pose information to obtain a local point cloud map containing dynamic objects.

[0182] The splicing module 50 is configured to perform dynamic object filtering and map splicing on the local point cloud map containing dynamic objects to obtain a global static point cloud map without dynamic objects.

[0183] The static map construction device based on automatic driving provided in the application adopts the static map construction method based on automatic driving in the above embodiment, and can solve the technical problem that the prior art cannot effectively deal with the interference of dynamic objects by only relying on laser radar data to construct an environment map, resulting in poor accuracy of obtained map information. Compared with the prior art, the beneficial effects of the static map construction device based on automatic driving provided in the application are the same as those of the static map construction method based on automatic driving provided in the above embodiment, and other technical features in the static map construction device based on automatic driving are the same as those disclosed in the above embodiment method, and will not be repeated here.

[0184] The above is only part of the embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation made by using the content of the specification and drawings, or direct / indirect application in other related technical fields under the technical concept of the application is included in the patent protection scope of the application.

Claims

1. An automatic driving-based static map construction method, characterized by, The method comprises: obtaining laser radar three-dimensional point cloud data and IMU motion information, and time synchronizing and spatially registering the three-dimensional point cloud data and the IMU motion information to obtain registration data; extracting landmarks and correlating data based on the registration data to obtain data correlation results; constructing a hybrid factor graph model based on the data correlation results, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights, the hybrid factor graph model is composed of edges and nodes, the nodes represent variables in a map, the edges represent constraints between variables, the variables in the map at least include landmarks and vehicle poses, and the constraints between the variables at least include time continuity constraints between vehicle poses, spatial constraints between landmark feature points, and observation constraints between vehicle poses and landmark feature points; jointly estimating vehicle pose information based on the hybrid factor graph model, and constructing a local map containing dynamic objects by using a sparse incremental voxel method according to the vehicle pose information to obtain the local point cloud map containing dynamic objects; filtering dynamic objects from the local point cloud map containing dynamic objects and splicing the map to obtain a global static point cloud map without dynamic objects; the jointly estimating vehicle pose information based on the hybrid factor graph model and the constructing a local map containing dynamic objects by using a sparse incremental voxel method according to the vehicle pose information to obtain the local point cloud map containing dynamic objects comprises: jointly estimating vehicle pose information by using a factor graph optimization algorithm based on the hybrid factor graph model, wherein the factor graph optimization algorithm at least includes a Gauss-Newton algorithm and a Levenberg-Marquardt algorithm; segmenting point cloud data in a vehicle driving process in a time sequence according to the vehicle pose information to obtain a series of point cloud frames; performing voxelization processing on each frame of point cloud data to generate a voxel grid, wherein each voxel of the voxel grid represents a cubic region in space; screening the generated voxel grid by using a sparse incremental voxel method to retain non-empty voxels containing point cloud data to obtain a sparse voxel grid; registering and fusing adjacent point cloud frames based on the sparse voxel grid and according to the vehicle pose information to obtain a local point cloud map; identifying and marking dynamic objects in the local point cloud map by using a dynamic object identification algorithm based on machine learning to obtain the local point cloud map containing dynamic objects.

2. The method of claim 1, wherein, the obtaining laser radar three-dimensional point cloud data and IMU motion information, and time synchronizing and spatially registering the three-dimensional point cloud data and the IMU motion information to obtain registration data comprises: obtaining laser radar three-dimensional point cloud data and IMU motion information; performing point cloud preprocessing on the laser radar three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data, wherein the point cloud preprocessing at least includes point cloud distortion correction, iterative error Kalman filtering, and noise point removal; time synchronizing the preprocessed three-dimensional point cloud data and the IMU motion information to obtain time synchronization data; The time synchronization data is spatially registered to obtain registration data, wherein the spatial registration at least includes coarse registration based on an initial pose estimation and fine registration based on an iterative closest point algorithm.

3. The method of claim 1, wherein, The landmark extraction and data association are performed based on the registration data to obtain a data association result, including: Landmark features are extracted based on the registration data, wherein the landmark features at least include geometric features and semantic features; Feature descriptors are obtained by performing feature description on the extracted landmark features; According to the feature descriptors, semantic similarity matching, geometric position similarity matching and target distance similarity matching are performed to obtain semantic similarity, geometric position similarity and target distance similarity; The semantic similarity, geometric position similarity and target distance similarity are weighted and fused to obtain a comprehensive matching score; Based on the comprehensive matching score, landmark screening and data association are performed to obtain a data association result, wherein the data association result at least includes an association relationship and an association confidence between landmarks.

4. The method of claim 1, wherein, The hybrid factor graph model is constructed based on the data association result, including: An initial hybrid factor graph is constructed based on the data association result, wherein the initial hybrid factor graph is composed of a loosely coupled factor graph and a tightly coupled factor graph with preset weights; A double-delay deep deterministic policy gradient-based factor graph weight adaptive algorithm is adopted to dynamically adjust the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial hybrid factor graph according to real-time scene information in the vehicle driving process, to obtain an adjusted initial hybrid factor graph model; A scene perception-based loosely and tightly coupled adaptive factor graph combination algorithm is adopted to dynamically select and combine different loosely and tightly coupled factors to expand and optimize the adjusted initial hybrid factor graph model, to obtain a hybrid factor graph model.

5. The method of claim 4, wherein, The initial hybrid factor graph is constructed based on the data association result, including: Frameworks of the loosely coupled factor graph and the tightly coupled factor graph are constructed based on the association relationship and the association confidence between landmarks; Pose nodes are added in the frameworks of the loosely coupled factor graph and the tightly coupled factor graph according to the current pose of the vehicle, to obtain an initial loosely coupled factor graph and an initial tightly coupled factor graph containing pose information, wherein each pose node represents the position and attitude of the vehicle at a certain time; Loosely coupled factors are added in the initial loosely coupled factor graph to obtain a loosely coupled factor graph, wherein the loosely coupled factors at least include semantic landmark observation factors, loop closure detection factors and GNSS prior factors; Tightly coupled factors are added in the initial tightly coupled factor graph to obtain a tightly coupled factor graph, wherein the tightly coupled factors at least include IMU pre-integration factors, laser odometry factors and point cloud registration factors; The loosely coupled factor graph and the tightly coupled factor graph are combined according to preset weights to obtain an initial hybrid factor graph model, wherein the preset weights are set according to the association confidence between landmarks, the importance of landmark features and the complexity of the vehicle driving environment.

6. The method of claim 4, wherein, The factor graph weight adaptive algorithm based on a double-delay deep deterministic policy gradient dynamically adjusts the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph according to real-time scene information in the vehicle driving process, to obtain an adjusted initial mixed factor graph model, which comprises: A double-delay deep deterministic policy gradient algorithm model is constructed, wherein the double-delay deep deterministic policy gradient algorithm model comprises an actor network and a critic network, the actor network is used to select an action according to a current policy, and the critic network is used to evaluate the value of the selected action; The weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph are taken as an action space of the double-delay deep deterministic policy gradient algorithm model, and real-time scene information in the vehicle driving process is taken as a state space of the double-delay deep deterministic policy gradient algorithm model; The actor network selects a weight adjustment action in the action space; The critic network evaluates the value of the selected weight adjustment action in the state space to obtain a value evaluation result; Based on the value evaluation result, a policy gradient update algorithm is used to update the policy parameters of the actor network to obtain an updated actor network; According to the updated actor network, a new weight adjustment action is selected in the action space again, and the critic network continuously evaluates and optimizes the value of the selected weight adjustment action in the state space until a preset convergence condition is reached, to obtain an optimal weight adjustment strategy; According to the optimal weight adjustment strategy, the weights of the loosely coupled factor graph and the tightly coupled factor graph in the initial mixed factor graph are dynamically adjusted to obtain an adjusted mixed factor graph model.

7. The method of claim 4, wherein, The combination algorithm of the loosely and tightly coupled adaptive factor graph based on scene perception dynamically selects and combines different loosely and tightly coupled factors to expand and optimize the adjusted initial mixed factor graph model to obtain a mixed factor graph model, which comprises: Real-time scene information in the vehicle driving process is obtained; Based on the real-time scene information, a rule-based decision tree algorithm is used to dynamically select a loosely coupled factor and a tightly coupled factor, wherein each node of the decision tree algorithm represents a scene feature, each branch represents a decision path based on the scene feature, and each leaf node represents the selected loosely coupled factor or tightly coupled factor; A temporary factor graph is constructed according to the selected loosely coupled factor and tightly coupled factor, wherein the temporary factor graph is composed of the selected loosely coupled factor and tightly coupled factor; The temporary factor graph is subjected to consistency checking and optimization to obtain an optimized temporary factor graph; The optimized temporary factor graph is fused with the adjusted initial mixed factor graph model to obtain a mixed factor graph model, wherein the fusion process comprises merging of factor graphs, updating of pose nodes, and re-establishment of relationships between factors.

8. The method of claim 1, wherein, The local point cloud map containing dynamic objects is subjected to dynamic object filtering and map stitching to obtain a global static point cloud map without dynamic objects, which comprises: The dynamic objects marked in the local point cloud map containing dynamic objects are identified and classified, and a corresponding filtering strategy is adopted to remove the dynamic objects based on the category and motion characteristics of the dynamic objects, so as to obtain a local point cloud map without dynamic objects; A feature matching-based stitching algorithm is adopted to extract feature points in the local point cloud map without dynamic objects and perform matching, so as to establish the relative pose relationship between different local point cloud maps without dynamic objects; An ICP algorithm-based stitching algorithm is adopted to optimize the relative pose relationship obtained through feature matching by using an iterative closest point algorithm, so as to obtain an optimized relative pose relationship; Based on the optimized relative pose relationship, the local point cloud maps without dynamic objects are stitched, so as to obtain a global static point cloud map without dynamic objects. 9.A static map construction apparatus based on autonomous driving, characterized by The static map construction device based on automatic driving comprises: A registration module is configured to acquire laser radar three-dimensional point cloud data and IMU motion information, and perform time synchronization and space registration on the three-dimensional point cloud data and the IMU motion information, so as to obtain registration data. An association module is configured to perform landmark extraction and data association based on the registration data, so as to obtain a data association result. A construction module is configured to construct a hybrid factor graph model based on the data association result, wherein the hybrid factor graph model is composed of loosely coupled factor graphs and tightly coupled factor graphs with different weights, the hybrid factor graph model is composed of edges and nodes, the nodes represent variables in a map, and the edges represent constraints between variables, the variables in the map at least include landmarks and vehicle poses, and the constraints between the variables at least include time continuity constraints between vehicle poses, spatial constraints between landmark feature points, and observation constraints between vehicle poses and landmark feature points. An estimation module is configured to jointly estimate vehicle pose information based on the hybrid factor graph model, and perform local map construction by using a sparse incremental voxel method according to the vehicle pose information, so as to obtain a local point cloud map containing dynamic objects. A stitching module is configured to perform dynamic object filtering and map stitching on the local point cloud map containing dynamic objects, so as to obtain a global static point cloud map without dynamic objects. The estimation module is further configured to jointly estimate vehicle pose information by using a factor graph optimization algorithm based on the hybrid factor graph model, wherein the factor graph optimization algorithm at least includes a Gauss-Newton algorithm and a Levenberg-Marquardt algorithm; to divide point cloud data in a vehicle driving process in a time sequence according to the vehicle pose information, so as to obtain a series of point cloud frames; to perform voxelization processing on each frame of point cloud data, so as to generate a voxel grid, wherein each voxel of the voxel grid represents a cubic region in space; to screen the generated voxel grid by using a sparse incremental voxel method, so as to retain non-empty voxels containing point cloud data, and obtain a sparse voxel grid; and to register and fuse adjacent point cloud frames based on the sparse voxel grid and according to the vehicle pose information, so as to obtain a local point cloud map; and to identify and mark dynamic objects in the local point cloud map by using a dynamic object identification algorithm based on machine learning, so as to obtain a local point cloud map containing dynamic objects.

Images