Lidar-based mapping and localization method, system, and engineering vehicle

Abstract

The application relates to the field of engineering vehicles, in particular to a mapping and positioning method and system based on a laser radar and an engineering vehicle. The method comprises the following steps: acquiring multi-sensor information; performing coordinate transformation on the multi-sensor information to generate an initial value of a pose; determining self-defined point cloud information according to laser point cloud information; calculating a constraint factor according to the self-defined point cloud information and optimizing a factor graph model according to the constraint factor; and outputting trajectory information and a global map according to the point cloud information, the multi-sensor information, the initial value of the pose and the factor graph model. The mapping and positioning method based on the laser radar improves the algorithm for map optimization, so that the mapping and positioning method can use a 6-axis inertial sensor to obtain pose information without affecting the positioning accuracy, and the accuracy and robustness of the LIO-SAM algorithm applied to an automatic driving vehicle and a data set are improved.

Description

Technical Field

[0001] This application relates to the field of engineering vehicles, specifically to a mapping and positioning method, system, and engineering vehicle based on lidar. Background Technology

[0002] SLAM (Simultaneous Localization and Mapping) is widely used in the field of robotics. SLAM algorithms enable robots in unfamiliar environments to perceive environmental information and build environmental maps using their onboard sensors, and to calculate their own pose, thus allowing unmanned vehicles to move in unknown environments.

[0003] LIO-SAM (Lidae Odometry and Mapping System with Scan Context-based Adaptive Mapping) is a simultaneous localization and mapping (SMR) algorithm for mobile robots based on LiDAR. It is developed based on the robot's operating system. Compared to other widely used algorithms, such as LOAM (Lidar Odometry and Mapping in Real-time), LIO-SAM uses frame-local mapping instead of frame-global mapping and keyframe selection strategies, thus improving algorithm performance.

[0004] However, the application of the LIO-SAM algorithm to autonomous vehicles currently faces incompatibility issues between localization and mapping technologies, which urgently need to be addressed. Summary of the Invention

[0005] In view of this, this application provides a mapping and localization method, system and engineering vehicle based on LiDAR, which solves or improves the technical problem in the prior art that the localization and map building technologies are not yet compatible when the LIO-SAM algorithm is applied to autonomous vehicles.

[0006] According to a first aspect of this application, this application provides a mapping and localization method based on lidar, which includes: acquiring multi-sensor information, including GPS information, IMU pose information, and lidar point cloud information; performing coordinate transformation on the multi-sensor information to generate initial pose values; determining custom point cloud information based on the lidar point cloud information; calculating constraint factors based on the custom point cloud information and optimizing a factor graph model based on the constraint factors; and outputting trajectory information and a global map based on the custom point cloud information, the multi-sensor information, the initial pose values, and the factor graph model.

[0007] In one possible implementation, obtaining IMU pose information includes: acquiring initial IMU information; pre-integrating the initial IMU information to generate IMU integral information and publishing inertial odometry; adding the IMU integral information and laser odometry to the factor graph model to obtain an optimized factor graph model; and outputting optimized IMU pose information from the optimized factor graph model.

[0008] In one possible implementation, determining the custom point cloud information based on the laser point cloud information includes: acquiring initial laser point cloud data output by the lidar; performing distortion correction on the initial point cloud data and simultaneously fusing the inertial odometry to generate custom point cloud information; extracting features from the initial laser point cloud data to generate feature point cloud data; and adding the feature point cloud data to the custom point cloud information to generate custom point cloud information.

[0009] In one possible implementation, the step of outputting trajectory information and a global map based on the custom point cloud information, the multi-sensor information, the initial pose value, and the factor graph model includes: initializing the initial pose value to obtain the initial laser pose value of the current frame; obtaining the corresponding keyframe set corresponding to the initial laser pose value of the current frame; constructing a local map based on the corresponding keyframe set; calculating the laser odometry based on the custom point cloud information and the multi-sensor information; optimizing the local map and the original global map based on the factor graph model and the laser odometry; and outputting trajectory information and a global map based on the optimized local map and the original global map.

[0010] In one possible implementation, initializing the pose initial value and obtaining the current frame laser pose initial value includes: when the pose initial value is the pose of the first frame, optimizing the pose initial value according to preset attitude angle information to obtain the current frame laser pose initial value; or when the pose initial value is the pose of a subsequent frame, obtaining the incremental pose transformation between the subsequent frame pose and the adjacent frame pose through the inertial odometry, and obtaining the current frame laser pose initial value according to the adjacent frame pose and the incremental pose transformation.

[0011] In one possible implementation, after outputting trajectory information and a global map based on the optimized local map and the original global map, the method further includes: updating the pose of historical keyframes; and publishing the laser odometry.

[0012] In one possible implementation, calculating the laser odometry based on the custom point cloud information and the multi-sensor information includes: obtaining a feature point set based on the custom point cloud information and the multi-sensor information; optimizing the feature point set to generate an optimized pose; weightedly fusing the initial value of the current frame laser pose with the optimized pose, and calculating the laser odometry.

[0013] In one possible implementation, the step of performing coordinate transformation on the multi-sensor information to generate initial pose values ​​includes: obtaining custom GPS information based on the GPS information; recording the initial coordinate values ​​of the custom GPS information and transforming the initial coordinate values ​​of the custom GPS information to a radar coordinate system to generate GPS information in the radar coordinate system; obtaining attitude angle information based on the GPS information in the radar coordinate system; and initializing the attitude angle information to obtain initial pose values.

[0014] According to a second aspect of this application, this application also provides a lidar-based mapping and positioning system, which includes: an information acquisition module for acquiring multi-sensor information, including GPS information, IMU pose information, and lidar point cloud information; a pose initial value generation module for performing coordinate transformation on the multi-sensor information to generate pose initial values; a custom point cloud information determination module for determining custom point cloud information based on the lidar point cloud information; a factor graph model optimization module for calculating constraint factors based on the custom point cloud information and optimizing a factor graph model based on the constraint factors; a map optimization module for outputting trajectory information and a global map based on the point cloud information, the multi-sensor information, the pose initial values, and the factor graph model; and a sensor unit, including GPS, lidar, and IMU, which is communicatively connected to the information acquisition module.

[0015] According to a third aspect of this application, this application also provides an engineering vehicle, which includes the aforementioned lidar-based mapping and positioning system.

[0016] This application provides a mapping and localization method, system, and engineering vehicle based on LiDAR. The LiDAR-based mapping and localization method improves the map optimization algorithm, enabling it to transform the GPS and IMU information into coordinates without affecting positioning accuracy. This transforms the information from the original coordinates to the LiDAR coordinate system, thereby addressing the problem that 6-axis inertial sensors cannot fully acquire vehicle attitude angles. The system using this method can acquire pose information using only 6-axis inertial sensors, further improving the real-time accuracy and robustness of the LIO-SAM algorithm applied to autonomous vehicles and data sets. The coordinate transformation algorithm based on different sensor information also addresses the issue that the installation location of the GPS positioning device may differ from that of the IMU in actual use.

[0017] Furthermore, the above method, which uses local map matching instead of global map matching, can significantly improve the real-time performance of the SLAM system.

[0018] Furthermore, this method determines whether the current laser frame needs to be set as a keyframe, which means that historical keyframes can be updated. This allows for the selective registration of new keyframes to a fixed-size set of prior sub-keyframes, thereby further improving real-time performance. Attached Figure Description

[0019] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 The diagram shown is a flowchart of a lidar-based mapping and positioning method provided in an embodiment of this application.

[0021] Figure 2 The diagram shown is a flowchart of a LiDAR-based mapping and positioning method provided in another embodiment of this application.

[0022] Figure 3 The diagram shown is a flowchart illustrating the method for determining custom point cloud information in a lidar-based mapping and positioning method provided in another embodiment of this application.

[0023] Figure 4 The diagram shown is a flowchart illustrating the trajectory information and global map output method in a lidar-based mapping and positioning method provided in another embodiment of this application.

[0024] Figure 5The diagram shown is a flowchart illustrating the laser odometry calculation method in a lidar-based mapping and positioning method provided in another embodiment of this application.

[0025] Figure 6 The diagram shown is a structural block diagram of a lidar-based mapping and positioning system provided in an embodiment of this application.

[0026] Figure 7 The diagram shown is a structural block diagram of a lidar-based mapping and positioning system provided in another embodiment of this application.

[0027] Figure 8 The diagram shown is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, top, bottom, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movement of the components in a specific posture (as shown in the figures). If the specific posture changes, the directional indication will also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0029] Furthermore, the reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] Application Overview

[0031] Further analysis reveals that, regarding the technical issue of incompatibility between localization and map building technologies when applying the LIO-SAM algorithm to autonomous vehicles, existing technologies are not yet compatible.

[0032] (1) The LIO-SAM algorithm uses a 9-axis inertial sensor by default, while autonomous vehicles and datasets mostly use 6-axis inertial sensors, and the attitude angles cannot be fully obtained.

[0033] (2) In actual use, the installation location of the GPS positioning device may be different from the installation location of the IMU, and the LIO-SAM algorithm does not take this situation into account.

[0034] This application provides a LiDAR-based mapping and localization method, system, and engineering vehicle. Specifically, the LiDAR-based mapping and localization method includes the following steps: acquiring multi-sensor information, including GPS information, IMU pose information, and LiDAR point cloud information; performing coordinate transformation on the multi-sensor information to generate initial pose values; determining custom point cloud information based on the LiDAR point cloud information; calculating constraint factors based on the custom point cloud information and optimizing a factor graph model based on the constraint factors; and outputting trajectory information and a global map based on the point cloud information, multi-sensor information, initial pose values, and factor graph model. This LiDAR-based mapping and localization method improves the map optimization algorithm, enabling the acquisition of pose information using a 6-axis inertial sensor without affecting localization accuracy. This further improves the real-time accuracy and robustness of the LIO-SAM algorithm when applied to autonomous vehicles and data sets.

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0036] Exemplary methods

[0037] Figure 1 The diagram shows a flowchart of a lidar-based mapping and localization method provided in this application. Figure 1 As shown, this lidar-based mapping and localization method may specifically include the following steps:

[0038] Step 100: Acquire multi-sensor information, including GPS information, IMU pose information, and laser point cloud information.

[0039] Multi-sensor information includes GPS (Global Positioning System) information, IMU (Inertial Measurement Unit) pose information, and laser point cloud information. GPS information refers to the position (latitude and longitude) and heading angle (the angle between the vehicle's front and true north) data acquired and transmitted by the GPS positioning system; IMU pose information is the vehicle's pose information detected and transmitted by the inertial sensor; and laser point cloud information is point cloud data detected and transmitted by LiDAR (Light Detection and Ranging). After acquiring multi-sensor information, the data in these three initial forms can be processed to generate corresponding initialization data. Then, the LIO-SAM (Lidae Odometry and Mapping System with Scan Context-based Adaptive Mapping) algorithm can generate an accurate digital 3D model, thereby obtaining the 3D reality surrounding the autonomous vehicle or robot.

[0040] Step 200: Perform coordinate transformation on the multi-sensor information to generate initial pose values.

[0041] Initial pose values ​​refer to the attitude angle data generated after coordinate transformation of the raw data acquired by multiple sensors, such as the Global Positioning System (GPS) and inertial sensors. By performing coordinate transformation on the aforementioned GPS and IMU information, the information is converted from its original coordinates to the lidar coordinate system, thereby improving the problem that 6-axis inertial sensors cannot comprehensively acquire the vehicle's attitude angles.

[0042] Step 300: Determine the custom point cloud information based on the laser point cloud information.

[0043] Custom point cloud information refers to point cloud data that has undergone distortion correction, or point cloud information that has been processed with feature extraction and has had feature point cloud data added. This type of custom point cloud information can be used to optimize local maps, thereby outputting global maps and trajectory information, etc.

[0044] Step 400: Calculate the constraint factors based on the custom point cloud information and optimize the factor graph model based on the constraint factors.

[0045] Constraint factors include laser odometry factors, GPS factors, and closed-loop factors. These constraints are calculated based on custom point cloud information and used to optimize the variables in the initial factor graph. By introducing a factor graph model and multiple constraint factors for robot trajectory estimation, the system can effectively utilize measurements from fused LiDAR and inertial sensors to identify robot positions across postures. Furthermore, the introduction of absolute measurements, i.e., GPS information, allows for joint optimization of the graph using these factor sets from different sources, contributing to improved map real-time performance and accuracy.

[0046] The calculation thread for the closed-loop factor includes: finding the historical keyframe that is closest to the current keyframe and has the largest time interval; extracting the feature point cloud of the current keyframe and the local map of the historical keyframe according to the strategy in the frame-local matching method; and using the ICP (Iterative Closest Point) algorithm to calculate the pose change between the current keyframe and the historical keyframe, and adding it to the closed-loop factor queue. This algorithm calculates the pose change between the current keyframe and the historical keyframe and adds it to the closed-loop factor queue.

[0047] The main function of the GPS factor calculation thread is to receive GPS messages and, if the distance interval is sufficient, construct GPS factors and add them to the queue.

[0048] Step 500: Based on the point cloud information, multi-sensor information, initial pose values, and factor graph model, output trajectory information and global map.

[0049] Trajectory information refers to the trajectory information of a robot or autonomous vehicle moving over time. Attitude angles are calculated using GPS information collected from other sources or the autonomous vehicle itself, and initial pose values ​​are set. This allows the LIO-SAM algorithm to be compatible with 6-axis inertial sensors, enabling the method to use local map matching instead of global map matching, significantly improving the real-time performance and accuracy of the SLAM system.

[0050] Specifically, in one embodiment, Figure 2 The diagram shown is a flowchart illustrating a lidar-based mapping and localization method according to another embodiment of this application. Figure 2 As shown, step 100 (acquiring multi-sensor information) may further include the following steps:

[0051] Step 101: Obtain initial IMU information.

[0052] The initial IMU information is the attitude angle data directly detected by the inertial sensor.

[0053] Step 102: Pre-integrate the initial IMU information to generate IMU integral information and publish the inertial odometry.

[0054] In practical applications, the LIO-SAM algorithm constructs two IMU information queues, one for optimization and the other for integration. The optimization queue adds the IMU integration information between two frames of laser odometry to the factor graph along with the laser odometry itself, calculates the optimized pose, and resets the pose at the corresponding time in the integration queue; the integration queue is used to publish the inertial odometry.

[0055] Step 103: Add the IMU integral information and laser odometry to the factor graph model to obtain the optimized factor graph model.

[0056] Add odometry, velocity, and bias prior factors as constraints to the factor graph model, and reset the factor graph model. Calculate the IMU pre-integration between the laser odometry of the previous frame and the current frame, construct the pre-integration factor and bias factor, add them to the factor graph, and perform factor graph optimization. After optimization, use the timestamp of the current laser odometry as a reference, perform repropagation, and publish the inertial odometry topic.

[0057] Step 104: Output the optimized pose information from the optimized factor graph model.

[0058] This allows for more real-time and accurate output of pose information.

[0059] Optional, Figure 3 The diagram shown is a flowchart illustrating a custom point cloud information determination method in a lidar-based mapping and localization method according to another embodiment of this application. Figure 3 As shown, step 300 (determining custom point cloud information based on laser point cloud information) may further include the following steps:

[0060] Step 301: Obtain the initial laser point cloud data output by the lidar.

[0061] Step 302: Perform distortion correction on the initial point cloud data and simultaneously integrate the inertial odometry to generate custom point cloud information.

[0062] After acquiring the initial laser point cloud data, the start and end times of the point cloud data are calculated to verify its validity. Simultaneously, IMU information is traversed, and IMU information corresponding to the start and end times of the aforementioned point cloud data is selected. The pose increment and rotation amount corresponding to this IMU information are calculated. The point cloud data is then traversed again, and motion distortion correction is performed. Finally, combining the above data, a custom point cloud information is constructed and published.

[0063] Step 303: Extract features from the initial laser point cloud data to generate feature point cloud data.

[0064] Step 304: Add feature point cloud data to the custom point cloud information to generate custom point cloud information.

[0065] After feature extraction from the initial laser point cloud data, the corner points and face points of the point cloud data can be obtained. By downsampling the corner points and face points, feature point cloud data can be generated. Finally, the feature point cloud data is added to the custom point cloud information to generate the custom point cloud information.

[0066] Specifically, in one embodiment, Figure 4 The diagram shown is a flowchart illustrating the trajectory information and global map output method in a lidar-based mapping and localization method according to another embodiment of this application. Figure 4 As shown, step 500 (outputting trajectory information and a global map based on point cloud information, multi-sensor information, initial pose values, and factor graph model) may further include the following steps:

[0067] Step 510: Initialize the pose initial value and obtain the laser pose initial value of the current frame.

[0068] The current frame refers to the current laser frame. Once the initial value of the laser pose in the current frame is determined, it can be used to obtain the key frame of the laser pose in the current frame.

[0069] Step 520: Obtain the set of keyframes corresponding to the initial laser pose value of the current frame.

[0070] A keyframe is the frame in which a critical action occurs during the movement of an object. Correctly obtaining the set of keyframes containing the initial laser pose value of the current frame is crucial for constructing a local map.

[0071] Step 530: Construct a local map based on the corresponding keyframe set.

[0072] A local map is a portion of the map distinct from the global map. Using local map matching instead of global map matching can significantly improve the real-time performance of a SLAM system. The selective introduction of keyframes and the sliding window method allow new keyframes to be input into a fixed window of prior "sub-keyframes." For the keyframe with the smallest time interval from the current laser frame, a set of adjacent keyframes in the spatiotemporal dimension is searched and downsampled. For each keyframe in this set, the corresponding corner points and planar points are extracted and added to the local map, thus constructing the current local map.

[0073] Step 540: Calculate the laser odometry based on the custom point cloud information and multi-sensor information.

[0074] Laser odometry is a method that uses lidar (LiDAR) to recursively estimate the position and pose of objects.

[0075] Step 550: Optimize the local map and the original global map based on the factor graph model and laser odometry.

[0076] The original global map is the global map before real-time localization and the construction of a new map in the current frame. The above map construction process is based on a tightly coupled lidar inertial odometry framework built on a factor graph model, which is suitable for multi-sensor fusion and global optimization, with better real-time performance and more accurate output maps.

[0077] Step 560: Based on the optimized local map and the original global map, output the trajectory information and the global map.

[0078] The above process improves the map optimization algorithm, enabling it to use only 6 axes instead of the original 9-axis IMU, reducing hardware costs and computational complexity while still maintaining good positioning accuracy.

[0079] Optionally, when the above-mentioned initial pose value is the pose of the first frame, step 510 (initializing the initial pose value and obtaining the initial laser pose value of the current frame) may further include:

[0080] Step 5101: When the initial pose value is the pose of the first frame, obtain the preset attitude angle information to optimize the initial pose value, and obtain the initial laser pose value of the current frame.

[0081] If the current laser frame is the first frame, initialize the pose of the current laser frame with attitude angle information obtained from other channels, such as using RTK information to calculate the attitude angle, thereby obtaining the pose of the current laser frame.

[0082] When the above initial pose value is a pose of a subsequent frame other than the first frame, step 510 (initializing the initial pose value and obtaining the initial laser pose value of the current frame) can include the following steps:

[0083] Step 5102: When the initial pose value is the pose of the subsequent frame, the incremental pose transformation between the pose of the subsequent frame and the pose of the adjacent frame is obtained through the inertial odometry, and the initial laser pose value of the current frame is obtained based on the pose of the adjacent frame and the incremental pose transformation.

[0084] If the current laser frame is a subsequent frame, i.e. not the first frame, then the incremental pose transformation between the current laser frame and its adjacent previous laser frame is calculated using an inertial odometry system. This incremental pose transformation is then applied to the laser pose of the previous laser frame to obtain the initial value of the laser pose of the current frame.

[0085] Specifically, in another embodiment, such as Figure 4As shown, after step 560 (outputting trajectory information and global map based on the optimized local map and the original global map), the method may further include the following steps:

[0086] Step 570: Update the pose of historical keyframes.

[0087] Step 580: Release the laser odometer.

[0088] First, determine whether the current laser frame needs to be set as a keyframe, i.e., update the historical keyframes. In this way, new keyframes can be selectively registered to a fixed-size set of prior sub-keyframes to achieve real-time performance.

[0089] In one possible implementation, such as Figure 2 As shown, step 200 (performing coordinate transformation on multi-sensor information to generate initial pose values) may further include:

[0090] Step 210: Obtain custom GPS information based on GPS information.

[0091] Custom GPS information is GPS data generated by parsing and correcting the coordinate system of the original GPS data. Obtaining custom GPS information facilitates the fusion of GPS data and IMU data.

[0092] Step 220: Record the initial coordinate values ​​of the custom GPS information, and transform the initial coordinate values ​​of the custom GPS information to the radar coordinate system to generate GPS information in the radar coordinate system.

[0093] Step 230: Obtain attitude angle information based on GPS information in the radar coordinate system.

[0094] Attitude angle information refers to data about the vehicle's attitude. If the custom GPS information directly contains attitude angle data, it will be directly output and saved. If the custom GPS information does not contain attitude angle information, the attitude angle information will be generated after calculation based on the custom GPS information.

[0095] Step 240: Initialize the attitude angle information and obtain the initial pose value.

[0096] After initializing the attitude angle information obtained in the above steps, the initial pose value can be generated. Then, the custom GPS information and the initial pose value can be published so that the factor graph model can combine the GPS information and the initial pose value to generate and output the map in real time.

[0097] It is worth mentioning that the GPS information mentioned above is preferably RTK (Real-time kinematic, i.e., real-time dynamic positioning technology based on carrier phase observations) information. RTK information can provide the three-dimensional positioning results of the measurement station in a specified coordinate system in real time, further improving the positioning accuracy of the global positioning system.

[0098] In another possible implementation Figure 5 The diagram shown is a schematic flowchart of the laser odometry calculation method in a lidar-based mapping and positioning method provided in another embodiment of this application. Figure 5 As shown, step 540 (calculating the laser odometry based on custom point cloud information and multi-sensor information) may further include the following steps:

[0099] Step 5401: Obtain the feature point set based on the custom point cloud information and multi-sensor information.

[0100] Calculate the curvature of each point in the custom point cloud information. For points that partially occlude or are parallel to each other, no feature extraction is performed. However, feature extraction is performed on unlabeled corner points and face points.

[0101] Specifically, the scan lines can be traversed in the current laser frame, and the point cloud involved in scanning one revolution of each scan line can be divided into 6 segments. Each segment can be sorted according to curvature, and corner points and face points can be selected. The selected corner points and face points can be downsampled. Feature point cloud data can be added to the custom point cloud information to form a feature point set, and this feature point set can be published.

[0102] Step 5402: Optimize the feature point set to generate the optimized pose of the feature point set.

[0103] The Newton-Newton iteration method is used to optimize the feature point set. Specifically, corner-related parameters and face-related parameters are calculated separately. The Jacobian matrix is ​​constructed using the corner-related parameters and face-related parameters. The Gauss-Newton equation is then solved to generate the optimized pose.

[0104] The corner-related parameter calculation process includes: updating the current pose with the result of the previous iteration, transforming the current frame corner point to the global map coordinate system; traversing the corner points in the current frame, finding the 5 nearest neighbors in the local map, and if they can form a straight line, calculating the distance from the current frame corner point to the straight line and the unit vector of the perpendicular line, and storing them as corner-related parameters.

[0105] The calculation process of face-point related parameters includes: updating the current pose with the result of the previous iteration, transforming the face points of the current frame to the global map coordinate system; traversing the face points in the current frame, finding the 5 nearest neighbors in the local map, and if they can form a plane, calculating the distance from the face point of the current frame to the plane and the unit vector of the perpendicular line, and storing them as face-point related parameters.

[0106] Step 5403: Weigh and fuse the initial laser pose value of the current frame with the optimized pose value, and calculate the laser odometry.

[0107] The above steps are to calculate the laser odometry using the frame-local matching method. That is, the initial laser pose value of the current frame is weighted and fused with the optimized pose, and the pitch angle, roll angle and z coordinate are constrained to calculate the laser odometry.

[0108] Corresponding to the aforementioned lidar-based mapping and localization method, this application also provides a lidar-based mapping and localization system. The following will combine... Figure 6 This paper provides a detailed explanation of the lidar-based mapping and positioning system.

[0109] Figure 6 The diagram shown is a structural block diagram of a lidar-based mapping and positioning system provided in an embodiment of this application. Figure 6 As shown, the lidar-based mapping and positioning system provided in this application may specifically include: an information acquisition module 101, a pose initial value generation module 102, a custom point cloud information determination module 103, a factor graph model optimization module 104, a map optimization module 105, and a sensor unit 200. The information acquisition module 101 acquires multi-sensor information, including GPS information, IMU pose information, and lidar point cloud information. The pose initial value generation module 102 performs coordinate transformation on the multi-sensor information to generate initial pose values. The custom point cloud information determination module 103 determines custom point cloud information based on the lidar point cloud information. The factor graph model optimization module 104 calculates constraint factors based on the custom point cloud information and optimizes the factor graph model based on the constraint factors. The map optimization module 105 outputs trajectory information and a global map based on the point cloud information, multi-sensor information, pose initial values, and the factor graph model. The sensor unit 200 is communicatively connected to the information acquisition module 101. Furthermore, the multi-sensor information includes GPS information, IMU pose information, and lidar point cloud information. The sensor unit 200 includes GPS, lidar, and IMU.

[0110] The LiDAR-based mapping and localization system provided in this application can be used to perform the LiDAR-based mapping and localization methods described in the above embodiments. Specifically, this LiDAR-based mapping and localization system can acquire multi-sensor information; perform coordinate transformation on the multi-sensor information to generate initial pose values; acquire custom point cloud information; calculate constraint factors based on the custom point cloud information and optimize the factor graph model based on the constraint factors; and output trajectory information and a global map based on the point cloud information, multi-sensor information, initial pose values, and the factor graph model. The multi-sensor information includes GPS information, IMU pose information, and LiDAR point cloud information. This LiDAR-based mapping and localization method improves the map optimization algorithm, enabling it to acquire pose information using a 6-axis inertial sensor without affecting localization accuracy. This further improves the real-time accuracy and robustness of the LIO-SAM algorithm when applied to autonomous vehicles and datasets.

[0111] In one possible implementation, Figure 7 The diagram shown is a structural block diagram of a lidar-based mapping and positioning system provided in another embodiment of this application. Figure 7 As shown, the aforementioned lidar-based mapping and positioning system can also include: a map optimization unit 100, an IMU pre-integration unit 400, an image projection unit 500, a feature extraction unit 600, and a GPS information processing unit 700. The information acquisition module 101, the pose initial value generation module 102, the custom point cloud information determination module 103, the factor graph model optimization module 104, and the map optimization module 105 are integrated into the map optimization unit 100. Specifically, the map optimization unit 100 receives point cloud information, calculates constraint factors such as the laser odometry factor, GPS factor, and closed-loop factor, adds these constraint factors to the factor graph model, and outputs optimized trajectory information and a global map based on this factor graph model. The IMU pre-integration unit 400 subscribes to the laser odometry topic, uses IMU pose information and the laser odometry to construct a factor graph model, and outputs the optimized inertial odometry. The image projection unit 500 subscribes to the point cloud information topic, performs distortion correction on it, and outputs custom point cloud information. The feature extraction unit 600 calculates the surface and line features of the point cloud and adds them to the custom point cloud information. The GPS information processing unit 700 subscribes to GPS information and performs coordinate transformation on the received GPS information to obtain attitude angle information.

[0112] In addition, corresponding to the above-mentioned LiDAR-based mapping and positioning method and system, this application also provides an engineering vehicle that includes the above-mentioned LiDAR-based mapping and positioning system. That is, this engineering vehicle can perform autonomous or unmanned driving and can apply the above-mentioned LiDAR-based mapping and positioning method. Its effect is similar to that of the above-mentioned LiDAR-based mapping and positioning method and system. Please refer to the above text, and it will not be repeated here.

[0113] Below, for reference Figure 8 This describes an electronic device according to embodiments of the present application.

[0114] Figure 8 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0115] like Figure 8 As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0116] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0117] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the lidar-based mapping and localization methods of the various embodiments of this application described above, and / or other desired functions.

[0118] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0119] When the electronic device is a standalone device, the input device 13 can be a communication network connector for receiving the collected input signals from the first device and the second device.

[0120] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0121] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0122] Of course, for the sake of simplicity, Figure 8 Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0123] As a third aspect of this application, this application provides a computer-readable storage medium storing a computer program for performing the following steps:

[0124] The system acquires multi-sensor information, including GPS information, IMU pose information, and laser point cloud information; performs coordinate transformation on the multi-sensor information to generate initial pose values; determines custom point cloud information based on the laser point cloud information; calculates constraint factors based on the custom point cloud information and optimizes the factor graph model based on the constraint factors; and outputs trajectory information and a global map based on the custom point cloud information, multi-sensor information, initial pose values, and factor graph model.

[0125] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program information. When the computer program information is run by a processor, it causes the processor to perform the steps in the lidar-based mapping and positioning methods according to various embodiments of this application as described in this specification.

[0126] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0127] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program information thereon, which, when run by a processor, causes the processor to execute the steps in the lidar-based mapping and positioning methods according to various embodiments of this application.

[0128] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0130] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0131] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

Claims

1. A mapping and localization method based on lidar, characterized in that, include: Acquire multi-sensor information, including GPS information, 6-axis IMU pose information, and laser point cloud information; The multi-sensor information is transformed to generate GPS information in the radar coordinate system, and an initial pose value is generated based on the GPS information in the radar coordinate system. The initial pose value includes attitude angle data. Custom point cloud information is determined based on the laser point cloud information. The custom point cloud information includes the laser point cloud information after distortion correction, and feature point cloud data obtained by feature extraction of the laser point cloud information. Based on the custom point cloud information, calculate the constraint factor and optimize the factor graph model based on the constraint factor; as well as Based on the custom point cloud information, the multi-sensor information, the initial pose value, and the factor graph model, the trajectory information and the global map are output.

2. The mapping and positioning method based on lidar according to claim 1, characterized in that, Obtaining IMU pose information includes: Obtain initial IMU information; The initial IMU information is pre-integrated to generate IMU integrated information, and the inertial odometry is published. The IMU integral information and laser odometry data are added to the factor graph model to obtain the optimized factor graph model; and The optimized factor graph model outputs the optimized IMU pose information.

3. The mapping and positioning method based on lidar according to claim 2, characterized in that, The step of determining the custom point cloud information based on the laser point cloud information includes: Acquire the initial laser point cloud data output by the lidar; The initial laser point cloud data is distorted by integrating the inertial odometry data to generate initial custom point cloud information; Feature extraction is performed on the initial laser point cloud data to generate feature point cloud data; and Feature point cloud data is added to the initial custom point cloud information to generate the final custom point cloud information.

4. The mapping and positioning method based on lidar according to claim 2, characterized in that, The step of outputting trajectory information and a global map based on the custom point cloud information, the multi-sensor information, the initial pose value, and the factor graph model includes: The initial pose value is initialized to obtain the initial laser pose value of the current frame; Obtain the set of keyframes corresponding to the initial laser pose value of the current frame; Based on the corresponding set of keyframes, a local map is constructed; The laser odometry is calculated based on the custom point cloud information and the multi-sensor information. Based on the factor graph model and the laser odometry, optimize the local map and the original global map; Based on the optimized local map and the original global map, output trajectory information and global map.

5. The mapping and positioning method based on lidar according to claim 4, characterized in that, The initialization of the pose initial value to obtain the laser pose initial value of the current frame includes: When the initial pose value is the pose of the first frame, the initial pose value is optimized according to preset attitude angle information to obtain the initial laser pose value of the current frame; or When the initial pose value is the pose of a subsequent frame, the incremental pose transformation between the pose of the subsequent frame and the pose of the adjacent frame is obtained through the inertial odometry, and the initial laser pose value of the current frame is obtained based on the pose of the adjacent frame and the incremental pose transformation.

6. The mapping and positioning method based on lidar according to claim 4, characterized in that, After outputting trajectory information and a global map based on the optimized local map and the original global map, the method further includes: Update the pose of historical keyframes; The laser odometer was released.

7. The mapping and positioning method based on lidar according to claim 4, characterized in that, The step of calculating the laser odometry based on the custom point cloud information and the multi-sensor information includes: Based on the custom point cloud information and the multi-sensor information, a feature point set is obtained; The feature point set is optimized to generate the optimized pose of the feature point set. The initial laser pose value of the current frame is weighted and fused with the optimized pose, and the laser odometry is calculated.

8. The mapping and positioning method based on lidar according to claim 1, characterized in that, The step of performing coordinate transformation on the multi-sensor information to generate GPS information in the radar coordinate system, and generating initial pose values ​​based on the GPS information in the radar coordinate system includes: Based on the GPS information, obtain custom GPS information, which is GPS data generated by parsing and coordinate system correction of the GPS information; Record the initial coordinate values ​​of the custom GPS information, and transform the initial coordinate values ​​of the custom GPS information to the radar coordinate system to generate GPS information in the radar coordinate system; Based on the GPS information in the radar coordinate system, obtain the attitude angle information; The attitude angle information is initialized to obtain the initial pose value.

9. A mapping and positioning system based on lidar, characterized in that, include: The information acquisition module is used to acquire information from multiple sensors, including GPS information, 6-axis IMU pose information, and laser point cloud information. The pose initial value generation module is used to perform coordinate transformation on the multi-sensor information to generate GPS information in the radar coordinate system, and generate pose initial values ​​based on the GPS information in the radar coordinate system. The pose initial values ​​include attitude angle data. A custom point cloud information determination module is used to determine custom point cloud information based on the laser point cloud information. The custom point cloud information includes the laser point cloud information after distortion correction, and feature point cloud data obtained by feature extraction of the laser point cloud information. The factor graph model optimization module is used to calculate constraint factors based on the custom point cloud information and optimize the factor graph model based on the constraint factors. The map optimization module is used to output trajectory information and a global map based on the point cloud information, the multi-sensor information, the initial pose value, and the factor graph model. The sensor unit includes GPS, lidar, and IMU, and is communicatively connected to the information acquisition module.

10. An engineering vehicle, characterized in that, Including the lidar-based mapping and positioning system as described in claim 9.

Images