Mapping methods, devices and controllers, and unmanned vehicles

By installing LiDAR, IMU, and GPS on autonomous vehicles, performing distortion correction and factor map optimization, and using a highly characteristic calibration board for loop closure detection, the problem of low accuracy in point cloud maps for autonomous vehicles was solved, and the accuracy of localization and perception was improved.

CN116399324BActive Publication Date: 2026-11-17CLEARWATER BAY (SHENZHEN) AUTONOMOUS DRIVING INTELLIGENCE RES CENT (LLP) +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202310357308.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-24
Publication Date
2026-11-17
Estimated Expiration
2043-03-24

AI Technical Summary

Technical Problem

The high-precision point cloud maps currently used for autonomous vehicles are not accurate enough, resulting in inaccurate positioning and perception, which can easily lead to safety hazards.

Method used

By installing LiDAR, IMU, and GPS on autonomous vehicles, raw point cloud and pose data are acquired, distortion correction is performed, and factor map optimization and loop closure detection are combined. Loop closure detection is performed using a calibration board with strong features to optimize the point cloud map.

Benefits of technology

It significantly improves the accuracy of point cloud maps, reduces errors in the mapping process, and enhances the positioning and path planning capabilities of autonomous vehicles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116399324B_ABST
    Figure CN116399324B_ABST
Patent Text Reader

Abstract

This application relates to a mapping method, apparatus, controller, and autonomous vehicle. The method includes: acquiring raw point cloud data sampled by LiDAR, pose data collected by IMU, and position data measured by GPS while the vehicle is in motion; correcting distortion of the raw point cloud based on the pose data; obtaining LiDAR pose estimation based on the point cloud matching after distortion correction; integrating the pose estimates between IMU frames to obtain a first relative pose between two adjacent frames; optimizing the position data, pose estimation, and first relative pose in an input factor graph to obtain an initial optimized point cloud map; performing loop closure detection on the current state of the vehicle using the initial optimized point cloud map and a preset calibration board; obtaining a second relative pose based on the detection results; and optimizing the second relative pose in the input factor graph to obtain a final optimized point cloud map. By placing a calibration board with strong features for feature matching in loop closure detection, loop closure detection is facilitated, and a high-precision point cloud map is established, helping the vehicle to perform better path planning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a mapping method, apparatus and controller, and an unmanned vehicle. Background Technology

[0002] With the development of computer technology, the application of high-precision point cloud maps is becoming increasingly widespread, and the accuracy requirements for point cloud maps are becoming increasingly stringent. However, due to the high speeds of autonomous vehicles and the complex surrounding environments, the accuracy of existing mapping systems is not high. Furthermore, because of the long driving distances, the accumulated errors are large, resulting in excessive offsets in the final point cloud maps. This makes the localization and perception processes of autonomous vehicles based on point cloud maps inaccurate, potentially leading to safety hazards. Summary of the Invention

[0003] The main technical problem addressed by the embodiments of this application is how to optimize existing high-precision map schemes to improve the safety of autonomous vehicles.

[0004] Firstly, this embodiment provides a mapping method applied to an autonomous vehicle equipped with a lidar, an IMU, and a GPS. The method includes: acquiring, while the autonomous vehicle is in motion, an original point cloud sampled by the lidar, pose data collected by the IMU, and position data measured by the GPS; performing distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud; performing matching based on the distortion-corrected point cloud to obtain a pose estimate of the lidar; integrating the motion information of the IMU between the pose estimates of the lidar to obtain a first relative pose between two adjacent frames; performing optimization operations on the position data measured by the GPS, the pose estimates of the lidar, and the first relative pose input factor map to obtain an initial optimized point cloud map; performing loop closure detection on the current state of the autonomous vehicle based on the initial optimized point cloud map and a preset calibration board; obtaining a second relative pose between two adjacent frames based on the loop closure detection result; and inputting the second relative pose into the factor map for optimization operations to obtain a final optimized point cloud map.

[0005] In some embodiments, the step of performing distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud includes: determining a reference point and distorted points based on the original point cloud; performing motion compensation on the points in the original point cloud using the pose data to obtain interpolated pose data for each point; calculating the spatial pose relationship between the distorted points and the reference point based on the interpolated pose data between the timestamps of the distorted points and the timestamps of the reference point; and transforming the distorted points to the coordinate system of the reference point based on the spatial pose relationship to obtain a distortion-corrected point cloud.

[0006] In some embodiments, the optimization operation of the GPS-measured location data, the LiDAR pose estimation, and the first relative pose input factor map to obtain an initial optimized point cloud map includes: acquiring the LiDAR point cloud at the current radar sampling time based on the LiDAR pose estimation, and all IMU data at each IMU sampling time from the previous radar sampling time to the current radar sampling time; calculating the predicted current state of the autonomous vehicle at the current radar sampling time based on all the IMU data and the state at the previous radar sampling time; acquiring the GPS-measured location data corresponding to the current radar sampling time; correcting the predicted current state based on the location data to obtain a current state estimate; and performing backpropagation on all the IMU data based on the current state estimate and the first relative pose to correct the LiDAR pose estimation and obtain the initial optimized point cloud map.

[0007] In some embodiments, when a preset number of calibration boards are placed on a preset road segment in which the autonomous vehicle is traveling, the loop closure detection of the current state of the autonomous vehicle based on the initial optimized point cloud map and the preset calibration boards includes: when the autonomous vehicle travels to the preset road segment, obtaining the current index of the autonomous vehicle according to the preset number of calibration boards; obtaining the index recorded when the autonomous vehicle traveled to the preset road segment in the previous time; comparing the current index with the previously recorded index, and if the contents of the two recorded indices are the same or the error is within a preset range, then determining that the current state of the autonomous vehicle is closed-loop driving; otherwise, determining that the current state of the autonomous vehicle is non-closed-loop driving.

[0008] In some embodiments, obtaining the current index of the autonomous vehicle based on the preset number of calibration boards when the autonomous vehicle travels to the preset road segment includes: receiving calibration board detection range and detection frequency instructions when the autonomous vehicle travels to the preset road segment; filtering the point cloud in the initial optimized point cloud map according to the calibration board detection range and detection frequency instructions to obtain the processed point cloud within the detection range of the calibration boards; extracting point clouds with intensity greater than a preset threshold based on the intensity information of the point clouds; performing Hough line detection on the extracted point clouds and selecting three straight lines that meet preset conditions; selecting one point from each of the three straight lines and calculating whether it satisfies the condition of forming a triangle; if it does, recording the current radar odometer index of the autonomous vehicle.

[0009] In some embodiments, obtaining the second relative pose between two adjacent frames based on the loop closure detection result specifically includes: when the loop closure detection result determines that the current state of the autonomous vehicle is closed-loop driving, then recording the second relative pose between the point clouds of two adjacent frames.

[0010] Secondly, embodiments of this application provide a mapping device applied to an autonomous vehicle. The autonomous vehicle is equipped with a lidar, an IMU, and a GPS. The device includes: a parameter acquisition module, used to acquire, while the autonomous vehicle is in motion, the original point cloud sampled by the lidar, the pose data collected by the IMU, and the position data measured by the GPS; a distortion processing module, used to perform distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud; a pose estimation acquisition module, used to obtain the pose estimation of the lidar by matching based on the distortion-corrected point cloud; and a first relative pose acquisition module, used to compare the pose estimations of the lidar with the... The motion information from the IMU is integrated to obtain a first relative pose between two adjacent frames; an initial optimized point cloud map acquisition module is used to optimize the GPS-measured position data, the LiDAR pose estimation, and the first relative pose input factor map to obtain an initial optimized point cloud map; a loop closure detection module is used to perform loop closure detection on the current state of the autonomous vehicle based on the first target point cloud and a preset calibration board; a second relative pose acquisition module is used to obtain a second relative pose between two adjacent frames based on the loop closure detection result; and a final optimized point cloud map acquisition module is used to input the second relative pose into the factor map for optimization to obtain a final optimized point cloud map.

[0011] Thirdly, embodiments of this application provide a controller, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the mapping method as described in the first aspect.

[0012] Fourthly, embodiments of this application provide an unmanned vehicle, including: a lidar, an IMU, a GPS, and a controller as described in the third aspect.

[0013] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer-executable instructions, which, when executed by a controller, cause the controller to perform the mapping method as described in the first aspect.

[0014] The beneficial effects of this application's embodiments: Unlike related technologies, this application provides a mapping method, apparatus, controller, and autonomous vehicle. The mapping method is applied to an autonomous vehicle equipped with a lidar, an IMU, and a GPS. The method includes: acquiring, while the autonomous vehicle is driving, original point clouds sampled by the lidar, pose data collected by the IMU, and position data measured by the GPS; performing distortion correction processing on the original point clouds based on the pose data to obtain distorted point clouds; and matching the distorted point clouds to obtain the pose of the lidar. The process involves estimating the position of the LiDAR and integrating the motion information of the IMU between the LiDAR pose estimates to obtain a first relative pose between two adjacent frames. The GPS-measured position data, the LiDAR pose estimates, and the first relative pose input factor map are then optimized to obtain an initial optimized point cloud map. Loop closure detection is performed on the current state of the autonomous vehicle based on the initial optimized point cloud map and a preset calibration board. A second relative pose between two adjacent frames is obtained based on the loop closure detection results. The second relative pose is then input into the factor map for optimization to obtain a final optimized point cloud map. This application addresses the problems of inaccurate localization and perception based on point cloud maps in existing autonomous driving technologies, particularly the susceptibility to loop closure detection errors. By placing a highly characteristic calibration board for feature matching in loop closure detection, the success rate of loop closure detection is improved, and errors in the mapping process are significantly reduced. This facilitates the creation of high-precision point cloud maps, enabling autonomous vehicles to perform better localization and path planning. Attached Figure Description

[0015] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0016] Figure 1 This is a schematic flowchart of a mapping method provided in an embodiment of this application;

[0017] Figure 2 This is a schematic diagram of a calibration plate placement scenario provided in an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of a factor graph adjustment calibration plate parameter provided in an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the calibration plate test results provided in an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the structure of a mapping device provided in an embodiment of this application;

[0021] Figure 6 This is a schematic diagram of the structure of a controller provided in an embodiment of this application;

[0022] Figure 7 This is a structural schematic diagram of an unmanned vehicle provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] It should be noted that, unless otherwise specified, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device schematic diagram or the order in the flowchart.

[0025] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0026] Please see Figure 1 , Figure 1 This is a flowchart illustrating a mapping method provided in an embodiment of this application. The mapping method is applied to an autonomous vehicle equipped with a lidar, IMU, and GPS. The method includes the following steps:

[0027] Step S1: While the autonomous vehicle is driving, acquire the original point cloud sampled by the lidar, the pose data collected by the IMU, and the position data measured by the GPS.

[0028] The raw point cloud of LiDAR sampling refers to the dataset of spatial points obtained by scanning with a 3D LiDAR device. Each point cloud contains 3D coordinate information, and some also contain color information, reflection intensity information, echo count information, etc.

[0029] IMU stands for Inertial Navigation System. It uses inertial measurement elements (gyroscopes, accelerometers) to measure the angular and linear motion parameters of a vehicle relative to inertial space. Under given initial motion conditions, it calculates the vehicle's velocity, position, attitude, and heading through navigation.

[0030] GPS, short for Global Positioning System, is a high-precision radio navigation positioning system based on artificial Earth satellites. It can provide accurate geographical location, vehicle speed, and precise time information anywhere in the world and in near-Earth space.

[0031] This application embodiment installs LiDAR, IMU, and GPS on an autonomous vehicle to collect 3D LiDAR point cloud data, IMU pose data, and GPS position data of the vehicle's external environment in real time. It should be noted that LiDAR and GPS provide LiDAR point cloud data and position data at relatively low frequencies, respectively, while the IMU provides pose data at a relatively high frequency. The time when the LiDAR collects point cloud data is recorded as the radar sampling time, the time when the GPS collects position data is recorded as the GPS sampling time, and the time when the IMU collects pose data is recorded as the IMU sampling time. Therefore, there will be multiple IMU sampling times between two radar sampling times, and there will also be multiple IMU sampling times between two GPS sampling times.

[0032] Step S2: Perform distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud.

[0033] First, some noise in the raw point cloud sampled by the LiDAR is filtered out. Since the vehicle is also moving while the LiDAR is rotating at high speed to collect data, motion distortion will occur in the collected point cloud. Therefore, distortion correction processing is required for the raw point cloud. The pose data of the IMU can provide a transformation relationship for the distortion correction of the point cloud, transforming the radar points at the current moment to a specific radar pose to eliminate the motion distortion of the point cloud.

[0034] In some embodiments, the step of performing distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud includes: determining a reference point and distorted points based on the original point cloud; performing motion compensation on the points in the original point cloud using the pose data to obtain interpolated pose data for each point; calculating the spatial pose relationship between the distorted points and the reference point based on the interpolated pose data between the timestamps of the distorted points and the timestamps of the reference point; and transforming the distorted points to the coordinate system of the reference point based on the spatial pose relationship to obtain a distortion-corrected point cloud.

[0035] A reference point is a type of point cloud in the original point cloud that can be used as a reference or basis for calibration of other point clouds.

[0036] Distortion points refer to point clouds that exhibit motion distortion during the acquisition process. In this application, all points except the reference point are distortion points.

[0037] A timestamp refers to the moment when a lidar captures each point cloud data.

[0038] High-frequency IMU data can provide a transformation relationship for distortion correction of the original point cloud. This transformation relationship allows distorted points to be converted to the coordinates of a reference point, thus obtaining a distortion-corrected point cloud. For example, the transformation relationship can be determined using quaternions. Record the first timestamp of the first point cloud acquisition and the second timestamp of the last point cloud acquisition in a given radar frame, and record all IMU data within the time interval between the first and second timestamps. Using the quaternion pose data provided by the IMU, perform spherical interpolation on the pose of each point in the original point cloud according to the timestamp to obtain the difference pose data for each point. Based on the interpolated pose data of the timestamps of the distorted points and the reference point, calculate the spatial quasi-relationship between the distorted points and the reference point. By using this spatial relationship, the distorted points in the original point cloud can be converted to the coordinate system of the reference point, thereby eliminating the distortion points caused by the motion of the lidar.

[0039] Step S3: Based on the distortion-free point cloud, perform matching to obtain the pose estimate of the lidar.

[0040] The pose estimation of LiDAR is obtained by matching the relative pose between radar frames through matching algorithms. Matching algorithms can utilize ICP (Iterative Closest Point) and NDT (Normal Distributed Transform), but these algorithms are not real-time during matching. Therefore, in this application, the characteristics of the ground during vehicle movement can be fully utilized to segment the displayed ground in the point cloud, solve the curvature information of each point, and extract line and surface features from the distorted point cloud based on curvature. The point cloud with large curvature is used as line features, and the point cloud with small curvature is used as surface features. To avoid feature density in one area, the point cloud data of a week can be divided into multiple parts, and features of each area can be extracted based on the curvature of each area. For example, the selected corner can be a wall corner, and the selected planar point can be a wall surface.

[0041] Step S4: Integrate the motion information of the IMU between the pose estimates of the lidar to obtain the first relative pose between two adjacent frames.

[0042] Specifically, the IMU between adjacent radar frames can be recorded, and the output data of the six-axis IMU can be pre-integrated. Furthermore, the angular velocity of the IMU output can be pre-integrated to obtain the Euler angle transformation between adjacent frames. In order to facilitate interpolation and prevent the problem of gimbal lock in Euler angles, it is converted into a quaternion. This quaternion is used as the first relative pose for matching between adjacent radar frames. By providing a more accurate first relative pose, the convergence speed and matching accuracy of feature matching between adjacent frames can be accelerated better and faster.

[0043] Step S5: Optimize the GPS-measured location data, the LiDAR pose estimation, and the first relative pose input factor map to obtain an initial optimized point cloud map.

[0044] G2O (General Graphic Optimization) is a C++ framework based on graph optimization, primarily used to optimize nonlinear error functions.

[0045] A factor graph is a representation of the factorization of a function. It generally contains two types of vertices: variable vertices and function vertices, with edges representing the functional relationships between the vertices.

[0046] In this application, the radar pose is used as a vertex in the factor graph of the G2O optimization library, and the lidar pose estimation and the first relative pose are used as binary edges in the factor graph of the G2O optimization library. Since each matching may have a small error, and each matching is based on the previous matching, the cumulative error increases due to the long vehicle travel distance, causing excessive offset in the output point cloud map. Therefore, GPS measured position data can be imported into the factor graph as unary edges of the radar pose vertices for optimization, resulting in a more accurate point cloud map.

[0047] GPS location data can be obtained by directly acquiring the GPS output data and converting the GPS-measured longitude, latitude, and altitude into the local northeast-northeast coordinate system to obtain the vehicle's current location data.

[0048] In some embodiments, based on the pose estimation of the lidar, the lidar point cloud at the current radar sampling time and all IMU data at each IMU sampling time between the previous radar sampling time and the current radar sampling time are obtained. Based on all the IMU data and the state at the previous radar sampling time, the predicted current state of the autonomous vehicle at the current radar sampling time is calculated. The GPS-measured position data corresponding to the current radar sampling time is obtained. The predicted current state is corrected based on the position data to obtain a current state estimate. Based on the current state estimate and the first relative pose, all IMU data are backpropagated to correct the lidar pose estimation and obtain an initial optimized point cloud map.

[0049] Step S6: Perform loop closure detection on the current state of the autonomous vehicle based on the initial optimized point cloud map and the preset calibration board.

[0050] A calibration board is an independently designed object with a distinctive and easily identifiable identifier.

[0051] Loop closure detection, also known as loop shut-in detection, refers to the ability of an autonomous vehicle to identify a scene it has previously visited, thus closing the loop in the map. Successful loop closure detection significantly reduces accumulated errors, enabling the autonomous vehicle to perform more accurate localization and perception tasks. Conversely, incorrect loop closure detection not only fails to eliminate errors but can also worsen the point cloud map and even cause the entire system to crash.

[0052] Understandably, loopback detection needs to determine whether a vehicle has reached a place it has been before, and it needs to know whether the vehicle has returned to the same place it has traveled to before at different times.

[0053] During the aforementioned process, vehicles encounter numerous dynamic obstacles, such as complex environments like intersections with traffic lights and large trucks passing by, or roads with monotonous layouts, such as residential areas or factories with identical building structures. When using common methods like odometer-based Euclidean distance or GPS-based Euclidean distance to find loop closures, the features identified on the first pass may differ significantly from those identified on the second pass, or too many identical features may be identified, leading to failure to detect loop closures or incorrect matches. To avoid these issues, pre-set calibration boards are placed at intersections and crossroads that the vehicle will traverse multiple times. The detection range and frequency are dynamically adjusted based on the placement of these calibration boards to effectively improve detection speed and accuracy. The number of calibration boards can be one or more, depending on the specific circumstances.

[0054] Please see Figure 2 , Figure 2This is a schematic diagram of a calibration plate placement scenario provided in an embodiment of this application, such as... Figure 2 As shown, during the driving process, the unmanned vehicle 10 passes through the same location where frame calibration plates 20, 30, and 40 are set up. By setting these calibration plates, the detection range and detection frequency are dynamically adjusted to effectively improve the speed and accuracy of loop closure detection.

[0055] In some embodiments, when a preset number of calibration boards are placed on a preset road segment in which the autonomous vehicle is traveling, the loop closure detection of the current state of the autonomous vehicle based on the initial optimized point cloud map and the preset calibration boards includes: when the autonomous vehicle travels to the preset road segment, obtaining the current index of the autonomous vehicle according to the preset number of calibration boards, obtaining the index recorded when the autonomous vehicle traveled to the preset road segment in the previous time, comparing the current index with the previously recorded index, and determining that the current state of the autonomous vehicle is closed-loop driving when the contents of the two recorded indices are the same or the error is within a preset range; otherwise, determining that the current state of the autonomous vehicle is non-closed-loop driving.

[0056] An index refers to a marker that an autonomous vehicle marks at a specific location. For example, when an autonomous vehicle passes through the first intersection, its index can be marked as 01, and when it passes through the intersection a second time, its index can be marked as 02. Relevant data information about the vehicle's passage through that intersection can be retrieved through the index.

[0057] In some embodiments, the step of obtaining the current index of the autonomous vehicle based on the preset number of calibration boards when the autonomous vehicle travels to the preset road segment includes: receiving a calibration board detection range and detection frequency instruction when the autonomous vehicle travels to the preset road segment; filtering the point cloud in the initial optimized point cloud map according to the calibration board detection range and detection frequency instruction to obtain the processed point cloud within the detection range of the calibration board; extracting point clouds with intensity greater than a preset threshold based on the intensity information of the point clouds; performing Hough line detection on the extracted point clouds; selecting three straight lines that meet preset conditions; selecting one point from each of the three straight lines; calculating whether the condition of forming a triangle is met; and if so, recording the current radar odometer index of the autonomous vehicle.

[0058] Please see Figure 3 , Figure 3 This is a schematic diagram of a factor graph adjustment calibration plate parameter provided in an embodiment of this application, as shown below. Figure 3As shown, when the autonomous vehicle travels to the preset road section, it receives the calibration board's detection range and detection frequency instructions. The entire parameter adjustment needs to ensure that the bounding box A can surround the calibration board in the front, back, left, right, up, and down directions. In this application, the calibration board's parameters support dynamic adjustment. During the mapping process, the bounding box A will move with the latest radar odometer. It should be noted that the adjustment of the calibration board's parameters will affect the size of the bounding box A, which in turn will affect the calculation speed. When the calibration board's parameter adjustment range is moderate, the detection loop closure speed will be faster. If the calibration board's parameter adjustment range is too large, the calculation time for the detection loop closure will be appropriately extended. If the calibration board's parameter adjustment range is too small, it will not surround all the calibration boards, and the calibration boards will not be detected, rendering the calibration board ineffective.

[0059] When there are three calibration boards, the range of the three calibration boards is determined by selecting a box on the point cloud map. For details, please refer to [link / reference needed]. Figure 3 After adjusting the detection range and frequency, bounding box A in the model does not require detection. First, the point cloud of the current frame within bounding box A is extracted. Due to the inherent characteristics of the calibration board—specifically, the point cloud intensity of the LiDAR points emitted onto the calibration board is higher than elsewhere—a high-intensity point cloud is obtained. Based on the point cloud intensity information, a preset threshold intensity is set to extract the required point cloud. This preset threshold intensity must be greater than the intensity of a normal point cloud and less than or equal to the intensity of the point cloud obtained from the calibration board. The extracted point cloud is then fitted with multiple straight lines using Hough line detection. One point on each of the three lines is selected, and the condition for forming a triangle is calculated. If satisfied, the current radar odometer index of the autonomous vehicle is recorded.

[0060] Specifically, please refer to Figure 4 , Figure 4 This is a schematic diagram of the calibration plate test results provided in an embodiment of this application, as shown below. Figure 4 As shown, the calibration plates detected around the driverless vehicle 10a are respectively frame calibration plate 20a, frame calibration plate 30a and frame calibration plate 40a.

[0061] Step S7: Obtain the second relative pose between two adjacent frames based on the loop closure detection result.

[0062] In some embodiments, obtaining the second relative pose between two adjacent frames based on the loop closure detection result specifically includes: when the loop closure detection result determines that the current state of the autonomous vehicle is closed-loop driving, then recording the second relative pose between the point clouds of two adjacent frames.

[0063] When the autonomous vehicle moves to the location where the calibration board is placed, the index of the radar odometer is recorded for the first time. When it passes by for the second time, the index of the radar odometer is recorded again. The features of the two frames of point clouds when passing the calibration board location are matched by the ICP algorithm to obtain the second relative pose with no cumulative error between the two frames.

[0064] Step S8: Input the second relative pose into the factor map for optimization to obtain the final optimized point cloud map.

[0065] The second relative pose obtained after matching by the ICP algorithm is used as a binary edge of the radar odometry vertex in the factor graph and imported into the factor graph for optimization to obtain the final optimized point cloud map.

[0066] The mapping method provided in this application is applied to an autonomous vehicle equipped with a lidar, an IMU, and a GPS. The method includes: acquiring, while the autonomous vehicle is in motion, original point clouds sampled by the lidar, pose data collected by the IMU, and position data measured by the GPS; performing distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud; performing matching based on the distortion-corrected point cloud to obtain a pose estimate of the lidar; integrating the motion information of the IMU between the pose estimates of the lidar to obtain a first relative pose between two adjacent frames; optimizing the position data measured by the GPS, the pose estimates of the lidar, and the first relative pose input factor map to obtain an initial optimized point cloud map; performing loop closure detection on the current state of the autonomous vehicle based on the initial optimized point cloud map and a preset calibration board; obtaining a second relative pose between two adjacent frames based on the loop closure detection result; and inputting the second relative pose into the factor map for optimization to obtain a final optimized point cloud map. This application addresses the problems of inaccurate localization and perception in existing autonomous driving technologies based on point cloud maps, and the susceptibility of loop closure detection errors. By placing a calibration board with strong features for feature matching in loop closure detection, loop closure detection is facilitated, and errors in the mapping process are significantly reduced. This is beneficial for building high-precision point cloud maps, helping autonomous vehicles to achieve better localization and path planning.

[0067] Please see Figure 5 , Figure 5This is a schematic diagram of a mapping device provided in an embodiment of this application, applied to an autonomous vehicle. The autonomous vehicle is equipped with a lidar, an IMU, and a GPS. The device 100 includes: a parameter acquisition module 101, a distortion processing module 102, a pose estimation acquisition module 103, a first relative pose acquisition module 104, an initial optimized point cloud map acquisition module 105, a loop closure detection module 106, a second relative pose acquisition module 107, and a final optimized point cloud map acquisition module 108.

[0068] The parameter acquisition module 101 is used to acquire the original point cloud sampled by the lidar, the pose data collected by the IMU, and the position data measured by the GPS when the unmanned vehicle is driving.

[0069] The distortion processing module 102 is used to perform distortion removal processing on the original point cloud based on the pose data to obtain a distortion-removed point cloud.

[0070] The pose estimation acquisition module 103 is used to obtain the pose estimate of the LiDAR by matching the distortion-free point cloud. The first relative pose acquisition module 104 is used to integrate the motion information of the IMU between the pose estimates of the LiDAR to obtain the first relative pose between two adjacent frames. The initial optimized point cloud map acquisition module 105 is used to optimize the GPS measured position data, the LiDAR pose estimate, and the first relative pose input factor map to obtain an initial optimized point cloud map. The loop closure detection module 106 is used to perform loop closure detection on the current state of the autonomous vehicle based on the first target point cloud and a preset calibration board. The second relative pose acquisition module 107 is used to obtain the second relative pose between two adjacent frames based on the loop closure detection result. The final optimized point cloud map acquisition module 108 is used to input the second relative pose into the factor map for optimization to obtain the final optimized point cloud map.

[0071] It should be noted that the above-described mapping apparatus can execute the mapping method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the mapping apparatus can be found in the mapping method provided in the embodiments of this application.

[0072] In the embodiments of this application, the mapping device can also be constructed from hardware devices. For example, the mapping device can be constructed from one or more chips, and the chips can work together to complete the mapping method described in the above embodiments. Furthermore, the mapping device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0073] The mapping device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0074] It should be noted that the information interaction and execution process between the various modules and units in the mapping device of this application embodiment are based on the same concept as the method embodiment of this application, and the specific content is also applicable to the mapping device. Each module in the embodiments of this application can be implemented as individual hardware or software, and the functions of each unit can be combined using individual hardware or software as needed.

[0075] This application also provides a controller, please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of a controller provided in an embodiment of this application. The controller 200 includes at least one processor 201 and a memory 202 communicatively connected to the at least one processor 201. The memory 202 stores instructions executable by the at least one processor 201. These instructions, when executed by the at least one processor 201, enable the at least one processor 201 to perform the mapping method in any of the above-described method embodiments. The processor 201 and the memory 202 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0076] Processor 201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0077] The memory 202, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the mapping method in the embodiments of this application. The processor 201, by running the non-transitory software programs, instructions, and modules stored in the memory 202, can implement the mapping method in any of the above method embodiments, that is, it can achieve... Figure 1 The entire process.

[0078] This application also provides an autonomous vehicle; please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of the structure of an unmanned vehicle provided in an embodiment of this application. The unmanned vehicle 300 includes: a lidar 301, an IMU 302, a GPS 303, and the aforementioned controller 200.

[0079] This application provides a non-volatile computer-readable storage medium that stores computer-executable instructions, such as a memory including program code, which can be executed by a processor to complete the mapping method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0080] This application provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. The processor of an intelligent vehicle reads the program code from the computer-readable storage medium and executes the program code to complete the mapping method steps provided in the above embodiments.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A mapping method applied to an autonomous vehicle, wherein the autonomous vehicle is equipped with a lidar, an IMU, and a GPS, characterized in that, The method includes: While the autonomous vehicle is in motion, the system acquires the raw point cloud sampled by the lidar, the pose data collected by the IMU, and the position data measured by the GPS. The original point cloud is subjected to distortion correction processing based on the pose data to obtain a distortion-corrected point cloud. The pose estimate of the lidar is obtained by matching the point cloud after distortion correction. The motion information of the IMU between the pose estimations of the lidar is integrated to obtain the first relative pose between two adjacent frames. The GPS-measured location data, the LiDAR pose estimation, and the first relative pose input factor map are optimized to obtain an initial optimized point cloud map. Based on the initial optimized point cloud map and the preset calibration board, loop closure detection is performed on the current state of the autonomous vehicle; The second relative pose between two adjacent frames is obtained based on the results of the loop closure detection. The second relative pose is input into the factor map for optimization to obtain the final optimized point cloud map. When a preset number of calibration boards are placed on a preset road segment in which the autonomous vehicle is traveling; the loop closure detection of the current state of the autonomous vehicle based on the initial optimized point cloud map and the preset calibration boards includes: when the autonomous vehicle travels to the preset road segment, receiving a calibration board detection range and detection frequency command; filtering the point cloud in the initial optimized point cloud map according to the calibration board detection range and detection frequency command to obtain the processed point cloud within the detection range of the calibration board; extracting point clouds with intensity greater than a preset threshold based on the intensity information of the point clouds; performing Hough line detection on the extracted point clouds and selecting three straight lines that meet preset conditions; selecting one point from each of the three straight lines and calculating whether it satisfies the condition of forming a triangle; if it does, recording the current radar odometer index of the autonomous vehicle.

2. The mapping method according to claim 1, characterized in that, The step of performing distortion correction processing on the original point cloud based on the pose data to obtain a distortion-corrected point cloud includes: The reference point and distortion point are determined based on the original point cloud. The pose data is used to perform motion compensation on the points in the original point cloud to obtain interpolated pose data for each point. Based on the interpolated pose data between the timestamp of the distorted point and the timestamp of the reference point, calculate the spatial pose relationship between the distorted point and the reference point; Based on the spatial pose relationship, the distorted points are transformed to the coordinate system of the reference point to obtain the distorted point cloud.

3. The mapping method according to claim 1, characterized in that, The step of optimizing the GPS-measured location data, the lidar pose estimation, and the first relative pose input factor map to obtain an initial optimized point cloud map includes: Based on the pose estimation of the lidar, the lidar point cloud at the current radar sampling time is obtained, and all IMU data at each IMU sampling time from the previous radar sampling time to the current radar sampling time are obtained. Based on all the IMU data and the state at the previous radar sampling time, calculate the predicted current state of the autonomous vehicle at the current radar sampling time; Obtain the GPS-measured location data corresponding to the current radar sampling time; The current state prediction value is corrected based on the location data to obtain the current state estimate. Based on the current state estimate and the first relative pose, backpropagation is performed on all IMU data to correct the pose estimation of the lidar and obtain an initial optimized point cloud map.

4. The mapping method according to any one of claims 1 to 3, characterized in that, When a preset number of calibration boards are placed on a preset road segment where the autonomous vehicle is traveling, the step of performing loop closure detection on the current state of the autonomous vehicle based on the initial optimized point cloud map and the preset calibration boards further includes: Obtain the index recorded when the autonomous vehicle last traveled to the preset road segment in its current state; The current index is compared with the index of the previous corresponding record. If the contents of the two recorded indexes are the same or the error is within a preset range, the current state of the autonomous vehicle is determined to be closed-loop driving; otherwise, the current state of the autonomous vehicle is determined to be non-closed-loop driving.

5. The mapping method according to claim 4, characterized in that, The step of obtaining the second relative pose between two adjacent frames based on the loop closure detection result specifically includes: When the loop closure detection result determines that the current state of the autonomous vehicle is closed-loop driving, the second relative pose between two adjacent point cloud frames is recorded.

6. A mapping device for use in an unmanned vehicle, wherein the unmanned vehicle is equipped with a lidar, an IMU, and a GPS, characterized in that, The device includes: The parameter acquisition module is used to acquire the original point cloud sampled by the lidar, the pose data collected by the IMU, and the position data measured by the GPS when the unmanned vehicle is driving. The distortion processing module is used to perform distortion removal processing on the original point cloud based on the pose data to obtain a distortion-removed point cloud. The pose estimation acquisition module is used to obtain the pose estimate of the lidar by matching based on the distortion-free point cloud; The first relative pose acquisition module is used to integrate the motion information of the IMU between the pose estimates of the lidar to obtain the first relative pose between two adjacent frames. The initial optimized point cloud map acquisition module is used to optimize the GPS measured location data, the LiDAR pose estimation and the first relative pose input factor map to obtain the initial optimized point cloud map. The loop closure detection module is used to perform loop closure detection on the current state of the autonomous vehicle based on the first target point cloud and a preset calibration board. The second relative pose acquisition module is used to obtain the second relative pose between two adjacent frames based on the loop closure detection result. The final optimized point cloud map acquisition module is used to input the second relative pose into the factor map for optimization to obtain the final optimized point cloud map. The mapping apparatus is used to perform the mapping method as described in any one of claims 1 to 5.

7. A controller, characterized in that, include: At least one processor; A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the mapping method according to any one of claims 1-5.

8. An unmanned vehicle, characterized in that, include: LiDAR, IMU, GPS, and the controller as described in claim 7.

9. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by a controller, cause the controller to perform the mapping method according to any one of claims 1-5.

Citation Information

Patent Citations

  • Robot simultaneous positioning and mapping method and device and readable medium

    CN115265523A