Method for calibrating external parameters of a laser radar and related device

Abstract

The application provides an external parameter calibration method and device of a laser radar, an autonomous vehicle and a computer readable storage medium. The method comprises: acquiring point cloud data corresponding to a first reference surface and a second reference surface by using the laser radar; acquiring a first normal vector corresponding to the first reference surface and a first distance d1 between the first reference surface and the laser radar; acquiring a second normal vector corresponding to the second reference surface and a second distance d2 between the second reference surface and the laser radar; acquiring attitude information of an inertial coordinate system corresponding to an inertial measurement unit in a radar coordinate system corresponding to the laser radar based on the first normal vector and the second normal vector; and acquiring position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2. The laser radar is used to perform laser scanning on different reference surfaces, and the accuracy of external parameter calibration is high.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and more particularly to a method, apparatus, autonomous vehicle, and computer-readable storage medium for calibrating the external parameters of a lidar. Background Technology

[0002] As autonomous driving technology matures, autonomous vehicles will become an important means of future transportation. Autonomous vehicles are typically equipped with LiDAR (Light Detection and Ranging) to detect obstacles such as pedestrians, vehicles, and traffic lights. During operation, the extrinsic parameters of the LiDAR need to be known, specifically the spatial transformation relationship between the LiDAR's coordinate system and other coordinate systems.

[0003] There are three common methods for extrinsic parameter calibration: 1. The LiDAR maps the trajectory and matches it with the trajectory obtained by the inertial measurement unit through integration to obtain the extrinsic parameters between the two. However, since the LiDAR's field of view is mainly the ground, which has limited feature information, the obtained trajectory is inaccurate. 2. A LiDAR with known extrinsic parameters and a LiDAR with unknown extrinsic parameters simultaneously scan a corner of a wall and then matches the two point clouds to obtain the extrinsic parameters. However, since the field of view of the LiDAR with unknown extrinsic parameters is too limited to be shared with other LiDARs, the accuracy of the extrinsic parameter calibration is low. 3. The extrinsic parameters are obtained based on the mechanical installation angle and position of the LiDAR itself. However, due to installation errors, the extrinsic parameters obtained in this way have a large error.

[0004] Therefore, there is an urgent need to provide external parameter calibration methods and related devices for lidar to solve the problems existing in the current technology. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, autonomous vehicle, and computer-readable storage medium for extrinsic parameter calibration of a lidar, which utilizes lidar to perform laser scanning on different reference surfaces and achieves high accuracy in extrinsic parameter calibration.

[0006] The objective of this application is achieved through the following technical solution:

[0007] In a first aspect, this application provides a method for calibrating the extrinsic parameters of a lidar, applied to an autonomous vehicle, the autonomous vehicle including the lidar and an inertial measurement unit, the method comprising:

[0008] The lidar is used to acquire point cloud data corresponding to the first reference surface and the second reference surface, respectively.

[0009] Based on the point cloud data of the first reference surface, obtain the first normal vector corresponding to the first reference surface. And the first distance d1 between the first reference surface and the lidar;

[0010] Based on the point cloud data of the second reference surface, obtain the second normal vector corresponding to the second reference surface. And the second distance d2 between the second reference surface and the lidar;

[0011] Based on the first normal vector and the second normal vector Obtain the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar;

[0012] Based on the first distance d1 and the second distance d2, the position information of the inertial coordinate system in the radar coordinate system is obtained;

[0013] Based on the attitude information and the position information, the spatial transformation relationship between the inertial coordinate system and the radar coordinate system is obtained, thereby realizing the external parameter calibration of the lidar.

[0014] The beneficial effects of this technical solution are as follows: Laser radar is used to scan the first reference surface and the second reference surface respectively, obtaining point cloud data corresponding to the first and second reference surfaces. The corresponding first normal vector is then obtained from the point cloud data of the first reference surface. And the first distance d1 between the first reference surface and the lidar, and the corresponding second normal vector obtained from the point cloud data of the second reference surface. And the second distance d2 between the second reference surface and the lidar, thereby based on and The attitude information of the inertial coordinate system in the radar coordinate system is obtained, and the position information of the inertial coordinate system in the radar coordinate system is obtained according to d1 and d2. In this way, the spatial transformation relationship between the inertial coordinate system and the radar coordinate system is obtained, and the external parameter calibration of the lidar is realized.

[0015] Compared to existing technologies that obtain extrinsic parameters through other lidars or self-mapping (which suffer from mechanical installation errors and limited field of view, resulting in lower accuracy of extrinsic parameter calibration), this application utilizes lidar to perform laser scanning on different reference surfaces (eliminating field of view limitations) and reverse-engineers the spatial transformation relationship between the inertial coordinate system and the radar coordinate system based on the obtained point cloud data, resulting in higher accuracy of extrinsic parameter calibration.

[0016] In some optional embodiments, the angle between the first reference plane and the second reference plane is 90°;

[0017] Wherein, the first reference surface is the ground, and the second reference surface is the wall.

[0018] The beneficial effects of this technical solution are as follows: the first reference surface can be the ground, the second reference surface can be the wall, and the angle between the ground and the wall can be 90°. In other words, the autonomous vehicle can be placed on the road and facing the wall. This method is easy to implement in practical applications, and the external parameter calibration process is simple and reliable.

[0019] In some optional embodiments, the process of acquiring point cloud data of the wall surface using the lidar includes:

[0020] The lidar is used to collect overall point cloud data of the ground and the wall.

[0021] The point cloud data of the ground is filtered out from the overall point cloud data to obtain the point cloud data of the wall.

[0022] The beneficial effects of this technical solution are as follows: Generally speaking, due to the actual needs of autonomous driving, most of the point cloud data collected by LiDAR is ground point cloud data, and the ground has relatively little feature information, which is relatively easy to filter out. By filtering out the ground point cloud data from the overall point cloud data, the wall point cloud data can be obtained. Compared with directly extracting the wall point cloud data from the overall point cloud data, the amount of computation in the data processing process is greatly reduced.

[0023] In some alternative embodiments, the statement based on the first normal vector... and the second normal vector Obtaining the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar includes:

[0024] Using the first normal vector The second normal vector and the third normal vector Construct a virtual coordinate system, in which,

[0025] Based on the correspondence between the virtual coordinate system and the inertial coordinate system, the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar is obtained.

[0026] The beneficial effect of this technical solution is that it can utilize the first normal vector. Second normal vector and the third normal vector A virtual coordinate system is constructed, which is to say, the virtual coordinate system is constructed using point cloud data obtained by scanning the first reference plane and the second reference plane by the lidar respectively. To know the attitude information of the inertial coordinate system in the radar coordinate system, it can be obtained directly through the correspondence between the virtual coordinate system and the inertial coordinate system. This method is simple, reliable, fast and convenient in calculation.

[0027] In some optional embodiments, obtaining the position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2 includes:

[0028] The third distance d3 between the first reference surface and the inertial measurement unit and the fourth distance d4 between the second reference surface and the inertial measurement unit are obtained respectively.

[0029] Calculate the first difference between the third distance d3 and the first distance d1, and the second difference between the fourth distance d4 and the second distance d2;

[0030] Based on the first difference and the second difference, the position information of the inertial coordinate system in the radar coordinate system is obtained.

[0031] The beneficial effects of this technical solution are as follows: Based on the first distance d1 between the first reference surface and the lidar and the third distance d3 between the first reference surface and the inertial measurement unit, the second distance d2 between the second reference surface and the lidar and the fourth distance d4 between the second reference surface and the inertial measurement unit, the relative positional relationship between the inertial measurement unit and the lidar can be obtained. By calculating the first difference between d3 and d1, and the second difference between d4 and d2, the relative positional relationship between the inertial measurement unit and the lidar can be simply and intuitively represented. The amount of calculation is small, which improves the efficiency of the external parameter calibration process.

[0032] In some optional embodiments, the autonomous vehicle further includes a camera and a path planning device, and the method further includes:

[0033] During the operation of the autonomous vehicle, the lidar is used to perceive the visual blind spots of the camera.

[0034] When the lidar detects an obstacle in a preset area corresponding to the autonomous vehicle, it converts the coordinates of the obstacle in the lidar coordinate system into reference coordinates corresponding to the inertial coordinate system.

[0035] The reference coordinates and the image data captured by the camera are sent to the path planning device, so that the path planning device can plan the path for the autonomous driving process of the autonomous vehicle based on the reference coordinates and the image data, thereby avoiding the obstacle.

[0036] The beneficial effects of this technical solution are as follows: the LiDAR perception method is laser scanning, which can detect visual blind spots that cannot be captured by cameras. When the LiDAR detects an obstacle in the preset area corresponding to the autonomous vehicle, it can convert the coordinates of the obstacle into reference coordinates corresponding to the inertial coordinate system applicable to the path planning device. This helps the path planning device to plan the path for the autonomous driving process based on the reference coordinates and the corresponding image data, and avoid obstacles.

[0037] Secondly, this application provides an extrinsic parameter calibration device for a lidar, applied to an autonomous vehicle, the autonomous vehicle including the lidar and an inertial measurement unit, the extrinsic parameter calibration device comprising:

[0038] The point cloud acquisition module is used to acquire point cloud data corresponding to the first reference surface and the second reference surface using the lidar, respectively.

[0039] The first point cloud module is used to obtain the first normal vector corresponding to the first reference surface based on the point cloud data of the first reference surface. And the first distance d1 between the first reference surface and the lidar;

[0040] The second point cloud module is used to obtain the second normal vector corresponding to the second reference surface based on the point cloud data of the second reference surface. And the second distance d2 between the second reference surface and the lidar;

[0041] The attitude acquisition module is used to acquire attitude based on the first normal vector. and the second normal vector Obtain the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar;

[0042] The position acquisition module is used to acquire the position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2;

[0043] The spatial transformation module is used to obtain the spatial transformation relationship between the inertial coordinate system and the radar coordinate system based on the attitude information and the position information, so as to realize the external parameter calibration of the lidar.

[0044] In some optional embodiments, the angle between the first reference plane and the second reference plane is 90°;

[0045] Wherein, the first reference surface is the ground, and the second reference surface is the wall.

[0046] In some optional embodiments, the process of acquiring point cloud data of the wall surface using the lidar includes:

[0047] The lidar is used to collect overall point cloud data of the ground and the wall.

[0048] The point cloud data of the ground is filtered out from the overall point cloud data to obtain the point cloud data of the wall.

[0049] In some optional embodiments, the attitude acquisition module includes:

[0050] Virtual coordinate system element, used to utilize the first normal vector The second normal vector and the third normal vector Construct a virtual coordinate system, in which,

[0051] The attitude analysis unit is used to obtain the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar, based on the correspondence between the virtual coordinate system and the inertial coordinate system.

[0052] In some optional embodiments, the location acquisition module includes:

[0053] The distance acquisition unit is used to acquire the third distance d3 between the first reference surface and the inertial measurement unit and the fourth distance d4 between the second reference surface and the inertial measurement unit, respectively.

[0054] The difference calculation unit is used to calculate the first difference between the third distance d3 and the first distance d1, and the second difference between the fourth distance d4 and the second distance d2;

[0055] The position analysis unit is used to obtain the position information of the inertial coordinate system in the radar coordinate system based on the first difference and the second difference.

[0056] In some optional embodiments, the autonomous vehicle further includes a camera and a path planning device, and the extrinsic parameter calibration device further includes:

[0057] A perception module is used to perceive the visual blind spots of the camera using the lidar during the operation of the autonomous vehicle.

[0058] The coordinate transformation module is used to convert the coordinates of the obstacle in the radar coordinate system into the reference coordinates corresponding to the inertial coordinate system when the lidar detects an obstacle in the preset area corresponding to the autonomous vehicle.

[0059] The obstacle avoidance module is used to send the reference coordinates and the image data captured by the camera to the path planning device, so that the path planning device can plan the path for the autonomous driving process of the autonomous vehicle based on the reference coordinates and the image data, thereby avoiding the obstacles.

[0060] Thirdly, this application provides an autonomous driving vehicle, which includes a lidar and an inertial measurement unit, and further includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0061] In some alternative embodiments, the lidar is positioned at a downward angle at the front of the autonomous vehicle.

[0062] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods. Attached Figure Description

[0063] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0064] Figure 1 This application provides a method for calibrating the external parameters of a lidar.

[0065] Figure 2 This is a schematic diagram illustrating the principle of external parameter calibration for a lidar provided in an embodiment of this application.

[0066] Figure 3 This is a structural block diagram of an external parameter calibration device for a lidar provided in an embodiment of this application.

[0067] Figure 4 This is a structural block diagram of an autonomous vehicle provided in an embodiment of this application.

[0068] Figure 5 This is a schematic diagram of the structure of a program product provided in an embodiment of this application. Detailed Implementation

[0069] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0070] See Figure 1 , Figure 1This application provides an embodiment of a lidar extrinsic parameter calibration method, which is applied to an autonomous vehicle. The autonomous vehicle includes the lidar and an inertial measurement unit. The method includes:

[0071] Step S101: Use the lidar to acquire point cloud data corresponding to the first reference surface and the second reference surface respectively;

[0072] Step S102: Based on the point cloud data of the first reference surface, obtain the first normal vector corresponding to the first reference surface. And the first distance d1 between the first reference surface and the lidar;

[0073] Step S103: Based on the point cloud data of the second reference surface, obtain the second normal vector corresponding to the second reference surface. And the second distance d2 between the second reference surface and the lidar;

[0074] Step S104: Based on the first normal vector and the second normal vector Obtain the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar;

[0075] Step S105: Based on the first distance d1 and the second distance d2, obtain the position information of the inertial coordinate system in the radar coordinate system;

[0076] Step S106: Based on the attitude information and the position information, obtain the spatial transformation relationship between the inertial coordinate system and the radar coordinate system to achieve the external parameter calibration of the lidar.

[0077] Therefore, by using a lidar to perform laser scanning on the first and second reference surfaces respectively, point cloud data corresponding to the first and second reference surfaces are obtained. The corresponding first normal vector is then obtained from the point cloud data of the first reference surface. And the first distance d1 between the first reference surface and the lidar, and the corresponding second normal vector obtained from the point cloud data of the second reference surface. And the second distance d2 between the second reference surface and the lidar, thereby based on and The attitude information of the inertial coordinate system in the radar coordinate system is obtained, and the position information of the inertial coordinate system in the radar coordinate system is obtained according to d1 and d2. In this way, the spatial transformation relationship between the inertial coordinate system and the radar coordinate system is obtained, and the external parameter calibration of the lidar is realized.

[0078] Compared to existing technologies that obtain extrinsic parameters through other lidars or self-mapping (which suffer from mechanical installation errors and limited field of view, resulting in lower accuracy of extrinsic parameter calibration), this application embodiment utilizes lidar to perform laser scanning on different reference surfaces (eliminating field of view limitations), and deduces the spatial transformation relationship between the inertial coordinate system and the radar coordinate system based on the obtained point cloud data, resulting in higher accuracy of extrinsic parameter calibration.

[0079] In this embodiment of the application, the spatial transformation relationship between the inertial coordinate system and the radar coordinate system can be represented by a transformation matrix.

[0080] An inertial measurement unit (IMU) is a device that measures the three-axis attitude angles (or angular rates) and acceleration of an object.

[0081] Gyroscopes and accelerometers are the main components of an IMU (Inertial Measurement Unit), and their accuracy directly affects the accuracy of the inertial system. In practical operation, various unavoidable interference factors cause errors in the gyroscopes and accelerometers. From the initial alignment, the navigation error increases over time, especially the position error, which is a major drawback of inertial navigation systems. Therefore, external information is needed to assist in achieving integrated navigation, effectively reducing the problem of error accumulation over time. To improve reliability, more sensors can be equipped for each axis. Generally, the IMU should be mounted at the center of gravity of the object being measured.

[0082] Typically, an IMU contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometers detect the acceleration signals of an object along three independent axes in the carrier coordinate system, while the gyroscopes detect the angular velocity signals of the carrier relative to the navigation coordinate system. By measuring the angular velocity and acceleration of the object in three-dimensional space, the object's attitude can be calculated. This has significant application value in navigation.

[0083] IMUs are mostly used in devices that require motion control, such as automobiles and robots. They are also used in applications that require precise displacement calculations based on attitude, such as inertial navigation systems for submarines, aircraft, missiles, and spacecraft.

[0084] The radar coordinate system can be an absolute coordinate system or a relative coordinate system, and correspondingly, the inertial coordinate system can be an absolute coordinate system or a relative coordinate system.

[0085] Absolute coordinate systems can be, for example, Cartesian coordinate systems (a general term for rectangular and oblique coordinate systems), polar coordinate systems, cylindrical coordinate systems, spherical coordinate systems, or geodetic coordinate systems (represented by longitude and latitude) and Gaussian plane rectangular coordinate systems in the field of geodesy, or raster coordinate systems, octree coordinate systems, etc. in geographic information systems, or screen coordinate systems in the field of computers, etc.

[0086] The relative coordinate system can be a rectangular coordinate system or other coordinate systems.

[0087] In this embodiment, the included angle between the first reference plane and the second reference plane is not limited. For example, the included angle between the first reference plane and the second reference plane is 30° to 90° (the included angle between any two planes is [0, 90°]).

[0088] The first reference plane and the second reference plane can be two non-parallel planes. The first reference plane can be a plane or a curved surface, and the second reference plane can be a plane or a curved surface.

[0089] In some optional embodiments, the angle between the first reference plane and the second reference plane is 90°;

[0090] Wherein, the first reference surface is the ground, and the second reference surface is the wall.

[0091] Therefore, the first reference surface can be the ground, the second reference surface can be the wall, and the angle between the ground and the wall can be 90°. In other words, the autonomous vehicle can be placed on the road and facing the wall. This method is easy to implement in practical applications, and the external parameter calibration process is simple and reliable.

[0092] In a specific application, the road surface can be a flat wall surface, the wall surface can be a flat wall surface, the road surface and the wall surface are perpendicular to each other, and the chassis of the autonomous vehicle is set parallel to the ground.

[0093] In another specific application, the autonomous vehicle can be placed on a test bench. The angle between the test bench and the wall can be 30°, 60°, or 90°. The plane where the test bench is located is used as the first reference plane, and the wall is used as the second reference plane. Since both the LiDAR and the inertial measurement unit are installed on the autonomous vehicle, the spatial transformation relationship between the radar coordinate system and the inertial coordinate system will not change due to the angle between the first and second reference planes. When the angle between the first and second reference planes is 90°, the corresponding first normal vector... Second normal vector The included angle is also 90°.

[0094] In some optional embodiments, the process of acquiring point cloud data of the wall surface using the lidar includes:

[0095] The lidar is used to collect overall point cloud data of the ground and the wall.

[0096] The point cloud data of the ground is filtered out from the overall point cloud data to obtain the point cloud data of the wall.

[0097] Therefore, generally speaking, due to the actual needs of autonomous driving, most of the point cloud data collected by LiDAR is ground point cloud data, and the ground has less feature information and is relatively easy to filter out. By filtering out the ground point cloud data from the overall point cloud data, the wall point cloud data can be obtained. Compared with directly extracting the wall point cloud data from the overall point cloud data, the amount of computation in the data processing process is greatly reduced.

[0098] In some alternative embodiments, the statement based on the first normal vector... and the second normal vector Obtaining the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar includes:

[0099] Using the first normal vector The second normal vector and the third normal vector Construct a virtual coordinate system, in which,

[0100] Based on the correspondence between the virtual coordinate system and the inertial coordinate system, the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar is obtained.

[0101] Therefore, the first normal vector can be used. Second normal vector and the third normal vector A virtual coordinate system is constructed, which is to say, the virtual coordinate system is constructed using point cloud data obtained by scanning the first reference plane and the second reference plane by the lidar respectively. To know the attitude information of the inertial coordinate system in the radar coordinate system, it can be obtained directly through the correspondence between the virtual coordinate system and the inertial coordinate system. This method is simple, reliable, fast and convenient in calculation.

[0102] See Figure 2 , Figure 2 This is a schematic diagram illustrating the principle of external parameter calibration for a lidar provided in an embodiment of this application.

[0103] The radar coordinate system corresponding to the lidar adopts a rectangular coordinate system. The inertial coordinate system corresponding to the inertial measurement unit adopts a rectangular coordinate system.

[0104] In a specific application, the angle between the first reference plane and the second reference plane is 90°, and correspondingly, the first normal vector... Second normal vector The included angle is also 90°, so in this case, as well as The constructed virtual coordinate system is a rectangular coordinate system.

[0105] It can be seen that the correspondence between the virtual coordinate system and the inertial coordinate system is as follows:

[0106] In some optional embodiments, obtaining the position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2 includes:

[0107] The third distance d3 between the first reference surface and the inertial measurement unit and the fourth distance d4 between the second reference surface and the inertial measurement unit are obtained respectively.

[0108] Calculate the first difference between the third distance d3 and the first distance d1, and the second difference between the fourth distance d4 and the second distance d2;

[0109] Based on the first difference and the second difference, the position information of the inertial coordinate system in the radar coordinate system is obtained.

[0110] Therefore, based on the first distance d1 between the first reference surface and the lidar and the third distance d3 between the first reference surface and the inertial measurement unit, the second distance d2 between the second reference surface and the lidar and the fourth distance d4 between the second reference surface and the inertial measurement unit, the relative positional relationship between the inertial measurement unit and the lidar can be obtained. By calculating the first difference between d3 and d1, and the second difference between d4 and d2, the relative positional relationship between the inertial measurement unit and the lidar can be simply and intuitively represented. The amount of calculation is small, which improves the efficiency of the external parameter calibration process.

[0111] In some optional embodiments, the method further includes:

[0112] The lidar is used to acquire point cloud data corresponding to the third reference surface;

[0113] Based on the point cloud data of the third reference surface, obtain the normal vector corresponding to the third reference surface. And the distance d0 between the third reference surface and the lidar;

[0114] The first reference surface can be the ground, and the second and third reference surfaces can all be walls. The first reference surface is perpendicular to the second reference surface and the third reference surface is perpendicular to each other. The second reference surface and the third reference surface are not parallel to each other.

[0115] use and Construct a reference coordinate system. Based on the correspondence between the reference coordinate system and the inertial coordinate system, the reference attitude information of the inertial coordinate system in the radar coordinate system is obtained;

[0116] Obtain the distance dp between the third reference plane and the inertial measurement unit, and calculate d0 and d... p The reference difference is used to obtain the reference position information of the inertial coordinate system in the radar coordinate system;

[0117] Based on the attitude information and reference attitude information of the inertial coordinate system in the radar coordinate system, and the position information and reference position information of the inertial coordinate system in the radar coordinate system, the spatial transformation relationship between the inertial coordinate system and the radar coordinate system is obtained, thereby realizing the external parameter calibration of the lidar.

[0118] By scanning three reference surfaces separately, the accuracy of extrinsic parameter calibration can be further improved. The ground is used as the first reference surface, and two non-parallel walls are used as the second and third reference surfaces (these two walls are perpendicular to the ground). The attitude information obtained from the first and second reference surfaces can be weighted or averaged with the reference attitude information obtained from the first and third reference surfaces to determine the actual attitude of the inertial coordinate system in the radar coordinate system. Correspondingly, the position information obtained from the first and second reference surfaces can be weighted or averaged with the reference position information obtained from the first and third reference surfaces to determine the actual position of the inertial coordinate system in the radar coordinate system, and thus the spatial transformation relationship between the inertial coordinate system and the radar coordinate system can be obtained.

[0119] In some optional embodiments, the autonomous vehicle further includes a camera and a path planning device, and the method further includes:

[0120] During the operation of the autonomous vehicle, the lidar is used to perceive the visual blind spots of the camera.

[0121] When the lidar detects an obstacle in a preset area corresponding to the autonomous vehicle, it converts the coordinates of the obstacle in the lidar coordinate system into reference coordinates corresponding to the inertial coordinate system.

[0122] The reference coordinates and the image data captured by the camera are sent to the path planning device, so that the path planning device can plan the path for the autonomous driving process of the autonomous vehicle based on the reference coordinates and the image data, thereby avoiding the obstacle.

[0123] Therefore, the perception method of LiDAR is laser scanning, which can detect visual blind spots that cameras cannot capture. When LiDAR detects an obstacle in the preset area corresponding to the autonomous vehicle, it can convert the coordinates of the obstacle into reference coordinates corresponding to the inertial coordinate system applicable to the path planning device. This helps the path planning device to plan the path for the autonomous driving process based on the reference coordinates and the corresponding image data, and avoid obstacles.

[0124] In some optional embodiments, the process of planning a driving route using a path planning device includes:

[0125] Receive the reference coordinates and the image data;

[0126] A high-precision map is constructed based on the reference coordinates and the image data;

[0127] Determine the current position of the autonomous vehicle on the high-precision map and the target position that the autonomous vehicle needs to reach;

[0128] Obstacles on the driving path are identified, and multiple driving paths from the current position to the target position are calculated using a state grid algorithm;

[0129] Calculate the travel cost of each of the aforementioned travel paths, and select the travel path with the lowest travel cost as the target travel path.

[0130] The embodiments of this application do not limit the obstacles, which can be pedestrians, other vehicles or traffic lights.

[0131] The embodiments of this application do not limit the preset area, which can be an area with a radius of 100 meters, 200 meters or 300 meters centered on the autonomous vehicle.

[0132] This application does not limit the type of lidar used; it can be any of the following: single-line lidar, multi-line lidar, and solid-state lidar. The number of lidars can be, for example, one or more.

[0133] Single-line lidar can detect the length, width, and height of vehicles traveling on highways and national and provincial roads.

[0134] Multi-line lidar adopts the laser-time-of-flight principle, integrating cutting-edge technologies from multiple disciplines such as high-frequency laser pulse generation technology, efficient and precise optical system, high-precision timing circuit, high-speed scanning system, multi-pulse echo technology and mechanical kinematics. It has wide environmental requirements and can work reliably both indoors and outdoors. It can still ensure reliable operation in harsh weather conditions such as rain, fog, snow, and high and low temperatures, and can be widely used in various transportation environments.

[0135] Solid-state LiDAR utilizes MEMS technology to miniaturize and electronically design mechanical mechanisms. Its mature technology allows for mass production and is suitable for applications such as intelligent transportation, security, autonomous driving, intelligent robotics, and industrial automation. It uses MEMS micro-mirrors for area scanning and fiber lasers as the light source, significantly reducing the need for transmitters and receivers, thus lowering costs. Its all-solid-state design eliminates the need for any moving parts compared to traditional LiDAR, offering excellent vibration resistance, stronger anti-interference capabilities, and a longer lifespan. With an ultra-long detection range exceeding 200m, ultra-high resolution, and a wide field of view, it meets the needs of obstacle avoidance in large-scale detection and can output raw point cloud data. It boasts automotive-grade quality assurance, a simple internal structure design, and can be manufactured in a single step, enabling mass production.

[0136] In this embodiment, the radar parameters of the lidar include laser wavelength, detection range, FOV (vertical + horizontal), ranging accuracy, angular resolution, number of output points, beamwidth, safety level, output parameters, IP protection level, power, power supply voltage, laser emission method (mechanical / solid-state), and lifespan. Lidar offers significant advantages, including a wider detection range and higher accuracy.

[0137] Laser wavelength: Currently, the most commonly used wavelengths for 3D imaging LiDAR on the market are 905nm and 1550nm. 1550nm wavelength LiDAR sensors can operate at higher power to increase detection range and have stronger penetration through rain and fog. The main advantage of 905nm is that silicon absorbs photons at this wavelength, and silicon-based photodetectors are generally cheaper than the indium gallium arsenide (InGaAs) near-infrared detectors required to detect 1550nm light.

[0138] Detection range: The range measurement of a lidar is related to the reflectivity of the target. The higher the reflectivity of the target, the farther the measured distance; the lower the reflectivity of the target, the closer the measured distance. Therefore, when checking the detection range of a lidar, it is important to know what level of target reflectivity that this measurement is based on.

[0139] FOV: The field of view of a lidar is both horizontal and vertical. For a mechanically rotating lidar, the horizontal field of view is 360 degrees.

[0140] Angular resolution includes both vertical and horizontal resolution. Achieving high horizontal resolution is relatively easy because the horizontal direction is driven by a motor, allowing for very high resolution, typically around 0.01 degrees. Vertical resolution is related to the transmitter's geometry and its arrangement; the smaller the spacing between adjacent transmitters, the lower the vertical resolution. Vertical resolution is typically in the range of 0.1 to 1 degree.

[0141] Points emitted: The number of laser points emitted by the lidar per second. The point output of a lidar typically ranges from tens of thousands to hundreds of thousands of points per second.

[0142] Linear LiDAR: Multi-line LiDAR uses multiple laser emitters distributed vertically, with motors rotating to create multiple scanning beams. The ideal number of lines in a LiDAR system refers to how many lines will capture objects that meet the algorithm's requirements. Theoretically, more and denser beams provide a more comprehensive environmental description, potentially reducing the algorithm's requirements. Common LiDAR beam configurations include 16 lines, 32 lines, and 64 lines.

[0143] Output parameters: obstacle position (3D), velocity (3D), direction, timestamp (available on some lidar systems), reflectivity.

[0144] Lifespan: Mechanically rotating lidar typically has a lifespan of several thousand hours; solid-state lidar can have a lifespan of up to 100,000 hours.

[0145] Laser emission methods: Traditional laser radars use a mechanically rotating structure, which is prone to wear and tear, limiting their lifespan. Solid-state laser radars mainly consist of three types: Flash, MEMS, and phased array. Flash laser radars, given a light source, can cover the entire field of view with a single pulse. Time-of-flight (ToF) data is then used to receive the relevant information and map the targets around the laser radar. MEMS laser radars have a relatively simple structure, requiring only a laser beam and a reflector. Specifically, the laser beam is directed at the rotating reflector, which controls the direction of the laser beam by rotating. Phased array laser radars utilize microarrays formed synchronously by independent antennas. Phased arrays can transmit radio waves in any direction, completely eliminating the "rotation" step. By controlling the timing or array of signals transmitted by each antenna, the signal can be directed to a specific location.

[0146] In a specific application, an autonomous vehicle includes a perception module, a localization module, a path planning device, a control module, a high-precision map module, and a cloud computing module. The perception module, as the core module of the autonomous driving system, detects and tracks obstacles such as pedestrians, vehicles, and traffic lights encountered during autonomous driving.

[0147] The perception module utilizes multiple sensors, including multi-line LiDAR, solid-state LiDAR, and cameras, to simultaneously detect obstacles. The solid-state LiDAR is positioned downwards at the front of the vehicle to detect blind spots of other sensors and to perceive obstacles and the road surface closer to the autonomous vehicle. After detecting an obstacle, the obstacle coordinates from the LiDAR's radar coordinate system are transformed to the inertial coordinate system (vehicle coordinate system). Under this unified coordinate system, the path planning device can operate more effectively.

[0148] See Figure 3 , Figure 3 This is a structural block diagram of a lidar extrinsic parameter calibration device 100 provided in an embodiment of this application. The extrinsic parameter calibration device 100 is applied to an autonomous vehicle, which includes the lidar and an inertial measurement unit. The extrinsic parameter calibration device 100 includes:

[0149] Point cloud acquisition module 101 is used to acquire point cloud data corresponding to the first reference surface and the second reference surface using the lidar respectively;

[0150] The first point cloud module 102 is used to obtain the first normal vector corresponding to the first reference surface based on the point cloud data of the first reference surface. And the first distance d1 between the first reference surface and the lidar;

[0151] The second point cloud module 103 is used to obtain the second normal vector corresponding to the second reference surface based on the point cloud data of the second reference surface. And the second distance d2 between the second reference surface and the lidar;

[0152] Attitude acquisition module 104 is used to acquire attitude based on the first normal vector. and the second normal vector Obtain the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar;

[0153] The position acquisition module 105 is used to acquire the position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2;

[0154] The spatial transformation module 106 is used to obtain the spatial transformation relationship between the inertial coordinate system and the radar coordinate system based on the attitude information and the position information, so as to realize the external parameter calibration of the lidar.

[0155] In some optional embodiments, the angle between the first reference plane and the second reference plane is 90°;

[0156] Wherein, the first reference surface is the ground, and the second reference surface is the wall.

[0157] In some optional embodiments, the process of acquiring point cloud data of the wall surface using the lidar includes:

[0158] The lidar is used to collect overall point cloud data of the ground and the wall.

[0159] The point cloud data of the ground is filtered out from the overall point cloud data to obtain the point cloud data of the wall.

[0160] In some optional embodiments, the attitude acquisition module 104 includes:

[0161] Virtual coordinate system element, used to utilize the first normal vector The second normal vector and the third normal vector Construct a virtual coordinate system, in which,

[0162] The attitude analysis unit is used to obtain the attitude information of the inertial coordinate system corresponding to the inertial measurement unit in the radar coordinate system corresponding to the lidar, based on the correspondence between the virtual coordinate system and the inertial coordinate system.

[0163] In some optional embodiments, the location acquisition module 105 includes:

[0164] The distance acquisition unit is used to acquire the third distance d3 between the first reference surface and the inertial measurement unit and the fourth distance d4 between the second reference surface and the inertial measurement unit, respectively.

[0165] The difference calculation unit is used to calculate the first difference between the third distance d3 and the first distance d1, and the second difference between the fourth distance d4 and the second distance d2;

[0166] The position analysis unit is used to obtain the position information of the inertial coordinate system in the radar coordinate system based on the first difference and the second difference.

[0167] In some optional embodiments, the autonomous vehicle further includes a camera and a path planning device, and the extrinsic parameter calibration device 100 further includes:

[0168] A perception module is used to perceive the visual blind spots of the camera using the lidar during the operation of the autonomous vehicle.

[0169] The coordinate transformation module is used to convert the coordinates of the obstacle in the radar coordinate system into the reference coordinates corresponding to the inertial coordinate system when the lidar detects an obstacle in the preset area corresponding to the autonomous vehicle.

[0170] The obstacle avoidance module is used to send the reference coordinates and the image data captured by the camera to the path planning device, so that the path planning device can plan the path for the autonomous driving process of the autonomous vehicle based on the reference coordinates and the image data, thereby avoiding the obstacles.

[0171] See Figure 4 , Figure 4 This is a structural block diagram of an autonomous vehicle 200 provided in an embodiment of this application.

[0172] The autonomous vehicle 200 includes a lidar, an inertial measurement unit, at least one memory 210 and at least one processor 220, and may also include a bus 230 connecting different platform systems.

[0173] The memory 210 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 211 and / or cache memory 212, and may further include read-only memory (ROM) 213.

[0174] The memory 210 also stores a computer program, which can be executed by the processor 220 to enable the processor 220 to perform the functions of any of the above-mentioned devices or to perform the steps of any of the above-mentioned methods. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above-mentioned method embodiments, and some contents will not be repeated.

[0175] The memory 210 may also include a utility 214 having at least one program module 215, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0176] Accordingly, processor 220 can execute the aforementioned computer program, and can also execute utility 214.

[0177] The processor 220 may employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0178] Bus 230 can be one or more of several types of bus structures, including memory bus or memory autonomous vehicle, peripheral bus, graphics acceleration port, processor, or local bus using any bus structure using multiple bus structures.

[0179] The autonomous vehicle 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with the autonomous vehicle 200, and / or with any device that enables the autonomous vehicle 200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed through input / output interfaces 250. Furthermore, the autonomous vehicle 200 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of the autonomous vehicle 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the autonomous vehicle 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0180] The embodiments of this application do not limit the installation location of the lidar.

[0181] In some alternative embodiments, the lidar can be mounted on the roof, hood, or bumper. It offers advantages such as a high vantage point, long-range visibility, wide field of vision, and high safety.

[0182] Currently, most lidar systems have a maximum detection range of 100 to 150 meters for targets with 10% reflectivity. If the same lidar is placed at different heights (roof, hood, bumper), given complex road conditions, a higher placement is clearly advantageous for monitoring distant road conditions and making advance predictions. When installed at a certain height, it can even detect vehicles ahead of it.

[0183] In some alternative embodiments, the lidar is positioned downwards at an angle at the front of the autonomous vehicle. This results in a smaller blind spot in the vertical direction and better identification of low-lying obstacles in the near field.

[0184] For example, when encountering complex scenarios such as entering or exiting underground parking garages, cutting in suddenly, or crossing intersections, it can more effectively detect the perspectives of vehicles and pedestrians approaching from the side.

[0185] This application also provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it implements the function of any of the above-described devices or the steps of any of the above-described methods. The specific implementation method is consistent with the implementation method and the technical effect achieved in the above-described method embodiments, and some contents will not be repeated.

[0186] See Figure 5 , Figure 5 A schematic diagram of a program product for implementing the above-described method is shown in this application. The program product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this application, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0187] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0188] This application elaborates on the purpose of use, effectiveness, progress, and novelty. Its practical progress meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are only preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc. that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A method for calibrating extrinsic parameters of a laser radar, characterized in that, The method is applied to an autonomous vehicle, the autonomous vehicle comprising the laser radar and an inertial measurement unit, the laser radar being arranged in a downward and oblique manner at a front of the autonomous vehicle; the method comprising: acquiring point cloud data corresponding to a first reference surface and a second reference surface respectively by using the laser radar; the first reference surface being a ground surface, and the second reference surface being a wall surface; obtain a first normal vector corresponding to the first reference surface based on the point cloud data of the first reference surface and a first distance d1 between the first reference surface and the lidar obtain a second normal vector corresponding to the second reference surface based on the point cloud data of the second reference surface and a second distance d2 between the second reference surface and the lidar based on the first normal vector and the second normal vector , obtain attitude information of an inertial coordinate system corresponding to the inertial measurement unit in a radar coordinate system corresponding to the laser radar. acquiring position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2; acquiring a spatial conversion relationship between the inertial coordinate system and the radar coordinate system based on the attitude information and the position information, so as to realize the external parameter calibration of the laser radar; the acquiring of the position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2 comprises: respectively acquiring a third distance d3 between the first reference surface and the inertial measurement unit and a fourth distance d4 between the second reference surface and the inertial measurement unit; calculating a first difference value between the third distance d3 and the first distance d1 and a second difference value between the fourth distance d4 and the second distance d2; acquiring the position information of the inertial coordinate system in the radar coordinate system based on the first difference value and the second difference value; the process of acquiring the point cloud data of the wall surface by using the laser radar comprises: collecting overall point cloud data of the ground surface and the wall surface by using the laser radar; filtering out the point cloud data of the ground surface from the overall point cloud data to obtain the point cloud data of the wall surface.

2. The method of calibrating extrinsic parameters of a lidar according to claim 1, wherein, An angle between the first reference surface and the second reference surface is 90°.

3. The method of calibrating extrinsic parameters of a lidar according to claim 1, wherein, the first normal vector and the second normal vector obtaining attitude information of an inertial coordinate system corresponding to the inertial measurement unit in a radar coordinate system corresponding to the laser radar comprises: using the first normal vector, the second normal vector and a third normal vector constructing a virtual coordinate system, wherein = (x, y, z) ; acquiring attitude information of an inertial coordinate system corresponding to the inertial measurement unit in a radar coordinate system corresponding to the laser radar based on a corresponding relationship between the virtual coordinate system and the inertial coordinate system.

4. The method of calibrating extrinsic parameters of a lidar according to claim 1, wherein, The autonomous vehicle further comprises a camera and a path planning device, and the method further comprises: in a process of driving the autonomous vehicle, perceiving a visual blind area of the camera by using the laser radar; when the laser radar perceives that an obstacle exists in a preset area corresponding to the autonomous vehicle, converting coordinates of the obstacle in the radar coordinate system into reference coordinates corresponding to the inertial coordinate system; sending the reference coordinates and image data obtained by the camera to the path planning device, so that the path planning device performs path planning on an autonomous driving process of the autonomous vehicle based on the reference coordinates and the image data, thereby avoiding the obstacle. 5.A device for calibrating extrinsic parameters of a laser radar, applied to an autonomous vehicle, the autonomous vehicle comprising the laser radar and an inertial measurement unit, the laser radar being arranged in a downwardly inclined manner on a front of the autonomous vehicle; characterized in that, The external parameter calibration device comprises: a point cloud acquisition module, configured to acquire point cloud data corresponding to a first reference surface and a second reference surface respectively by using the laser radar; the first reference surface being a ground surface, and the second reference surface being a wall surface; a first point cloud module, configured to acquire a first normal vector corresponding to the first reference surface based on point cloud data of the first reference surface and a first distance d1 between the first reference surface and the lidar. a second point cloud module, configured to acquire a second normal vector corresponding to the second reference surface based on point cloud data of the second reference surface and a second distance d2 between the second reference surface and the lidar. An attitude obtaining module is configured to obtain attitude information of an inertial coordinate system corresponding to the inertial measurement unit in a radar coordinate system corresponding to the laser radar based on the first normal vector and the second normal vector . a position acquisition module, configured to acquire position information of the inertial coordinate system in the radar coordinate system based on the first distance d1 and the second distance d2. The space conversion module is configured to acquire a space conversion relationship between the inertial coordinate system and the radar coordinate system based on the attitude information and the position information, and to realize the external parameter calibration of the laser radar. The position information of the inertial coordinate system in the radar coordinate system is acquired based on the first distance d1 and the second distance d2, and the position information includes: A third distance d3 between the first reference surface and the inertial measurement unit and a fourth distance d4 between the second reference surface and the inertial measurement unit are acquired respectively; A first difference value between the third distance d3 and the first distance d1 and a second difference value between the fourth distance d4 and the second distance d2 are calculated; The position information of the inertial coordinate system in the radar coordinate system is acquired based on the first difference value and the second difference value. The process of acquiring the point cloud data of the wall surface by the laser radar includes: The laser radar is used to collect the overall point cloud data of the ground and the wall surface; The point cloud data of the ground is filtered out from the overall point cloud data to obtain the point cloud data of the wall surface.

6. An autonomous vehicle, comprising: The automatic driving vehicle includes a laser radar and an inertial measurement unit, and further includes a memory and a processor, the memory stores a computer program, and the processor realizes the steps of the method of any one of claims 1-4 when executing the computer program.

7. A computer readable storage medium characterized by The computer program is stored and executed by the processor to realize the steps of the method of any one of claims 1-4.

Images