A laser radar calibration method, device and storage medium

Abstract

A laser radar calibration method, device, and storage medium. The method includes: obtaining a point cloud collected by a laser radar when a vehicle passes through a target road, wherein at least one side of the target road is provided with a marker (601); performing preliminary screening on the collected point cloud according to a preset threshold value; the preset threshold value is determined by the installation height of the laser radar (602); performing multiple fitting processes on the preliminarily screened point cloud to obtain a ground point cloud (603); extracting a marker point cloud from the collected point cloud (604); and calibrating the external parameters of the laser radar according to the marker point cloud and the ground point cloud (605). The method reduces the requirements for the calibration site and improves the calibration efficiency and accuracy.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a laser radar calibration method, device, and storage medium. Background Art

[0002] In the field of intelligent driving, LiDAR is an essential component for achieving high-level autonomous driving. The accuracy of LiDAR's external parameter calibration plays a vital role in achieving functions such as perception, positioning, and fusion, as well as ensuring vehicle safety.

[0003] Existing methods for calibrating lidars have low accuracy, high requirements for the site, and high calibration costs. The calibration process relies on manual operation or mapping, resulting in low calibration efficiency. Summary of the Invention

[0004] In view of this, a laser radar calibration method, device and storage medium are proposed.

[0005] In the first aspect, an embodiment of the present application provides a laser radar calibration method, the method comprising: obtaining a point cloud collected by the laser radar when a vehicle passes through a target road, wherein at least one side of the target road is provided with a marker; performing preliminary screening on the collected point cloud according to a preset threshold; the preset threshold is determined by the installation height of the laser radar; performing multiple fitting processes on the preliminary screened point cloud to obtain a ground point cloud; extracting a marker point cloud from the collected point cloud; and calibrating the external parameters of the laser radar based on the marker point cloud and the ground point cloud.

[0006] Based on the above technical solution, the markers can be curbs, road barriers, etc. There are no special requirements for the site, and there is no need to set up additional calibration plates, targets, reflective stickers, etc., which reduces the calibration cost. On open roads (such as city streets, highways, etc.), the online dynamic calibration of the LiDAR can be completed by using the natural scene of the road. At the same time, by automatically extracting the marker point cloud from the collected point cloud; and calibrating the external parameters of the LiDAR based on the marker point cloud and the ground point cloud, fully automatic online dynamic calibration is achieved without manual operation, thereby improving the calibration efficiency. In addition, when extracting the ground point cloud, the collected point cloud is preliminarily screened according to the preset threshold; the preliminarily screened point cloud is subjected to multiple fitting processes, thereby adaptively extracting high-precision ground point clouds based on threshold filtering and multiple fitting processes; and the point cloud slicing can be further used to automatically extract high-precision marker point clouds, thereby improving the accuracy of the external parameters of the calibrated LiDAR.

[0007] According to the first aspect, in a first possible implementation of the first aspect, the method further includes: obtaining the position information of the marker point cloud and the position information of the ground point cloud corresponding to each laser radar based on the calibrated external parameters of multiple laser radars; obtaining an intersection feature point or an intersection domain based on the position information of the marker point cloud and the position information of the ground point cloud corresponding to each laser radar, wherein the intersection domain represents an area parallel to the direction of travel of the vehicle and centered on the intersection feature point; optimizing the calibrated external parameters of any laser radar among the multiple laser radars based on the intersection feature point or the intersection domain.

[0008] Based on the above technical solution, the position information of the marker point cloud and the ground point cloud corresponding to each laser radar are obtained, and then the external parameters of any laser radar are optimized by extracting cross feature points and cross domain features, thereby further improving the accuracy of the external parameters of the laser radar.

[0009] According to the first possible implementation method of the first aspect, in the second possible implementation method of the first aspect, the multiple laser radars include a master laser radar and a slave laser radar, wherein the master laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the side and / or rear environment of the vehicle; the position information of the marker point cloud and the ground point cloud corresponding to each laser radar are obtained based on the calibrated external parameters of the multiple laser radars, including: according to the calibrated external parameters of the master laser radar, the marker point cloud and the ground point cloud corresponding to the master laser radar are converted into the vehicle body coordinate system to obtain the position information of the marker point cloud and the ground point cloud corresponding to the master laser radar; according to the calibrated external parameters of the slave laser radar, the marker point cloud and the ground point cloud corresponding to the slave laser radar are converted into the vehicle body coordinate system to obtain the position of the marker point cloud corresponding to the slave laser radar information and the position information of the ground point cloud; the obtaining of intersection feature points or intersection domains according to the position information of the marker point cloud corresponding to each laser radar and the position information of the ground point cloud, including: obtaining a first intersection feature point or a first intersection domain according to the position information of the ground point cloud corresponding to the master laser radar and the position information of the ground point cloud corresponding to the slave laser radar; obtaining a second intersection feature point or a second intersection domain according to the position information of the marker point cloud corresponding to the master laser radar and the position information of the marker point cloud corresponding to the slave laser radar; the optimizing of the calibrated external parameters of any laser radar among the multiple laser radars according to the intersection feature points or intersection domains, including: optimizing the pitch angle and roll angle of the calibrated slave laser radar according to the first intersection feature point or the first intersection domain; optimizing the yaw angle of the calibrated slave laser radar according to the second intersection feature point or the second intersection domain.

[0010] Based on the above technical solution, both the main laser point cloud and the side laser point cloud are converted to the vehicle body coordinate system through the corresponding calibrated external parameters. Then, based on the marker point cloud, the intersection feature points and the intersection domain are extracted to optimize and compensate the yaw angle of the side laser radar to the vehicle body coordinate system; based on the ground points, the intersection feature points and the intersection domain are extracted to optimize and compensate the pitch angle and roll angle of the side laser radar to the vehicle body coordinate system, thereby completing the joint optimization of the external parameters of multiple laser radars, making the external parameters of the laser radar to the vehicle body coordinate system more accurate.

[0011] According to the first aspect or various possible implementations of the above-mentioned first aspect, in a third possible implementation of the first aspect, the external parameters include at least one of the pitch angle, roll angle, and yaw angle; the extrinsic parameters of the lidar are calibrated according to the marker point cloud and the ground point cloud, including: calibrating the pitch angle and roll angle of the lidar according to the ground point cloud; and calibrating the yaw angle of the lidar according to the marker point cloud.

[0012] Based on the above technical solution, the high-precision ground point cloud extracted is used to make the calibrated pitch angle and roll angle more accurate; the high-precision marker point cloud extracted is used to make the calibrated yaw angle more accurate.

[0013] According to the first aspect or various possible implementations of the above-mentioned first aspect, in a fourth possible implementation of the first aspect, the extracting of the marker point cloud from the collected point cloud includes: filtering out the ground point cloud from the collected point cloud; dividing the filtered point cloud into multiple slices along a direction perpendicular to the vehicle's travel; extracting the marker point cloud, the marker point cloud including feature points in a slice set that meets preset conditions, wherein the slice set includes one or more adjacent target slices, and the number of feature points in the target slice exceeds a threshold.

[0014] Based on the above technical solution, by dividing the point cloud into slices and extracting the marker point cloud, automatic extraction of high-precision marker point cloud is achieved.

[0015] According to the first aspect or various possible implementations of the above-mentioned first aspect, in the fifth possible implementation of the first aspect, the method also includes: obtaining first beam information of the ground point cloud, and downsampling the ground point cloud according to the first beam information; and / or obtaining second beam information of the marker point cloud, and downsampling the marker point cloud according to the second beam information; calibrating the external parameters of the lidar according to the marker point cloud and the ground point cloud, including: calibrating the external parameters of the lidar according to the marker point cloud and the ground point cloud after downsampling.

[0016] Based on the above technical solution, downsampling is performed based on the line bundle information of each ground point to extract accurate ground points, while improving processing efficiency and fully preserving the ground's texture structure, thus ensuring the accuracy of the ground point cloud. Alternatively, downsampling can be performed based on the line bundle information of each feature point to extract accurate feature point clouds; while improving processing efficiency and fully preserving the texture structure of the landmark, thus ensuring the accuracy of the landmark point cloud.

[0017] According to the first aspect or various possible implementations of the above-mentioned first aspect, in a sixth possible implementation of the first aspect, the acquired point cloud is a point cloud collected by a laser radar when the vehicle is traveling in a straight line.

[0018] Based on the above technical solution, when the vehicle is traveling in a straight line, the markers on both sides of the road are parallel to the vehicle's forward direction. The point cloud collected by the lidar in this state is used for calibration, thereby improving the accuracy of external parameters such as the lidar's yaw angle.

[0019] According to the first aspect or various possible implementations of the above-mentioned first aspect, in a seventh possible implementation of the first aspect, the marker includes at least one of a curb, a guardrail, and a building.

[0020] Based on the above technical solution, the markers can be road curbs, guardrails, buildings, etc. There are no special requirements for the site, and there is no need to set up additional calibration plates, targets, reflective stickers, etc., which reduces the calibration cost. Online calibration can be completed on open roads (such as city streets, highways, etc.).

[0021] In the second aspect, an embodiment of the present application provides a laser radar calibration method, the method comprising: obtaining a point cloud collected by the laser radar when a vehicle passes through a target area; a marker is vertically arranged on at least one side of the target area; extracting the marker point cloud from the collected point cloud; obtaining the fitting line information of the marker based on the marker point cloud; the fitting line information includes the position and direction information of the fitting line; and obtaining the numerical value of the laser radar external parameter based on the fitting line information.

[0022] Based on the above technical solution, at least one side of the target area is vertically set with a marker. The marker is simple to set, which reduces the requirements for the site and has low construction costs. According to the marker point cloud, the fitting line information of the marker is obtained. The fitting line based on the vertical marker needs to meet the vertical constraint to obtain the value of the laser radar external parameter. In this way, the value of the laser radar external parameter can be calculated based on the marker point cloud. At the same time, since the calibration process does not rely on the ground point cloud, it can be applied to scenes with insufficient ground information (for example, a laser radar with a small vertical field of view cannot collect nearby ground point clouds, a laser radar cannot collect valid ground point clouds due to limited site size, a laser radar with an excessively large pitch angle is lifted upward, resulting in missing or less ground point clouds, etc.), thereby achieving high-precision calibration of a single laser radar in scenes with insufficient ground information. In addition, compared with methods such as mapping and calibration, the entire calibration process can be automatically executed, which improves the efficiency of single laser radar calibration.

[0023] According to the second aspect, in a first possible implementation of the second aspect, the external parameters include a pitch angle, and the method further includes: when the angle between the laser radar and the vertical upward direction is less than a first preset threshold, and the value of the pitch angle is greater than a second preset threshold, calibrating the external parameters of the laser radar according to the marker point cloud, wherein the second preset threshold is determined by the vertical field of view angle of the laser radar.

[0024] Based on the above technical solution, when the angle between the laser radar direction and the vertical upward direction is less than the first preset threshold, and the value of the pitch angle is greater than the second preset threshold, it indicates that the laser radar direction is biased upward and the ground point cloud may be insufficient. In this way, based on the marker point cloud, high-precision calibration of the laser radar external parameters can be completed; it can be effectively applied to scenarios where ground information is insufficient.

[0025] According to the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the external parameter includes a yaw angle; the method also includes: obtaining the position information of the vehicle and the position information of the laser radar; determining the heading angle of the vehicle based on the position information of the vehicle and the position information of the laser radar; and optimizing the yaw angle of the calibrated laser radar based on the heading angle.

[0026] Based on the above technical solution, considering that it is difficult for a vehicle to travel in a straight line during driving, the vehicle's driving angle can be unrestricted and the calibrated yaw angle can be optimized by combining the vehicle's motion information.

[0027] According to various possible implementations of the second aspect above, in a third possible implementation of the second aspect, the vehicle is equipped with a master lidar and a slave lidar, wherein the master lidar is used to scan the front environment of the vehicle, and the slave lidar is used to scan the side and / or rear environment of the vehicle; the method also includes: determining the position information of the multiple markers based on the calibrated external parameters of the master lidar and the multiple marker point clouds collected by the master lidar; obtaining the predicted position of the first marker based on the position information of the multiple markers; obtaining the measured position of the first marker based on the calibrated external parameters of the slave lidar and the first marker point cloud collected by the slave lidar; and optimizing the external parameters of the slave lidar by comparing the predicted position with the measured position.

[0028] Based on the above technical solution, based on the results of single laser calibration, the distance and / or orientation between the two solved markers is used to predict the positions of the remaining markers, and the joint optimization of the pitch angle, yaw angle, and roll angle between multiple laser radars in any orientation is achieved. For scenarios where multiple laser radars have large deviations in installation position and angle, and point clouds are projected to different spatial positions, which are not suitable for direct point cloud registration, this method can effectively improve the accuracy of calibration, thereby achieving joint calibration of multiple laser radars with no common view area or a small common view area. In addition, this method does not require advance mapping, which significantly improves the efficiency of multi-lidar calibration.

[0029] According to various possible implementations of the second aspect above, in a fourth possible implementation of the second aspect, the method further includes: extracting a ground point cloud from the collected point cloud; calibrating the external parameters of the lidar based on the marker point cloud, and also includes: calibrating the external parameters of the lidar based on the marker point cloud and the ground point cloud.

[0030] Based on the above technical solution, when there is a valid ground point cloud, the ground point cloud can be fully utilized to further improve the calibration accuracy and stability.

[0031] According to the second aspect or various possible implementations of the above-mentioned second aspect, in the fifth possible implementation of the second aspect, obtaining the fitting line information of the marker based on the marker point cloud includes: determining an initial value of the rotation angle based on the marker point cloud, the initial value of the rotation angle minimizes the projection area of ​​the horizontal plane in the lidar coordinate system after the marker point cloud is rotated; rotating the marker point cloud according to the initial value of the rotation angle; and obtaining the fitting line information of the marker using the rotated marker point cloud.

[0032] According to various possible implementations of the second aspect above, in a sixth possible implementation of the second aspect, the method further includes: when the angle between the direction of the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than the second preset threshold, calibrating the external parameters of the laser radar based on the marker point cloud and the ground point cloud.

[0033] Based on the above technical solution, when there is a valid ground point cloud, the ground point cloud can be fully utilized to further improve the calibration accuracy and stability.

[0034] According to the second aspect or various possible implementations of the above-mentioned second aspect, in a seventh possible implementation of the second aspect, the external parameter includes at least one of a pitch angle, a roll angle, and a yaw angle, and the method further includes: when the angle between the orientation of the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than the second preset threshold, calibrating the pitch angle and roll angle of the laser radar according to the ground point cloud; and calibrating the yaw angle of the laser radar according to the position information of the fitting line.

[0035] Based on the above technical solution, the pitch angle and roll angle of the lidar can be calibrated using the ground point cloud, thereby improving the accuracy and stability of the calibrated pitch angle and roll angle; the yaw angle of the lidar can be calibrated using the position information of the fitting line, thereby improving the accuracy of the calibrated yaw angle.

[0036] According to the second aspect or various possible implementations of the above-mentioned second aspect, in an eighth possible implementation of the second aspect, a plurality of markers are vertically arranged on at least one side of the target area, and the intersection points of the plurality of markers and the ground are on the same straight line.

[0037] In a third aspect, an embodiment of the present application provides a laser radar calibration device, the device comprising: an acquisition module for acquiring a point cloud collected by the laser radar when a vehicle passes through a target road, wherein at least one side of the target road is provided with a marker; a screening module for performing preliminary screening of the collected point cloud according to a preset threshold; the preset threshold is determined by the installation height of the laser radar; a first extraction module for performing multiple fitting processes on the preliminarily screened point cloud to obtain a ground point cloud; a second extraction module for extracting a marker point cloud from the collected point cloud; a calibration module for calibrating the external parameters of the laser radar based on the marker point cloud and the ground point cloud.

[0038] According to the third aspect, in a first possible implementation of the third aspect, the device further includes: a conversion module for obtaining the position information of the marker point cloud and the ground point cloud corresponding to each laser radar based on the calibrated external parameters of multiple laser radars; a third extraction module for obtaining the intersection feature point or intersection domain based on the position information of the marker point cloud and the ground point cloud corresponding to each laser radar, wherein the intersection domain represents an area parallel to the direction of travel of the vehicle and centered on the intersection feature point; an optimization module for optimizing the calibrated external parameters of any laser radar among the multiple laser radars based on the intersection feature point or intersection domain.

[0039] According to a first possible implementation of the third aspect, in a second possible implementation of the third aspect, the multiple laser radars include a master laser radar and a slave laser radar, wherein the master laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the side and / or rear environment of the vehicle; the conversion module is further used to: convert the marker point cloud and the ground point cloud corresponding to the master laser radar into the vehicle body coordinate system according to the calibrated external parameters of the master laser radar, and obtain the position information of the marker point cloud and the ground point cloud corresponding to the master laser radar; according to the calibrated external parameters of the slave laser radar, The calibrated external parameters of the radar convert the marker point cloud and the ground point cloud corresponding to the slave laser radar into the vehicle coordinate system to obtain the position information of the marker point cloud and the ground point cloud corresponding to the slave laser radar; the third extraction module is also used to: obtain the first intersection feature point or the first intersection domain according to the position information of the ground point cloud corresponding to the master laser radar and the position information of the ground point cloud corresponding to the slave laser radar; obtain the second intersection feature point or the second intersection domain according to the position information of the marker point cloud corresponding to the master laser radar and the position information of the marker point cloud corresponding to the slave laser radar.

[0040] The optimization module is also used to: optimize the pitch angle and roll angle calibrated from the laser radar according to the first intersection feature point or the first intersection domain; and optimize the yaw angle calibrated from the laser radar according to the second intersection feature point or the second intersection domain.

[0041] According to the third aspect or various possible implementations of the above-mentioned third aspect, in the third possible implementation of the third aspect, the external parameters include at least one of the pitch angle, roll angle, and yaw angle; the calibration module is also used to: calibrate the pitch angle and roll angle of the laser radar according to the ground point cloud; calibrate the yaw angle of the laser radar according to the marker point cloud.

[0042] According to the third aspect or various possible implementations of the third aspect, in a fourth possible implementation of the third aspect, the second extraction module is further used to: filter out the ground point cloud in the collected point cloud; divide the filtered point cloud into multiple slices along a direction perpendicular to the vehicle's travel; extract the marker point cloud, the marker point cloud including feature points in a slice set that meets preset conditions, wherein the slice set includes one or more adjacent target slices, and the number of feature points in the target slice exceeds a threshold.

[0043] According to the third aspect or various possible implementations of the above-mentioned third aspect, in a fifth possible implementation of the third aspect, the device further includes a downsampling module, which is used to: obtain first line beam information of a ground point cloud, and downsample the ground point cloud according to the first line beam information; and / or obtain second line beam information of a marker point cloud, and downsample the marker point cloud according to the second line beam information; the calibration module is further used to: calibrate the external parameters of the lidar according to the marker point cloud and the ground point cloud after downsampling.

[0044] According to the third aspect or various possible implementations of the third aspect, in a sixth possible implementation of the third aspect, the acquired point cloud is a point cloud collected by a lidar when the vehicle is traveling in a straight line.

[0045] According to the third aspect or various possible implementations of the third aspect, in a seventh possible implementation of the third aspect, the marker includes at least one of a curb, a guardrail, and a building.

[0046] In a fourth aspect, an embodiment of the present application provides a laser radar calibration device, which includes: an acquisition module for acquiring a point cloud collected by the laser radar when a vehicle passes through a target area; a marker is vertically arranged on at least one side of the target area; an extraction module for extracting the marker point cloud from the collected point cloud; a fitting module for obtaining fitting line information of the marker based on the marker point cloud; the fitting line information includes the position and direction information of the fitting line; and a calculation module for obtaining the numerical value of the laser radar external parameter based on the fitting line information.

[0047] According to the fourth aspect, in a first possible implementation of the fourth aspect, the external parameters include a pitch angle; the device also includes: a calibration module, which is used to calibrate the external parameters of the laser radar according to the marker point cloud when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is greater than a second preset threshold, wherein the second preset threshold is determined by the vertical field of view angle of the laser radar.

[0048] According to the first possible implementation of the fourth aspect, in the second possible implementation of the fourth aspect, the external parameter includes a yaw angle; the device also includes: an optimization module, used to obtain the position information of the vehicle and the position information of the laser radar; based on the position information of the vehicle and the position information of the laser radar, determine the heading angle of the vehicle; based on the heading angle, optimize the yaw angle of the calibrated laser radar.

[0049] According to various possible implementations of the fourth aspect above, in a third possible implementation of the fourth aspect, the vehicle is equipped with a master lidar and a slave lidar, wherein the master lidar is used to scan the front environment of the vehicle, and the slave lidar is used to scan the side and / or rear environment of the vehicle; the device also includes: a determination module, used to determine the position information of the multiple markers based on the calibrated external parameters of the master lidar and the multiple marker point clouds collected by the master lidar; a prediction module, used to obtain the predicted position of the first marker based on the position information of the multiple markers; a measurement module, used to obtain the measured position of the first marker based on the calibrated external parameters of the slave lidar and the first marker point cloud collected by the slave lidar; a matching module, used to optimize the external parameters of the slave lidar by matching the predicted position with the measured position.

[0050] According to various possible implementations of the fourth aspect above, in a fourth possible implementation of the fourth aspect, the extraction module is further used to extract the ground point cloud from the collected point cloud; the calibration module is further used to calibrate the external parameters of the lidar based on the marker point cloud and the ground point cloud.

[0051] According to the fourth aspect or various possible implementations of the above-mentioned fourth aspect, in the fifth possible implementation of the fourth aspect, the fitting module is further used to: determine an initial value of the rotation angle based on the marker point cloud, and the initial value of the rotation angle minimizes the projection area of ​​the horizontal plane in the lidar coordinate system after the marker point cloud is rotated; rotate the marker point cloud according to the initial value of the rotation angle; and obtain the fitting line information of the marker using the rotated marker point cloud.

[0052] According to the fourth aspect or various possible implementations of the above-mentioned fourth aspect, in a sixth possible implementation of the fourth aspect, the calibration module is further used to: calibrate the external parameters of the laser radar according to the marker point cloud and the ground point cloud, when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than the second preset threshold.

[0053] According to the fourth aspect or various possible implementations of the above-mentioned fourth aspect, in a seventh possible implementation of the fourth aspect, the external parameter includes at least one of a pitch angle, a roll angle, and a yaw angle; the calibration module is further used to: when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than the second preset threshold, calibrate the pitch angle and roll angle of the laser radar according to the ground point cloud; and calibrate the yaw angle of the laser radar according to the position information of the fitting line.

[0054] According to the fourth aspect or various possible implementations of the above-mentioned fourth aspect, in an eighth possible implementation of the fourth aspect, a plurality of markers are vertically arranged on at least one side of the target area, and the intersection points of the plurality of markers and the ground are on the same straight line.

[0055] In a fifth aspect, an embodiment of the present application provides a laser radar calibration device, comprising: at least one processor; a memory for storing processor-executable instructions; wherein the at least one processor is configured to implement the above-mentioned first aspect or one or more laser radar calibration methods of the first aspect when executing the instructions, or to implement the above-mentioned second aspect or one or more laser radar calibration methods of the second aspect.

[0056] In the sixth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, they implement the calibration method of the first aspect or one or more of the laser radars of the first aspect, or implement the calibration method of the second aspect or one or more of the laser radars of the second aspect.

[0057] In the seventh aspect, an embodiment of the present application provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in an electronic device, the processor in the electronic device implements the calibration method of the first aspect or one or more of the laser radars of the first aspect, or implements the calibration method of the second aspect or one or more of the laser radars of the second aspect.

[0058] For the technical effects of each of the above-mentioned aspects from the third to the seventh, and various possible implementation methods of each aspect, please refer to the above-mentioned first or second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram illustrating an application scenario of a laser radar calibration method according to an embodiment of the present application is shown;

[0060] Figure 2 (a)- Figure 2 (b) shows a schematic diagram of a road where a vehicle is located according to an embodiment of the present application;

[0061] Figure 3 A schematic diagram illustrating an application scenario of a laser radar calibration method according to an embodiment of the present application is shown;

[0062] Figure 4 (a)- Figure 4 (e) shows a schematic diagram of a road where a vehicle is located according to an embodiment of the present application;

[0063] Figure 5 (a)- Figure 5 (d) shows a schematic diagram of several coordinate systems according to an embodiment of the present application;

[0064] Figure 6 A flowchart of a laser radar calibration method according to an embodiment of the present application is shown;

[0065] Figure 7 A schematic diagram of a point cloud slice according to an embodiment of the present application is shown;

[0066] Figure 8 A flowchart of a laser radar calibration method according to an embodiment of the present application is shown;

[0067] Figure 9 A schematic diagram of time synchronization according to an embodiment of the present application is shown;

[0068] Figure 10 A schematic diagram of an intersection feature point according to an embodiment of the present application is shown;

[0069] Figure 11 A schematic diagram of a cross-domain according to an embodiment of the present application is shown;

[0070] Figure 12 A flowchart of another laser radar calibration method according to an embodiment of the present application is shown;

[0071] Figure 13 A schematic diagram of a single laser radar calibration according to an embodiment of the present application is shown;

[0072] Figure 14 A schematic diagram of a laser radar scanning scene according to an embodiment of the present application is shown;

[0073] Figure 15 (a)- Figure 15 (b) shows a schematic diagram of a production line environment according to an embodiment of the present application;

[0074] Figure 16A comparative schematic diagram of calibrating the pitch angle according to an embodiment of the present application is shown;

[0075] Figure 17 A flowchart of a laser radar calibration method according to an embodiment of the present application is shown;

[0076] Figure 18 (a)-18(b) show a schematic diagram of a multi-lidar joint optimization according to an embodiment of the present application;

[0077] Figure 19 A structural diagram of a laser radar calibration device according to an embodiment of the present application is shown;

[0078] Figure 20 A structural diagram of a laser radar calibration device according to an embodiment of the present application is shown;

[0079] Figure 21 A schematic structural diagram of a laser radar calibration device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0080] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0081] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0082] The following first illustrates the application scenarios of the laser radar calibration method provided in the embodiments of the present application.

[0083] Figure 1 A schematic diagram showing an application scenario of a laser radar calibration method according to an embodiment of the present application is shown. Figure 1 As shown, the application scenario may include: vehicle 101, road 102, laser radar 103, and marker 104; wherein, the laser radar 103 is installed on the vehicle 101, and the marker 104 may be a curb, road guard, building facade, etc.

[0084] For example, the road 103 may be an open road, for example, Figure 2 (a)- Figure 2 (b) shows a schematic diagram of a road where a vehicle is located according to an embodiment of the present application, such as Figure 2 As shown in (a), the road 103 may be an urban road, which is relatively flat and has curbs or guardrails of a certain height on both sides; Figure 2 As shown in (b), the road 103 may be a highway, which has a relatively flat road surface and clear curbs.

[0085] In some examples, the vehicle 101 may be Figure 2 (a) or Figure 2 Driving on the road shown in (b), the vehicle's own speed can be greater than 40 km / h, and it maintains a straight line for a period of time. During the driving process, the laser radar 103 scans the external environment of the vehicle and completes the point cloud collection work. The point cloud collected by the laser radar 103 is processed using the laser radar calibration method of the embodiment of the application (described in detail below), thereby achieving the external parameter calibration of the laser radar 103.

[0086] Figure 3 A schematic diagram showing an application scenario of a laser radar calibration method according to an embodiment of the present application is shown. Figure 3 As shown, the application scenario may include: vehicle 301, road 302, laser radar 303, and marker 304; wherein, the laser radar 303 is installed on the vehicle 301, and the marker 304 can be vertically set on the road 303. For example, the road 303 can be a dedicated road with a length of 30-100m and a width of 3-8m; for example, the marker 304 can be made of metal, or a reflective sticker can be affixed to the surface of the marker 304 to improve the laser reflectivity; the cross-section of the marker 304 can be circular, square, triangular, etc., and the geometric characteristics of the cross-section are known. For example, it can be a cylindrical vertical rod, the cross-section type of the vertical rod can be a circular cross-section, and the radius is known, and the surface of the vertical rod can be affixed with reflective tape; illustratively, the number of markers 304 can be multiple, and the markers 304 can be set on one side or both sides of the road 302. When multiple markers 304 are set on any side of the road 302, the intersection points of each marker 304 with the ground are on the same straight line, and the spacing between each marker 304 can be the same or different.

[0087] For example, Figure 4 (a)- Figure 4 (e) shows a schematic diagram of a road where a vehicle is located according to an embodiment of the present application, such as Figure 4 As shown in (a), a row of markers can be set up on one side of the road (the figure takes the markers set up on the left side of the road as an example), such as Figure 4 As shown in (b), a row of markers is set on each side of the road, and the two rows of markers are symmetrically distributed along the center line of the road, and the distance between adjacent markers in any row is the same; Figure 4 As shown in (c), a row of markers is set on each side of the road. The two rows of markers are parallel to each other, and the distance between adjacent markers in any row is the same; Figure 4As shown in (d), a row of markers is set on each side of the road. The two rows of markers are parallel to each other, and the adjacent markers in any row are not exactly the same; Figure 4 As shown in (e), the road surface can be uneven, and the road can be set up as above. Figure 4 (a)- Figure 4 (d) Any of the markers shown.

[0088] In some examples, the vehicle 301 may be Figure 4 (a)- Figure 4 The vehicle enters from one end of the road shown in (e) and exits from the other end of the road. The vehicle speed can be 5-40 km / h, and can maintain a constant speed or a straight line. During the driving process, the laser radar 303 scans the external environment of the vehicle and completes the point cloud collection work. The point cloud collected by the laser radar 303 is processed using the laser radar calibration method in the embodiment of the present application (described in detail below), thereby achieving the external parameter calibration of the laser radar 303.

[0089] It should be noted that the embodiment of the present application does not limit the number and type of laser radars installed on the vehicle. For example, the number of laser radars 103 or laser radars 303 can be one or more. Figure 1 and Figure 3 In each example, two laser radars are used. More laser radars 103 can be installed on vehicle 101, or more laser radars 303 can be installed on vehicle 301, as needed. For example, the laser radars 103 and 303 can include a master laser radar, which detects the environment in front of the vehicle, behind the vehicle, or all around the vehicle. They can also include slave laser radars, which detect the environment to the sides of the vehicle (also known as side laser radars) or behind the vehicle (also known as rear laser radars). Compared to the slave laser radars, the master laser radar can detect obstacles in front of the vehicle.

[0090] For example, the vehicle 101 or 301 may further include a positioning device (not shown in the figure), which may include a wheel speed meter, a global navigation satellite system (GNSS), an inertial navigation system (INS), etc., for obtaining the vehicle's position information.

[0091] For example, Figure 1 or Figure 3The application scenario shown may also include a lidar calibration device (not shown in the figure). The lidar calibration method provided in the embodiment of the present application can be implemented by the lidar calibration device, which is used to perform efficient automatic data processing on the point cloud collected by the above-mentioned lidar 103 or lidar 303, and the external parameter accuracy of the calibrated lidar is relatively high.

[0092] The embodiments of the present application do not limit the type of the laser radar calibration device.

[0093] Exemplarily, the laser radar calibration device can be the above-mentioned vehicle 101 (or vehicle 301), or other components with data processing functions in the vehicle 101 (or vehicle 301), such as: a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, a vehicle-mounted unit, and a vehicle-mounted sensor. The vehicle can use the vehicle-mounted terminal, vehicle-mounted controller, vehicle-mounted module, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip, vehicle-mounted unit, and vehicle-mounted sensor.

[0094] Exemplarily, the lidar calibration device is integrated into an automated driving system (ADS) or an advanced driver assistance system (ADAS) or an on-board computing platform of the vehicle 101 or the vehicle 301 .

[0095] For example, the laser radar calibration device can also be a smart terminal with data processing capabilities other than a vehicle, or a component or chip installed in the smart terminal. For example, the smart terminal can be a device equipped with a laser radar, such as a smart transportation device, a smart wearable device, a smart home device, a smart auxiliary aircraft, a robot, or an unmanned aerial vehicle.

[0096] For example, the laser radar calibration device can be a general-purpose device or a dedicated device. In a specific implementation, the device can also be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, or other device with data processing capabilities, or a component or chip within such devices.

[0097] For example, the laser radar calibration device may also be a chip or processor with processing capabilities, and the laser radar calibration device may include multiple processors. The processor may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The chip or processor with processing capabilities may be set in the laser radar, or may not be set in the laser radar, but may be set at the receiving end of the laser radar output signal.

[0098] It should be noted that the above-mentioned application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems when other similar or new application scenarios emerge.

[0099] Figure 5 (a)- Figure 5 (d) shows a schematic diagram of several coordinate systems according to an embodiment of the present application, Figure 5 (a)- Figure 5 (b) shows the vehicle coordinate system. The origin of the vehicle coordinate system can coincide with the vehicle's center of mass. When the vehicle is stationary on a level surface, the x-axis is parallel to the ground and points forward, the z-axis passes through the vehicle's center of mass and points upward, and the y-axis points to the driver's left. Exemplarily, the external parameters of the lidar may include one or more of the following: pitch, roll, and yaw angles. They may also include the lidar's installation height. The pitch angle represents the angle of rotation around the y-axis, the yaw angle represents the angle of rotation around the z-axis, and the roll angle represents the angle of rotation around the x-axis. Figure 5 (c) shows the laser radar coordinate system. The origin of the laser radar coordinate system can coincide with the laser radar center of mass. Figure 5 (d) shows the world coordinate system. In the X'Y' plane, the yaw angle of the lidar in the vehicle coordinate system, the vehicle heading angle, and the lidar heading angle can be seen.

[0100] Based on the above Figure 1 The application scenario described above provides a detailed description of the laser radar calibration method provided in the embodiment of the present application.

[0101] Figure 6 FIG1 shows a flow chart of a laser radar calibration method according to an embodiment of the present application; the method can be executed by the laser radar calibration device described above; Figure 6 As shown, the method may include the following steps:

[0102] Step 601: Obtain a point cloud collected by a laser radar when a vehicle passes through a target road, where a marker is set on at least one side of the target road.

[0103] The laser radar may be any laser radar installed on the vehicle, for example, a main laser radar or a side laser radar.

[0104] For example, the target road may be a flat open road, and the landmarks may be curbs, guardrails, or buildings on one or both sides of the open road; for example, the vehicle may be the aforementioned vehicle 101, and the target road may be the aforementioned Figure 2 (a) or Figure 2 The road shown in (b).

[0105] For example, the acquired point cloud may be a point cloud collected by a laser radar when the vehicle is traveling in a straight line. Figure 2 (a) or Figure 2 (b) Point cloud collected by the laser radar 103 when driving in a straight line on the road shown.

[0106] For example, the driving state of the vehicle can be determined based on the posture information of the vehicle when passing the target road; for example, the inertial navigation system can be used to determine that the vehicle is in a straight-moving state when passing the target road, and the point cloud collected by the on-board lidar can be extracted.

[0107] Step 602: Perform a preliminary screening on the collected point cloud according to a preset threshold.

[0108] Exemplarily, the preset threshold is determined by the installation height of the laser radar, wherein the installation height of the laser radar can be predetermined; for example, the preset threshold can be equal to the installation height of the laser radar.

[0109] Exemplarily, the collected point cloud includes the coordinate information (X value, Y value, Z value) of each scanning point under the laser radar coordinates; when the scanning point is a ground point, the corresponding height value (i.e., Z value) is usually small and will not exceed the installation height of the laser radar; therefore, by judging whether the height value of each scanning point in the collected point cloud exceeds a preset threshold, the scanning points whose height values ​​do not exceed the preset threshold can be screened out, and all the screened out scanning points can be regarded as a coarse-grained ground point cloud.

[0110] In this step, any frame point cloud obtained in the above step 601 is preliminarily screened according to a preset threshold determined by the installation height of the laser radar to screen out coarse-grained ground point clouds, thereby reducing the number of point clouds and improving data processing efficiency.

[0111] Step 603: Perform multiple fitting processes on the initially screened point cloud to obtain a ground point cloud.

[0112] For example, a multi-step random sampling consistency fitting algorithm may be applied to the point cloud initially screened in step 602 to adaptively extract high-precision ground points.

[0113] In one possible implementation, a plane is fitted using a random sampling consensus algorithm for a coarse-grained ground point cloud; scanning points in the coarse-grained ground point cloud whose distance from the fitted plane is greater than a preset distance threshold are filtered out to obtain a medium-grained ground point cloud, where the initial value of the preset distance threshold can be an empirical value; the distance threshold is reduced, and the above-mentioned step of fitting the plane using the random sampling consensus algorithm is continued, and scanning points in the medium-grained ground point cloud whose distance from the fitted plane is greater than the reduced distance threshold are further filtered out, thereby obtaining a fine-grained ground point cloud. It is understood that the operation of reducing the distance threshold and fitting the plane using the random sampling consensus algorithm can be further performed through multiple iterations as needed to obtain a more accurate ground point cloud.

[0114] In one possible implementation, first line bundle information of the ground point cloud can also be obtained, and the ground point cloud can be downsampled based on the first line bundle information. The collected point cloud can include line bundle information for each feature point, or the line bundle information for each feature point in the collected point cloud can be determined based on the configuration parameters of the lidar. Thus, for the fine-grained ground points obtained above, based on the line bundle information of each ground point, downsampling is performed at intervals to extract precise ground points. This improves data processing efficiency while fully preserving the ground's texture structure, thereby ensuring the accuracy of the ground point cloud.

[0115] Through the above steps 602 and 603, without manual operation, threshold filtering and multiple fitting processes are used to automatically extract accurate ground point clouds, and the final ground point clouds can be obtained by downsampling according to the line bundle information of the ground points.

[0116] Step 604: extract the marker point cloud from the collected point cloud.

[0117] In one possible implementation, this step may include: filtering out the ground point cloud in the collected point cloud. The scan points in the filtered point cloud may include feature points of markers, or non-ground points such as vehicles and pedestrians on the road; the filtered point cloud is divided into multiple slices along a direction perpendicular to the direction of vehicle travel. The marker point cloud is extracted, and the marker point cloud includes feature points in a set of slices that meet preset conditions, wherein the set of slices includes one or more adjacent target slices, and the number of feature points in the target slices exceeds a preset threshold. In this way, by dividing the point cloud slices and extracting the marker point cloud, automatic extraction of high-precision marker point clouds is achieved.

[0118] For example, the width of each slice and the preset threshold value can be set as needed; the preset condition can include that the positions of all feature points in the slice set exceed the length range along the direction of vehicle travel.

[0119] For example, Figure 7 FIG. 1 shows a schematic diagram of a point cloud slice according to an embodiment of the present application; FIG. Figure 7 As shown, taking the main lidar installed on the vehicle as an example, Figure 7 The middle scanning point is: the scanning points in the filtered point cloud after filtering out the ground point cloud in a frame of point cloud collected by the main laser radar. Along the direction perpendicular to the vehicle's travel (such as along the y-axis of the vehicle coordinate system), the filtered point cloud is divided into multiple slices, namely S1, S2...SN in the figure, N is a positive integer, and the width of each slice can be 1m; starting from slice S1, for any slice Si, where i is a positive integer not greater than N, if the number of scanning points in Si is greater than the preset threshold, Si is marked as the target slice; for the target slice Si, continue to judge whether the next slice is the target slice, and so on, until the current slice is not the target slice or the number of target slices exceeds 5. For example, for the target slice Si, if Si+1 is still the target slice and Si+1 is not the target slice, then slices Si and Si+1 form a slice set. In this way, one or more slice sets can be obtained based on the N slices in S1-SN; for any slice set, judge whether the X value range of all feature points in the slice set exceeds the length range, that is, judge whether it satisfies |X max -X min |>L threshold , where X max Indicates the maximum value of the X values ​​of all feature points in the slice set, X min Indicates the minimum value of the X values ​​of all feature points in the slice set, L threshold Indicates the length range; then the feature points in the slice set that meet the conditions are used as the marker point cloud, so as to extract high-precision marker point cloud; Figure 7 As shown, slice S1 is the point cloud of the marker on the left side of the road, and slice SN is the point cloud of the marker on the right side of the road.

[0120] In one possible implementation, second-line information for the marker point cloud can also be obtained, and the marker point cloud can be downsampled based on this second-line information. Thus, based on the line information for each feature point, the marker point cloud obtained above is downsampled at intervals to extract a precise feature point cloud. This improves data processing efficiency while fully preserving the texture structure of the marker, thereby ensuring the accuracy of the marker point cloud.

[0121] Step 605: Calibrate the external parameters of the lidar based on the marker point cloud and the ground point cloud.

[0122] The high-precision ground point cloud and marker point cloud extracted in steps 603 and 604 are used to calibrate the external parameters of the laser radar.

[0123] For example, the external parameters of the lidar can be calibrated based on the marker point cloud and ground point cloud after the above-mentioned downsampling processing, thereby improving processing efficiency.

[0124] In one possible implementation, the pitch angle and roll angle of the lidar can be calibrated based on the above-mentioned ground point cloud; thereby utilizing the above-mentioned extracted high-precision ground point cloud to make the calibrated pitch angle and roll angle more accurate.

[0125] For example, the ground plane and the vehicle body horizontal plane can be taken as a constraint to establish an L1 loss function to solve the pitch angle and roll angle of the lidar.

[0126]

[0127] In the above formula (1), L g Represents the loss function corresponding to the pitch angle and roll angle, represents the z value of the i-th ground point in the vehicle coordinate system, n represents the number of ground points contained in the ground point cloud, Represents the mean z value of the ground point cloud. Use formula (1) to solve, so that L g The pitch angle and roll angle corresponding to the minimum are the pitch angle and roll angle of the lidar.

[0128] Furthermore, the yaw angle of the laser radar can be calibrated based on the above-mentioned marker point cloud; thereby, the accuracy of the calibrated yaw angle can be higher by utilizing the above-mentioned extracted high-precision marker point cloud.

[0129] For example, the parallelism between the marker and the vehicle's forward direction can be used as a constraint to establish an L1 loss function and calculate the yaw angle from the lidar coordinate system to the vehicle coordinate system.

[0130]

[0131] In the above formula (2), L w Represents the loss function corresponding to the yaw angle solution, represents the y value of the i-th feature point in the vehicle coordinate system, m represents the number of feature points contained in the landmark point cloud, Represents the mean y value of the landmark point cloud. Use formula (1) to solve, so that L w The yaw angle corresponding to the minimum is the yaw angle of the lidar.

[0132] Furthermore, after performing the above processing on a frame of point cloud extracted in step 601, the above processing can be performed on multiple frames of point cloud accumulated over a period of time, and the obtained calibration results can be statistically analyzed to optimize the external parameters of the final laser radar to vehicle coordinate system.

[0133] In this way, through the above steps 601-605, the scene information in the open road is used to extract the ground point cloud and marker point cloud of the point cloud collected by a single lidar, and the external parameters of the lidar to the vehicle coordinate system are solved. In some examples, multi-step fitting can be combined with threshold filtering to extract high-precision ground point clouds, and then the pitch and roll angles of the lidar can be solved. High-precision marker point clouds can be extracted through point cloud slicing, and then the yaw angle of the lidar can be solved; thereby achieving high-precision online dynamic calibration of a single lidar.

[0134] In the embodiment of the present application, the markers can be curbs, road barriers, etc. There are no special requirements for the site, and there is no need to set up additional calibration plates, targets, reflective stickers, etc., which reduces the calibration cost. On open roads (such as city streets, highways, etc.), the online dynamic calibration of the laser radar can be completed by using the natural scene of the road. At the same time, by automatically extracting the marker point cloud from the collected point cloud; and calibrating the external parameters of the laser radar based on the marker point cloud and the ground point cloud, fully automatic online dynamic calibration is achieved without manual operation, thereby improving the calibration efficiency. In addition, when extracting the ground point cloud, the collected point cloud is preliminarily screened according to a preset threshold; the preliminarily screened point cloud is subjected to multiple fitting processes, thereby adaptively extracting a high-precision ground point cloud based on threshold filtering and multiple fitting processes; and the high-precision marker point cloud can be automatically extracted by further slicing the point cloud, thereby improving the accuracy of the external parameters of the calibrated laser radar.

[0135] Furthermore, with the development of intelligent driving, low-cost, small-angle, and high-beam laser radars are becoming more widely used. Multiple laser radars on a vehicle can achieve scene coverage and complementarity. When a vehicle is equipped with multiple laser radars, the calibration results of any two laser radars can be further optimized based on the calibration results of the vehicle coordinate system. For example, the external parameters of the master laser radar calibrated through steps 601-605 and the external parameters of the slave laser radar calibrated through steps 601-605 can be used to optimize the external parameters of the slave laser radar.

[0136] Figure 8 FIG. 1 shows a flow chart of a laser radar calibration method according to an embodiment of the present application; FIG. Figure 8 As shown, the method may include:

[0137] Step 801: Based on the calibrated external parameters of multiple laser radars, obtain the position information of the marker point cloud and the position information of the ground point cloud corresponding to each laser radar.

[0138] Where multiple laser radars are installed on the same vehicle, illustratively, the extrinsic parameters of any laser radar can be calibrated through steps 601 to 605. The marker point cloud corresponding to each laser radar can be the marker point cloud extracted in step 604, and the ground point cloud corresponding to each laser radar can be the marker point cloud obtained in step 603. The position information of the marker point cloud represents the three-dimensional coordinates (x, y, and z) of the marker point cloud in the vehicle coordinate system.

[0139] Exemplarily, the multiple lidars may include a master lidar and slave lidars (e.g., side lidars).

[0140] In one possible implementation method, the marker point cloud and ground point cloud corresponding to the main laser radar can be converted into the vehicle body coordinate system according to the calibrated external parameters of the main laser radar, and the position information of the marker point cloud and the ground point cloud corresponding to the main laser radar can be obtained; according to the calibrated external parameters of the side laser radar, the marker point cloud and the ground point cloud corresponding to the side laser radar can be converted into the vehicle body coordinate system, and the position information of the marker point cloud and the ground point cloud corresponding to the side laser radar can be obtained.

[0141] In one possible implementation, the point cloud collected by the main laser radar can be converted into the vehicle body coordinate system based on the calibrated external parameters of the main laser radar to obtain the position information of the point cloud collected by the main laser radar, and then the position information of the ground point cloud corresponding to the main laser radar can be obtained by extracting the ground point cloud as described above; the position information of the marker point cloud corresponding to the main laser radar can be obtained by extracting the marker point cloud as described above. Similarly, the point cloud collected by the side laser radar can be converted into the vehicle body coordinate system based on the calibrated external parameters of the side laser radar to obtain the position information of the point cloud collected by the side laser radar, and then the position information of the ground point cloud corresponding to the side laser radar can be obtained by extracting the ground point cloud as described above; the position information of the marker point cloud corresponding to the side laser radar can be obtained by extracting the marker point cloud as described above.

[0142] For example, before performing the above conversion, time synchronization may be performed between multiple laser radars. Figure 9 A schematic diagram of time synchronization according to an embodiment of the present application is shown. Figure 9As shown in the figure, the arrow indicates the direction of the time axis, and the points on the time axis represent point cloud data packets. According to the timestamp of each point cloud data packet (i.e., the corresponding position of the point cloud data packet on the time axis), the data packets on the time axis of the main lidar are matched with the data packets on the time axis of the side lidar. If the time difference between the two adjacent nearest data packets on the time axis is less than the threshold, that is, the data packets are located at Figure 9 If the time is within the elliptical area shown in , the time synchronization is successful.

[0143] Step 802: Obtain intersection feature points or intersection domains based on the position information of the marker point clouds and the position information of the ground point clouds corresponding to each laser radar.

[0144] The intersection feature point represents the intersection of two types of point clouds, and the intersection domain represents the area centered on the intersection feature point, parallel to the direction of vehicle travel.

[0145] In a possible implementation, a first intersection feature point or a first intersection domain may be obtained based on the position information of the ground point cloud corresponding to the main lidar and the position information of the ground point cloud corresponding to the side lidar.

[0146] The first intersection feature point represents the intersection of the ground point cloud corresponding to the main laser radar and the ground point cloud corresponding to the side laser radar. The intersection point can be a ground point pair, which includes a ground point in the ground point cloud corresponding to the main laser radar and a ground point in the ground point cloud corresponding to the side laser radar; for example, Figure 10 A schematic diagram of an intersection feature point according to an embodiment of the present application is shown. Figure 10 The intersection feature point shown in is the intersection of the ground point cloud corresponding to the main laser radar and the ground point cloud corresponding to the side laser radar. The first intersection domain represents the area centered on the first intersection feature point. The first intersection domain may include multiple ground point pairs, where each ground point pair includes a ground point in the ground point cloud corresponding to the main laser radar and a ground point in the ground point cloud corresponding to the side laser radar; for example, Figure 11 A schematic diagram of a cross domain according to an embodiment of the present application is shown. Figure 11 As shown, the area in the ellipse is the first intersection domain, and each ellipse includes multiple ground point pairs.

[0147] Exemplarily, the position information of the ground point cloud corresponding to the main laser radar and the position information of the ground point cloud corresponding to the side laser radar are gridded in a 5m×5m grid on the xy plane in the vehicle coordinate system; within any grid, based on any ground point in the ground point cloud corresponding to the side laser radar, find the ground point in the ground point cloud corresponding to the main laser radar with the smallest Manhattan distance therebetween, where the Manhattan distance represents the sum of the absolute wheelbases of the two ground points in the coordinate system. For example, the Manhattan distance between the ground point a (x1, y1) in the ground point cloud corresponding to the side laser radar and the ground point b (x2, y2) in the ground point cloud corresponding to the main laser radar is: |x1-x2|+|y1-y2|. Then, the multiple pairs of ground points with the smallest Manhattan distance obtained are filtered according to the preset distance threshold. A pair of ground points with a Manhattan distance less than the preset distance threshold is the first intersection feature point, as mentioned above. Figure 10 As shown. Then, based on the line information of the ground points, a certain number of ground point pairs can be expanded with the first intersection feature point as the center to form the first intersection domain, as mentioned above. Figure 11 shown.

[0148] In one possible implementation, a second intersection feature point or a second intersection domain may be obtained based on the position information of the marker point cloud corresponding to the main lidar and the position information of the marker point cloud corresponding to the side lidar.

[0149] The second intersection feature point represents the intersection of the marker point cloud corresponding to the main lidar and the marker point cloud corresponding to the side lidar. This intersection point can be a feature point pair, which includes one feature point in the marker point cloud corresponding to the main lidar and one feature point in the marker point cloud corresponding to the side lidar. The second intersection domain represents the area centered on the second intersection feature point. The second intersection domain can include multiple feature point pairs, where each feature point pair includes one feature point in the marker point cloud corresponding to the main lidar and one feature point in the marker point cloud corresponding to the side lidar.

[0150] For example, based on the position information of the marker point cloud corresponding to the main lidar and the position information of the marker point cloud corresponding to the side lidar, the second intersection feature point or the second intersection domain will be obtained on the xz plane in the vehicle coordinate system with reference to the above-mentioned method of extracting the first intersection feature point or the first intersection domain, which will not be repeated here.

[0151] Step 803: Optimize the calibrated extrinsic parameters of any one of the multiple laser radars based on the intersection feature points or the intersection domain.

[0152] In a possible implementation, the yaw angle after the side laser radar is calibrated can be optimized according to the second intersection feature point or the second intersection domain.

[0153] Exemplarily, an objective function can be constructed: min|y2-y1| to optimize and compensate the yaw angle of the side lidar; wherein, when the number of second intersection feature points is not less than a preset threshold, the second intersection feature points can be used to solve the objective function. At this time, y2 and y1 respectively represent the y values ​​of the feature point pairs corresponding to each second intersection feature point. For example, y2 can represent the y value of the feature point in the marker point cloud corresponding to the main lidar in a feature point pair in the vehicle body coordinate system, and y1 can represent the y value of the feature point in the marker point cloud corresponding to the side lidar in the feature point pair in the vehicle body coordinate system; when the number of second intersection feature points is less than a preset threshold, the second intersection domain can be used to solve the objective function. At this time, y2 represents the average y value of all feature points in the marker point cloud corresponding to the main lidar in the second intersection domain in the vehicle body coordinate system, and y1 represents the y value of any feature point in the marker point cloud corresponding to the side lidar in the second intersection domain in the vehicle body coordinate system.

[0154] In a possible implementation, the pitch angle and roll angle after the side laser radar is calibrated can be optimized according to the first intersection feature point or the first intersection domain.

[0155] Exemplarily, an objective function can be constructed: min|z2-z1| to optimize and compensate the pitch angle and roll angle of the side laser radar, wherein, when the number of first intersection feature points is not less than a preset threshold, the first intersection feature points can be used to solve the objective function. At this time, z2 and z1 respectively represent the z values ​​of the ground point pairs corresponding to each first intersection feature point. For example, z2 can represent the z value of the ground point in the ground point cloud corresponding to the main laser radar in a ground point pair in the vehicle body coordinate system, and z1 can represent the z value of the ground point in the ground point cloud corresponding to the side laser radar in the ground point pair in the vehicle body coordinate system; when the number of first intersection feature points is less than a preset threshold, the first intersection domain can be used to solve the objective function. At this time, z2 represents the average z value of all ground points in the ground point cloud corresponding to the main laser radar in the vehicle body coordinate system in the first intersection domain, and z1 represents the z value of any ground point in the ground point cloud corresponding to the side laser radar in the vehicle body coordinate system in the first intersection domain.

[0156] For ease of understanding, the principle of optimizing the pitch and roll angles of the side lidar is explained: the compensation amount of the rotation matrix R can be expressed as:

[0157]

[0158] In formula (3), α represents the yaw angle, β represents the pitch angle, and γ represents the roll angle;

[0159]

[0160] Assuming that the translation matrix is ​​not compensated, the ground point pairs corresponding to the first intersection feature point have the following relationship:

[0161]

[0162] In formula (4), (x1, y1, z1) represents the coordinate value of the ground point in the ground point cloud corresponding to the side laser radar of a ground point pair in the vehicle body coordinate system, and (x2, y2, z2) represents the coordinate value of the ground point in the ground point cloud corresponding to the main laser radar of the ground point pair in the vehicle body coordinate system.

[0163] Based on the above optimized compensated yaw angle, the above formula (4) can be expanded to obtain:

[0164] z2=-sinβx1+cosβsinγy1+cosβcosγz1……………………(5)

[0165] In formula (5), z2 represents the z value of the ground point in the ground point cloud corresponding to the main laser radar in the ground point pair in the vehicle body coordinate system, and x1, y1, and z1 represent the x value, y value, and z value of the ground point in the ground point cloud corresponding to the side laser radar in the ground point pair in the vehicle body coordinate system, respectively.

[0166] Expand the trigonometric function Taylor of formula (5) to obtain:

[0167] z2-z1≈-βx1+γy1………………(6)

[0168] From formula (6), we can construct the objective function min|z2-z1| and use the ant colony algorithm to search for the compensation amount of the pitch angle and roll angle when |z2-z1| is minimized within a certain range. The compensation amount can then be used to optimize the compensation of the pitch angle and roll angle of the calibrated side lidar.

[0169] Furthermore, after performing the above processing on a frame of point cloud collected by the main lidar and the side lidar, the above processing can be performed on multiple frames of point cloud accumulated over a period of time, and the multiple sets of optimized calibration results obtained can be statistically analyzed to obtain the final optimized external parameters of the rear side lidar to the vehicle coordinate system.

[0170] Furthermore, the calibration parameters of the lidar can be updated according to the optimized and compensated side laser yaw angle; thereby, the intelligent driving function can be enabled or updated. Based on the high-precision external parameters after the optimized and compensated lidar, the accuracy of the perception, positioning or fusion functions of the intelligent driving is improved.

[0171] In this way, through the above steps 801-803, the point cloud collected by each laser radar is converted into the vehicle body coordinate system, and the position information of the marker point cloud and the ground point cloud corresponding to each laser radar is obtained, and then the external parameters of the laser radar are optimized by extracting cross feature points and cross domain features; in some examples, the main laser point cloud and the side laser point cloud can be converted to the vehicle body coordinate system through the corresponding calibrated external parameters, and the time synchronization of the main laser radar and the side laser radar is completed, and then on the basis of the marker point cloud, the cross feature points and cross domain optimization are extracted to compensate for the yaw angle of the side laser radar to the vehicle body coordinate system; on the basis of the ground points, the cross feature points and cross domain optimization are extracted to compensate for the pitch angle and roll angle of the side laser radar to the vehicle body coordinate system, thereby completing the joint optimization of the external parameters of multiple laser radars, so that the external parameters of the laser radar to the vehicle body coordinate system are more accurate. In addition, there are no special requirements for the vehicle driving site, and calibration can be completed on daily road sections. The road surface can also be uneven. Concave, uneven road surfaces or marker surfaces can all be used for calibration optimization; at the same time, the accuracy and efficiency of the calibration external parameters are effectively improved.

[0172] above Figure 6 or Figure 8 The lidar calibration method shown can be used in general scene calibration such as urban areas or elevated roads, service calibration, user self-calibration calibration and other scenarios; in some examples, during daily use of the vehicle, over time, due to the influence of uncertain factors such as object deformation, temperature, and small touches, the external parameters of the on-board lidar will change; at this time, the vehicle does not need to return to the factory and can maintain a straight line for a short distance on open urban roads or highways. Based on the above-mentioned embodiment of the present application, the fully automatic online calibration method of single lidar external parameter calibration based on open roads and / or multi-lidar joint optimization can complete the calibration and optimization of lidar external parameters, and the external parameters in the system can be updated, so that users can adjust and correct the external parameters of the lidar in real time online in daily life, ensure the safe use of intelligent driving functions, and improve intelligent driving performance.

[0173] Based on the above Figure 3 The application scenario described above provides a detailed description of the laser radar calibration method provided in the embodiment of the present application.

[0174] Figure 12 FIG1 shows a flow chart of another laser radar calibration method according to an embodiment of the present application; the method can be executed by the above-mentioned laser radar calibration device; Figure 12 As shown, the method may include the following steps:

[0175] Step 1201: Obtain the point cloud collected by the laser radar when the vehicle passes through the target area.

[0176] The laser radar may be any laser radar installed on the vehicle, for example, a main laser radar or a side laser radar.

[0177] At least one side of the target area is vertically provided with a marker. For example, at least one side of the target area is vertically provided with multiple markers, and the intersections of the multiple markers and the ground are on the same straight line. For example, the vehicle can be the above-mentioned self-vehicle 301, and the target area can be the above-mentioned Figure 4 (a)- Figure 4 any of the roads shown in (e).

[0178] For example, the acquired point cloud may be a point cloud collected by a laser radar when the vehicle is traveling in a straight line at a constant speed. Figure 4 (a)- Figure 4 (e) Point cloud collected by the laser radar 303 when driving in a straight line at a constant speed on any of the roads shown.

[0179] For example, the driving state of the vehicle can be determined based on the posture information of the vehicle when it passes through the target area; for example, the inertial navigation system can be used to determine that the vehicle is in a uniform straight-moving state through the target area, and the point cloud collected by the on-board lidar can be extracted.

[0180] For example, the target area can be Figure 4 For the road shown in (a), a row of parallel vertical markers can be set up on the left side (or right side) of the road, with equal distances between the markers. The markers can be cylindrical straight poles with reflective stickers. This target area is easy to construct and saves costs.

[0181] The target area can be Figure 4 On the road shown in (b), equidistant cylindrical rods are symmetrically distributed on both sides of the target area. The two rows of cylindrical rods are parallel to each other, and the cylindrical rods on the left and right sides are symmetrically distributed. There are markers on both sides of the target area, and the laser radars installed on both sides of the vehicle can scan the markers, so that the target area can be used to calibrate the slave laser radars installed on both sides of the vehicle. At the same time, for the forward main laser radar, more markers can be scanned, thereby improving the accuracy and stability of the external parameter calibration.

[0182] The target area can be Figure 4In the road shown in (c), equidistant cylindrical rods are staggered on both sides of the target area. The two rows of cylindrical rods are parallel, and the cylindrical rods on the left and right sides are staggered, with the staggered offset being half the spacing between adjacent cylindrical rods on the same side. Compared to a symmetrical distribution pattern, the number of markers scanned by the forward-facing main lidar in the target area varies less in the time domain, and the distance to the nearest marker is reduced by half. Therefore, before and after the nearest marker disappears from the scanning field of view, there is no excessive jump in the distance between the frames before and after the marker is scanned, further improving the accuracy, stability, and efficiency of the extrinsic calibration.

[0183] The target area can be Figure 4 In the road shown in (d), cylindrical rods are distributed at arbitrary intervals on both sides of the target area, with the two rows of cylindrical rods in a parallel relationship. Within this target area, at least one row of cylindrical rods can be spaced unequally. This facilitates construction within the target area, eliminating the need for high-precision measurement and precise construction, and improving flexibility and versatility. The spacing between adjacent markers can be estimated as needed using methods such as marker point cloud fitting.

[0184] Step 1202: extract the landmark point cloud from the collected point cloud.

[0185] For example, a marker point cloud can be extracted based on the material characteristics of the marker. For example, if the marker is a cylindrical vertical rod, the cylindrical vertical rod point cloud can be filtered out from the point cloud collected by the lidar based on the reflection intensity.

[0186] For example, the above Figure 6 The method for extracting the marker point cloud shown in is used to extract the marker point cloud from the point cloud collected in step 1201 .

[0187] For example, the collected point cloud may be subjected to motion dedistortion processing to extract an accurate marker point cloud.

[0188] Step 1203: Obtain the fitting line information of the marker according to the marker point cloud.

[0189] The fitting line information includes the position and direction information of the fitting line. For example, the fitting line of the marker may include the center line of the marker, the generatrix of the marker, the edge line of the marker and other straight lines perpendicular to the ground, wherein the center line of the marker refers to a straight line passing through the centers of the upper and lower sections of the marker; the embodiment of the present application takes the center line of the marker as an example to illustrate the method of obtaining the fitting line information of the marker; for example, the center line of the marker can be represented by the marker vector. Indicates, where a, b, and c represent the three components of the direction vector of the marker vector l, i.e., the direction information of the center line; The position vector representing the intersection of the marker vector l and the XY plane, that is, the position information of the center line.

[0190] In one possible implementation, this step may include: determining an initial value of the rotation angle based on the marker point cloud, wherein the initial value of the rotation angle minimizes the projection area of ​​the horizontal plane in the lidar coordinate system after the marker point cloud is rotated; rotating the marker point cloud according to the initial value of the rotation angle; and obtaining the fitting line information of the marker using the rotated marker point cloud.

[0191] For example, taking the fitting line of the marker as the center line of the marker, for a single set of marker point clouds, the initial value of the rotation angle R can be obtained by adjusting the laser radar rotation angle R so that the projection area of ​​the marker point cloud on the Z=0 plane of the laser radar coordinate system is the smallest. init ; Then according to R init , rotate the marker point cloud, and use the rotated marker point cloud to perform optimization processing to obtain the direction vector [a, b, c] of the marker vector l and the intersection position of l and the XY plane Thus, the marker vector l is obtained.

[0192] For example, Figure 13 FIG. 1 shows a schematic diagram of a single laser radar calibration according to an embodiment of the present application, as shown in FIG. Figure 13 As shown, the marker vector l before rotation (i.e., the marker center line) is rotated by R to obtain a vertical marker vector l′ (i.e., the direction vector is the unit vector [0, 0, 1]). The relationship between l and l′ is as follows:

[0193] [0, 0, 1] T =R[a, b, c] T .....................................(7)

[0194] In formula (7), [0, 0, 1] T Represents the transposed matrix of the unit vector [0, 0, 1], [a, b, c] T Represents the transposed matrix of the vector [a, b, c].

[0195] Figure 13 In the example, pi represents the coordinate of the laser point i on the marker before rotation. After the rotation R, the coordinate of the laser point on the rotated marker is P. i =Rp i .

[0196] It can be achieved by combining:<a,b,c],[0,0,1]> ,ω=[a,b,c]×[0,0,1], and Rodrigues rotation formula to calculate the rotation matrix Calculate the direction vector of the marker vector l, that is, the three components a, b, and c of l in the unit vector: where θ represents the rotation angle and ω represents the rotation axis information. Represents the normalized rotation axis information, and <[a, b, c], [0, 0, 1]> represents the angle between [a, b, c] and [0, 0, 1].

[0197] Furthermore, the position vector of the marker vector is estimated:

[0198] The marker vector l before rotation, after rotation R, in the laser radar coordinate system, the intersection of the marker vector l′ and the XY plane is Among them, x l ,y l is the coordinate value of the intersection point.

[0199] Construct the following optimization function:

[0200] argmin0.5∑(f(p i , l)-r) 2 ........................................(8)

[0201] In formula (8), r is the radius of the marker cross section; is the laser point p i The distance to the landmark vector l; where, is a point on the marker vector l; express and R[a, b, c] T The vector angle is: Right now The angle between a line segment and the vertical.

[0202] The amount to be optimized is The initial value of optimization is R init Based on the known radius of the marker cross section, the marker vector l can be obtained by optimizing and solving the above formula (8).

[0203] Step 1204: Obtain the values ​​of the laser radar external parameters based on the fitting line information.

[0204] Among them, the numerical value of at least one of the pitch angle, roll angle, and yaw angle of the laser radar can be obtained based on the fitting line information.

[0205] For example, the pitch angle of the laser radar can be obtained based on the direction information of the center line obtained above (such as a single marker vector). Since each directional component of a single marker vector represents the projection length of the marker vector on each directional axis, the pitch angle can be estimated using the components of the single marker vector. For example, The a and c components of the direction vector [a, b, c] are used to calculate the pitch angle using atan(a / c).

[0206] In this way, the value of the lidar extrinsic parameter can be obtained based on the point cloud of a landmark collected by the lidar.

[0207] Since the pitch angle of the lidar plays a vital role in whether there are enough laser points on the ground near the vehicle; at the same time, ground information can increase the accuracy and stability of calibration optimization to a certain extent, therefore, when the lidar is facing upward, it is also possible to determine whether the obtained pitch angle value of the lidar is greater than the second preset threshold, that is, the pitch angle value is used as a reference value for deciding whether to use ground information for calibration optimization; this can avoid the problem of unavailable ground information caused by lidar installation deviation (for example, pitch angle), and can also make full use of effective ground information.

[0208] In one possible implementation, when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle of the laser radar is greater than a second preset threshold, the external parameters of the laser radar are calibrated according to the marker point cloud.

[0209] The first preset threshold value may be 90°. If the angle between the LiDAR orientation and the vertically upward direction is less than 90°, it indicates that the LiDAR orientation is biased upward. The second preset threshold value TH may be determined by the vertical field of view (V-FOV) of the LiDAR. For example, TH = 0.5 * (V-FOV), meaning the second preset threshold value may be half the vertical field of view.

[0210] For example, if the LiDAR is oriented upward and the pitch angle of the LiDAR is greater than a second preset threshold, the pitch, yaw, and roll angles of the LiDAR can be jointly calibrated based on the centerline information of the marker. Since the centerline of the marker is vertically upward in the world coordinate system, the centerline information can be used to simultaneously calibrate the pitch, yaw, and roll angles of the LiDAR.

[0211] For example, the marker vector Approximate position vector Based on the objective function shown in formula (9), the pitch angle, yaw angle and roll angle of the lidar are jointly calibrated.

[0212] argmin 0.5α∑∑(f(P′ ij , l j (0, 0, 1, d j ))-r)+0.5β∑[z]......(9)

[0213] In formula (9), α and β are the residual term proportional coefficients, r is the radius of the marker cross section, z is the height of the laser radar, j is the number of the scanned marker, and P′ ij is the i-th laser point of the j-th marker after rotation, l j (0, 0, 1, d j ) represents the jth marker vector, d j is the intersection of the jth marker and the XY plane. ij =R roll R pitch R yaw P ij +[0, 0, z] T , R yaw represents the yaw angle, R pitch Indicates the pitch angle, R roll represents the roll angle; P ij is the i-th laser point of the j-th marker before rotation.

[0214] This step does not rely on the ground point cloud. When the ground point cloud is insufficient, the marker point cloud can be used to achieve high-precision calibration of the LiDAR's external parameters. High-precision external parameter calibration can be achieved for LiDARs with a small vertical field of view that cannot scan nearby ground point clouds; side LiDARs that cannot collect valid ground point clouds due to limited site width; LiDARs with excessively large installation pitch angles that make them unable to scan ground point clouds or scan a small number of ground point clouds; LiDARs installed at high altitudes that scan ground point clouds at a distance, etc.

[0215] For example, in a scene where the field of view of the lidar is small, Figure 14 A schematic diagram of a laser radar scanning scene according to an embodiment of the present application is shown in FIG. Figure 14 As shown in the figure, the vertical field of view of the main laser radar is small, and the main laser radar cannot scan the ground close to the vehicle, that is, it cannot obtain the nearby ground point cloud. If the solution of calibrating the main laser radar external parameters based on ground information in related technologies is adopted, the calibration accuracy is usually low. Therefore, you can refer to the following Figure 4 (a)- Figure 4(e) In scenarios such as setting up markers on both sides of the road, by executing the above steps 1201-1204, it is possible to complete the high-precision calibration of the main lidar's extrinsic parameters using only the marker point cloud collected by the main lidar when the main lidar's field of view is small.

[0216] For example, in a production line environment, the width of the assembly line is limited. Figure 15 (a)- Figure 15 (b) shows a schematic diagram of a production line environment according to an embodiment of the present application, such as Figure 15 (a)- Figure 15 As shown in (b), due to the limited width of the site, the vehicle's side LiDAR cannot scan the ground, that is, it cannot obtain the ground point cloud; or the scanned ground area is small, that is, the number of ground point clouds obtained is small. If the solution of the related technology that relies on ground information to calibrate the side LiDAR external parameters is adopted, it usually cannot operate normally. Therefore, you can refer to Figure 4 (a)- Figure 4 (e) In scenarios such as setting up markers on the production line, by executing steps 1201 to 1204 above, high-precision calibration of the external parameters of the side lidar can be completed in a narrow production line using only the marker point cloud collected by the side lidar.

[0217] For example, for the above Figure 4 In the scenario (e) where the road surface is uneven and there are markers on the road, by executing steps 1201 to 1204 above, high-precision calibration of the side lidar extrinsic parameters is completed in the road with ups and downs in the local area using only the point cloud of the markers collected by the lidar. Figure 16 FIG. 4 shows a comparative diagram of calibrating the pitch angle according to an embodiment of the present application. Figure 16 As shown, on uneven roads, the solution of calibrating the main lidar external parameters based on ground information in related technologies has a large deviation in the calibrated pitch angle. Setting markers (such as Figure 4 (e) shows), adopting the method in the embodiment of the present application and calibrating the pitch angle only by using the marker point cloud can reduce or avoid the influence of uneven ground information, thereby improving the accuracy of the calibrated pitch angle.

[0218] In one possible implementation, a ground point cloud can be extracted from the collected point cloud, and the ground point cloud and the marker point cloud can be used to calibrate the lidar extrinsic parameters, thereby fully utilizing the ground and marker information and improving the accuracy of the extrinsic parameter calibration. For example, when the angle between the lidar's orientation and the vertical upward direction is less than a first preset threshold, and the pitch angle is greater than a second preset threshold, the lidar extrinsic parameters are calibrated based on the marker point cloud and the ground point cloud. In this way, when a valid ground point cloud exists, the ground point cloud can be fully utilized, further improving the calibration accuracy and stability.

[0219] For example, the marker vector Position vector The pitch angle, yaw angle and roll angle of the lidar are jointly calibrated based on the objective function shown in formula (10) after intersecting the ground point cloud.

[0220] argmin 0.5α∑∑(f(P′ ij , l j (0, 0, 1, d j ))-r) 2 +0.5β∑P′ G [z].............(10)

[0221] In formula (10), α and β are the residual term proportional coefficients, r is the radius of the marker cross section, z is the height of the laser radar, j is the number of the scanned marker, and P′ ij is the i-th laser point of the j-th marker after rotation, l j (0, 0, 1, d j ) represents the jth marker vector, d j is the intersection of the jth marker and the XY plane; P′ G is the rotated ground point cloud.

[0222] When a frame of point cloud contains multiple laser points corresponding to markers and the distance between the markers is unknown: the pitch angle, yaw angle, roll angle and height of the laser radar are optimized based on formula (10). At this time, P′ in formula (10) ij =R roll R pitch R yaw P ij +[0, 0, z] T , P′ G =R roll R pitch R yaw P G +[0, 0, z] T , where z is the height of the lidar, R yawrepresents the yaw angle, R pitch Indicates the pitch angle, R roll represents the roll angle; P ij is the i-th laser point of the j-th marker before rotation, P G Represents the ground point cloud before rotation.

[0223] When a frame of point cloud contains laser points corresponding to multiple markers and the distance between the markers is known: Based on formula (10), optimize the pitch angle, yaw angle, roll angle of the lidar, the height of the lidar and the position of the first marker in the frame of point cloud d0 = [x0, y0] T , where x0, y0 are the coordinate values ​​of d0, let W j Represents the distance from the jth marker to the j+1th marker, then the position of the second marker in the point cloud of this frame is d1 = [x1 + W0, y0] T .

[0224] Furthermore, when the angle between the laser radar direction and the vertical upward direction is less than a first preset threshold and the value of the laser radar's pitch angle is not greater than a second preset threshold, the external parameters of the laser radar are calibrated based on the marker point cloud and the ground point cloud.

[0225] When the angle between the laser radar direction and the vertical upward direction is less than the first preset threshold and the value of the laser radar's pitch angle is not greater than the second preset threshold, the laser radar is directed upward and can scan the ground at the same time, that is, there is a ground point cloud. Therefore, the ground point cloud and the marker point cloud can be used to calibrate the external parameters of the laser radar, thereby making full use of the ground information and improving the accuracy of the laser radar's external parameter calibration.

[0226] In a possible implementation, when the angle between the laser radar and the vertical upward direction is less than a first preset threshold, and the pitch angle of the laser radar is not greater than a second preset threshold, the pitch angle and roll angle of the laser radar are calibrated based on the ground point cloud, and the height of the laser radar can also be calibrated; the yaw angle of the laser radar is calibrated based on the position information of the fitting line. For example, the marker vector Position vector Calibrate the yaw angle of the lidar.

[0227] In this way, when the angle between the laser radar direction and the vertical upward direction is less than the first preset threshold and the value of the laser radar pitch angle is not greater than the preset threshold, the laser radar can scan the ground and there is a ground point cloud. Therefore, the ground point cloud can be used to calibrate the pitch angle and roll angle of the laser radar, thereby improving the calibration accuracy and stability.

[0228] Considering that it's difficult for a vehicle to travel in a straight line, the embodiments of this application do not limit the vehicle's yaw angle. After completing the calibration, the calibration results can be optimized by combining vehicle motion information. In one possible implementation, the vehicle's position information and the LiDAR's position information can be obtained; the vehicle's heading angle can be determined based on the vehicle's position information and the LiDAR's position information; and the yaw angle of the calibrated LiDAR can be optimized based on the heading angle.

[0229] For example, based on markers, the lidar can locate the position information of the lidar; the vehicle can estimate the position information of the vehicle through information such as its own chassis; the vehicle's position information and the lidar's position information are used to filter and estimate the vehicle's heading angle; the parallel constraint of the vehicle's heading angle is used to compensate for the dynamic changes of the yaw angle.

[0230] Furthermore, after performing the above processing on a frame of point cloud extracted in step 1201, the above processing can be performed on multiple frames of point cloud accumulated over a period of time, and the obtained calibration results can be statistically analyzed to optimize the external parameters of the final lidar.

[0231] In this way, through the above steps 1201-1204, based on the markers vertically set in the target area (for example, a row of parallel vertical markers), the value of the laser radar external parameter is obtained by using the principle that the fitting line of the vertical marker needs to conform to the vertical constraint; in some examples, based on the calculated pitch angle value, when the angle between the laser radar direction and the vertical upward direction is less than the first preset threshold, it can be judged whether the pitch angle value is greater than the second preset threshold, so as to automatically judge the availability of the ground point cloud and improve the calibration efficiency and automation; and only using the marker point cloud, the pitch angle, yaw angle and roll angle of a single laser radar can be calibrated at the same time. In some examples, the dynamic changes of the yaw angle can be compensated in combination with the vehicle motion information, thereby achieving high-precision dynamic calibration of a single laser radar.

[0232] In the embodiment of the present application, a marker is vertically set on at least one side of the target area. The marker is simple to set, which reduces the requirements for the site and has low construction cost. According to the marker point cloud, the fitting line information of the marker is obtained. The fitting line of the vertical marker needs to meet the vertical constraint to obtain the value of the laser radar extrinsic parameter. In this way, the value of the laser radar extrinsic parameter can be calculated according to the marker point cloud. In some examples, when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the laser radar pitch angle is greater than a second preset threshold, the laser radar extrinsic parameter can be calibrated with high precision according to the marker point cloud. At the same time, since the calibration process does not rely on the ground point cloud, it can be applied to scenes with insufficient ground information (for example, a laser radar with a small vertical field of view angle cannot collect nearby ground point clouds, a laser radar cannot collect valid ground point clouds due to limited site size, a laser radar with an excessively large pitch angle is lifted upward, resulting in missing or less ground point clouds, etc.), thereby achieving high-precision calibration of a single laser radar in scenes with insufficient ground information. In addition, compared with methods such as map building calibration, the entire calibration process can be performed automatically, which improves the efficiency of single lidar calibration.

[0233] Furthermore, when a vehicle is equipped with multiple lidars, the calibration results of any one lidar can be further optimized based on the calibration results of any two lidars to the vehicle coordinate system, thereby achieving multi-laser calibration. For example, the extrinsic parameters of the slave lidar can be optimized based on the extrinsic parameters of the master lidar calibrated through steps 1201-1204 and the extrinsic parameters of the slave lidar calibrated through steps 1201-1204.

[0234] For example, the master and slave lidars may have no common view area, or a small common view area. In a frame of point cloud, the master lidar can scan at least two landmarks; by predicting and matching the positions of the landmarks, the extrinsic parameters of the slave lidar are optimized.

[0235] Figure 17 FIG. 1 shows a flow chart of a laser radar calibration method according to an embodiment of the present application; FIG. Figure 17 As shown, the method may include the following steps:

[0236] Step 1701: Determine the position information of multiple markers based on the calibrated external parameters of the main lidar and the multiple marker point clouds collected by the main lidar.

[0237] The external parameters of the master laser radar may be the external parameters calibrated through the above steps 1201 to 1204. The position information of the multiple markers may include the position information of the fitting lines of the multiple markers converted based on the above calibrated external parameters.

[0238] For example, the position vectors of multiple markers can be obtained by fitting the multiple marker point clouds collected by the main laser radar. Then, using the calibrated external parameters of the main lidar, the position vector Converted to the vertical direction, the position vector obtained at this time is the position information of multiple markers.

[0239] Over a period of time, the main lidar continuously tracks the markers, determines and records the unique number j of the marker by scanning in sequence, and at the same time, can measure the distance Wj between any two markers; let Wj represent the distance from the jth marker to the j+1th marker.

[0240] For example, Figure 18 (a)-18(b) show a schematic diagram of a multi-lidar joint optimization according to an embodiment of the present application; Figure 18 As shown in (a), the markers can be equally spaced, and the main laser radar numbers the markers tracked continuously, i.e., W0, W1…Wj; Figure 18 As shown in (b), the laser radar can currently track the markers W0 and W1, and can also obtain the distance between W0 and W1 based on the determined position information of the markers W0 and W1.

[0241] Step 1702: Obtain a predicted position of a first marker based on the position information of multiple markers.

[0242] Among them, the first marker is a marker that can be tracked by the laser radar and is behind the above-mentioned multiple markers.

[0243] For example, the predicted position of the first marker can be calculated using the first marker number and the distance Wj between the markers. The predicted position of the first marker can include the position information of the fitting line of the first marker (eg, the position information of the center line of the first marker).

[0244] In one possible implementation, the direction can be predicted first, and then the distance can be predicted to obtain the predicted position of the first marker. For example, since multiple vertical markers on the same side are parallel, a straight line can be determined based on the position information of the two markers, and the remaining markers can be known to be on this straight line. Then, combined with the order of the two markers, the arrangement direction of the markers on the straight line can be obtained. Then, based on the arrangement direction of the markers on the straight line, the distance Wj estimated by the main laser radar tracking markers, such as W2, W3, is used to estimate the predicted position of the first marker. For example, Figure 18 As shown in (a), W2 can be tracked from the laser radar, that is, the first marker can be W2, as shown in Figure 18As shown in (b), the position information of W0 and W1 can be used to determine the direction of the marker arrangement on the straight line where W0 and W1 are located, and then based on this direction, the distance between W0 and W1 and the position information of W1 can be used to infer the predicted position of W2.

[0245] Step 1703: Obtain the measured position of the first marker based on the calibrated external parameters of the laser radar and the first marker point cloud collected from the laser radar.

[0246] The external parameters from the laser radar may be the external parameters calibrated through the above steps 1201 to 1204. For example, the position information of the first marker may include the position information of the center line of the first marker converted based on the above calibrated external parameters.

[0247] For example, the position vector of the first marker can be obtained by using the first marker point cloud collected from the laser radar in the manner described above. Then, using the external parameters after calibration from the laser radar, the position vector Converted to the vertical direction, the position vector obtained at this time is the measured position of the first marker.

[0248] Step 1704: Optimize the external parameters of the laser radar by predicting the position and measuring the position.

[0249] Using the predicted position of the first marker and the actual measured position of the slave lidar, the extrinsic parameters of the side lidar are optimized to optimize the position vector coincidence of the first marker. For example, the measured position can be transformed using the extrinsic parameters of the slave lidar relative to the master lidar, and then the transformed measured position is matched with the predicted position to complete the joint optimization of the extrinsic parameters. It is understandable that if there is a deviation in the extrinsic parameters of the slave lidar relative to the master lidar, the measured position obtained from the slave lidar will be inconsistent with the predicted position after being converted to the relative extrinsic parameters with the deviation.

[0250] For example, the least squares form can be used based on the optimization objective function: 0.5[(Δa) 2 +(Δb) 2 +(Δc) 2 ]Optimize the solution, where Δa, Δb, and Δc are the differences between the three components of the direction vector of the master lidar and the slave lidar.

[0251] In this way, through the above steps 1701 to 1704, based on the results of single laser calibration, the distance and / or orientation between the two solved markers are used to predict the positions of the remaining markers, and the joint optimization of the pitch angle, yaw angle, and roll angle between multiple laser radars in any orientation is achieved. For scenarios where multiple laser radars have large deviations in installation position and angle, and point clouds are projected to different spatial positions, which are not suitable for direct point cloud registration, this method can effectively improve the accuracy of calibration, thereby achieving joint calibration of multiple laser radars with no common view area or a small common view area. In addition, this method does not require advance mapping, which significantly improves the efficiency of multi-lidar calibration.

[0252] above Figure 12 or Figure 17 The LiDAR calibration method shown can be used in scenarios such as end-of-line calibration, service calibration, online self-calibration with high-definition maps, etc. In some examples, end-of-line calibration needs to meet the requirements of multi-model and multi-sensor adaptation; fast calibration; low cost, simple site, and can be promoted and built in different factories; a large angle calibration tolerance range is required, and when there is a large deviation in the installation, an abnormal alarm is required; it can work all day and night and can work normally when the navigation system is unavailable, etc. At this time, the vehicle is provided by the embodiment of the present application. Figure 4 (a)- Figure 4 (e) The vehicle enters from one end of any road and exits from the other end. After driving a short distance, the vehicle-mounted lidar calibration can be completed based on the single lidar extrinsic parameter calibration and / or multi-lidar joint optimization method of the above-mentioned embodiment of the present application to meet the calibration needs of the end of the production line; and the extrinsic parameters in the system can be updated to ensure the safe use of the intelligent driving function and improve the intelligent driving performance.

[0253] Based on the same inventive concept of the above method embodiment, an embodiment of the present application also provides a laser radar calibration device, which can be used to execute the technical solution described in the above method embodiment.

[0254] Figure 19 FIG. 1 shows a structural diagram of a laser radar calibration device according to an embodiment of the present application. Figure 19As shown, the device includes: an acquisition module 1901, which is used to acquire the point cloud collected by the laser radar when the vehicle passes through the target road, and a marker is set on at least one side of the target road; a screening module 1902, which is used to preliminarily screen the collected point cloud according to a preset threshold; the preset threshold is determined by the installation height of the laser radar; a first extraction module 1903, which is used to perform multiple fitting processing on the point cloud preliminarily screened to obtain a ground point cloud; a second extraction module 1904, which is used to extract the marker point cloud in the collected point cloud; a calibration module 1905, which is used to calibrate the external parameters of the laser radar according to the marker point cloud and the ground point cloud.

[0255] In one possible implementation, the device also includes: a conversion module for obtaining the position information of the marker point cloud and the ground point cloud corresponding to each laser radar based on the calibrated external parameters of multiple laser radars; a third extraction module for obtaining the intersection feature point or intersection domain based on the position information of the marker point cloud and the ground point cloud corresponding to each laser radar, wherein the intersection domain represents an area parallel to the direction of travel of the vehicle and centered on the intersection feature point; an optimization module for optimizing the calibrated external parameters of any laser radar among the multiple laser radars based on the intersection feature point or intersection domain.

[0256] In one possible implementation, the multiple laser radars include a master laser radar and a slave laser radar, wherein the master laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the side and / or rear environment of the vehicle; the conversion module is further used to: according to the calibrated external parameters of the master laser radar, convert the marker point cloud and the ground point cloud corresponding to the master laser radar into the vehicle body coordinate system to obtain the position information of the marker point cloud and the ground point cloud corresponding to the master laser radar; according to the calibrated external parameters of the slave laser radar, convert the marker point cloud and the ground point cloud corresponding to the slave laser radar into the vehicle body coordinate system to obtain the position information of the marker point cloud corresponding to the slave laser radar Position information and position information of the ground point cloud; the third extraction module is also used to: obtain a first intersection feature point or a first intersection domain according to the position information of the ground point cloud corresponding to the main laser radar and the position information of the ground point cloud corresponding to the slave laser radar; obtain a second intersection feature point or a second intersection domain according to the position information of the marker point cloud corresponding to the main laser radar and the position information of the marker point cloud corresponding to the slave laser radar; the optimization module is also used to: optimize the pitch angle and roll angle after calibration of the slave laser radar according to the first intersection feature point or the first intersection domain; optimize the yaw angle after calibration of the slave laser radar according to the second intersection feature point or the second intersection domain.

[0257] In one possible implementation, the external parameters include at least one of the pitch angle, roll angle, and yaw angle; the calibration module is further used to: calibrate the pitch angle and roll angle of the laser radar based on the ground point cloud; and calibrate the yaw angle of the laser radar based on the marker point cloud.

[0258] In one possible implementation, the second extraction module is further used to: filter out the ground point cloud in the collected point cloud; divide the filtered point cloud into multiple slices along a direction perpendicular to the vehicle's travel; extract the marker point cloud, the marker point cloud including feature points in a slice set that meets preset conditions, wherein the slice set includes one or more adjacent target slices, and the number of feature points in the target slice exceeds a threshold.

[0259] In one possible implementation, the device also includes a downsampling module, which is used to: obtain first beam information of the ground point cloud, and downsample the ground point cloud based on the first beam information; and / or obtain second beam information of the marker point cloud, and downsample the marker point cloud based on the second beam information; the calibration module is also used to: calibrate the external parameters of the lidar based on the marker point cloud and the ground point cloud after downsampling.

[0260] In one possible implementation, the acquired point cloud is a point cloud collected by a laser radar when the vehicle is traveling in a straight line.

[0261] In one possible implementation, the marker includes at least one of a curb, a guardrail, and a building.

[0262] In the above embodiments, the technical effects and specific descriptions of the laser radar calibration device and its various possible implementation methods can be found in the above laser radar calibration method, which will not be repeated here.

[0263] Figure 20 FIG. 1 shows a structural diagram of a laser radar calibration device according to an embodiment of the present application. Figure 20 As shown, the device includes: an acquisition module 2001, used to acquire a point cloud collected by a laser radar when a vehicle passes through a target area; a marker is vertically set on at least one side of the target area; an extraction module 2002, used to extract the marker point cloud from the collected point cloud; a fitting module 2003, used to obtain fitting line information of the marker based on the marker point cloud; the fitting line information includes position and direction information of the fitting line; a calculation module 2004, used to obtain the value of the laser radar extrinsic parameter based on the fitting line information;

[0264] In one possible implementation, the external parameters include a pitch angle; the device also includes: a calibration module, which is used to calibrate the external parameters of the laser radar according to the marker point cloud when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is greater than a second preset threshold, wherein the second preset threshold is determined by the vertical field of view angle of the laser radar.

[0265] In one possible implementation, the external parameter includes a yaw angle; the device also includes: an optimization module for obtaining the position information of the vehicle and the position information of the laser radar; determining the heading angle of the vehicle based on the position information of the vehicle and the position information of the laser radar; and optimizing the yaw angle of the calibrated laser radar based on the heading angle.

[0266] In one possible implementation, the vehicle is equipped with a main laser radar and a slave laser radar, wherein the main laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the side and / or rear environment of the vehicle; the device also includes: a determination module, used to determine the position information of the multiple markers based on the calibrated external parameters of the main laser radar and the multiple marker point clouds collected by the main laser radar; a prediction module, used to obtain the predicted position of the first marker based on the position information of the multiple markers; a measurement module, used to obtain the measured position of the first marker based on the calibrated external parameters of the slave laser radar and the first marker point cloud collected by the slave laser radar; a matching module, used to optimize the external parameters of the slave laser radar by comparing the predicted position with the measured position.

[0267] In a possible implementation, the extraction module is further used to extract the ground point cloud from the collected point cloud; the calibration module is further used to calibrate the external parameters of the lidar based on the marker point cloud and the ground point cloud.

[0268] In one possible implementation, the fitting module is also used to: determine an initial value of the rotation angle based on the marker point cloud, wherein the initial value of the rotation angle minimizes the projection area of ​​the horizontal plane in the lidar coordinate system after the marker point cloud is rotated; rotate the marker point cloud according to the initial value of the rotation angle; and obtain the fitting line information of the marker using the rotated marker point cloud.

[0269] In one possible implementation, the calibration module is also used to calibrate the external parameters of the laser radar based on the marker point cloud and the ground point cloud when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than the second preset threshold.

[0270] In one possible implementation, the external parameters include at least one of a pitch angle, a roll angle, and a yaw angle; the calibration module is further used to: calibrate the pitch angle and roll angle of the laser radar according to the ground point cloud when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than the second preset threshold; and calibrate the yaw angle of the laser radar according to the position information of the fitting line.

[0271] In a possible implementation, a plurality of markers are vertically arranged on at least one side of the target area, and intersection points of the plurality of markers and the ground are on the same straight line.

[0272] In the above embodiments, the technical effects and specific descriptions of the laser radar calibration device and its various possible implementation methods can be found in the above laser radar calibration method, which will not be repeated here.

[0273] An embodiment of the present application provides a laser radar calibration device, comprising: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-mentioned laser radar calibration method when executing the instructions.

[0274] Figure 21 A schematic structural diagram of a laser radar calibration device according to an embodiment of the present application is shown in FIG. Figure 21 As shown, the laser radar calibration device may include: at least one processor 2101, a communication line 2102, a memory 2103 and at least one communication interface 2104.

[0275] The processor 2101 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0276] The communication link 2102 may include a path to transmit information between the above components.

[0277] The communication interface 2104 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

[0278] The memory 2103 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory can be independent and connected to the processor via a communication line 2102. The memory can also be integrated with the processor. The memory provided in the embodiment of the present application can generally have non-volatility. Among them, the memory 2103 is used to store the computer execution instructions for executing the solution of the present application, and is controlled by the processor 2101 to execute. The processor 2101 is used to execute the computer-executable instructions stored in the memory 2103, thereby implementing the method provided in the above embodiments of the present application.

[0279] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0280] Exemplarily, the processor 2101 may include one or more CPUs, such as Figure 21 CPU0 and CPU1 in.

[0281] Exemplarily, the laser radar calibration device may include multiple processors, such as Figure 21 2101 and processor 2107 in FIG. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0282] In a specific implementation, as an embodiment, the laser radar calibration device may further include an output device 2105 and an input device 2106. The output device 2105 communicates with the processor 2101 and can display information in a variety of ways. For example, the output device 2105 can be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 2106 communicates with the processor 2101 and can receive user input in a variety of ways. For example, the input device 2106 can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0283] An embodiment of the present application also provides a laser radar calibration system, which includes at least one laser radar calibration device mentioned in the above embodiment of the present application.

[0284] An embodiment of the present application also provides a vehicle, which includes at least one laser radar calibration device or laser radar calibration system mentioned in the above embodiments of the present application.

[0285] An embodiment of the present application provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor.

[0286] An embodiment of the present application provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0287] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof.

[0288] The computer-readable program instructions or codes described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0289] The computer program instructions for performing the operations of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, by utilizing the state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present application.

[0290] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0291] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0292] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0293] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, systems, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and the part for the module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be performed substantially in parallel, and they can sometimes also be performed in the opposite order, depending on the function involved.

[0294] It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by hardware that performs the corresponding function or action (such as a circuit or ASIC (Application Specific Integrated Circuit)), or can be implemented by a combination of hardware and software, such as firmware.

[0295] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0296] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A laser radar calibration method, characterized in that: The method comprises: Obtaining a point cloud collected by a laser radar when the vehicle passes through a target road, wherein at least one side of the target road is provided with a marker; Performing a preliminary screening of the collected point cloud according to a preset threshold value; the preset threshold value is determined by the installation height of the laser radar; Performing multiple fitting processes on the initially screened point cloud to obtain a ground point cloud; Filtering out the ground point cloud from the collected point cloud; Divide the filtered point cloud into multiple slices along the direction perpendicular to the vehicle's travel direction; Extracting the marker point cloud, the marker point cloud including feature points in a slice set that meets a preset condition, wherein the slice set includes one or more adjacent target slices, the number of feature points in the target slice exceeds a threshold, and the preset condition includes positions of all feature points in the slice set exceeding a length range along a direction of vehicle travel; The external parameters of the laser radar are calibrated according to the marker point cloud and the ground point cloud.

2. The method according to claim 1, characterized in that The method further comprises: According to the calibrated external parameters of multiple laser radars, the position information of the marker point cloud and the position information of the ground point cloud corresponding to each laser radar are obtained; Obtaining an intersection feature point or an intersection domain based on the position information of the landmark point cloud and the position information of the ground point cloud corresponding to each laser radar, wherein the intersection domain represents an area parallel to the direction of travel of the vehicle and centered on the intersection feature point; According to the intersection feature points or the intersection domain, the calibrated external parameters of any one of the multiple laser radars are optimized.

3. The method according to claim 2, characterized in that The multiple laser radars include a master laser radar and a slave laser radar, wherein the master laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the side and / or rear environment of the vehicle; The method of obtaining the position information of the marker point cloud and the ground point cloud corresponding to each laser radar based on the calibrated external parameters of the plurality of laser radars includes: According to the calibrated external parameters of the main laser radar, the marker point cloud and the ground point cloud corresponding to the main laser radar are converted into the vehicle coordinate system to obtain the position information of the marker point cloud and the ground point cloud corresponding to the main laser radar; According to the calibrated external parameters of the slave laser radar, the marker point cloud and the ground point cloud corresponding to the slave laser radar are converted into the vehicle coordinate system to obtain the position information of the marker point cloud and the ground point cloud corresponding to the slave laser radar; Obtaining intersection feature points or intersection domains based on the position information of the marker point clouds and the position information of the ground point clouds corresponding to the laser radars includes: Obtain a first intersection feature point or a first intersection domain based on the position information of the ground point cloud corresponding to the master laser radar and the position information of the ground point cloud corresponding to the slave laser radar; obtain a second intersection feature point or a second intersection domain based on the position information of the marker point cloud corresponding to the master laser radar and the position information of the marker point cloud corresponding to the slave laser radar; The optimizing the calibrated extrinsic parameters of any one of the plurality of laser radars according to the intersection feature points or the intersection domain includes: According to the first intersection feature point or the first intersection domain, the pitch angle and the roll angle calibrated from the laser radar are optimized; according to the second intersection feature point or the second intersection domain, the yaw angle calibrated from the laser radar is optimized.

4. The method according to any one of claims 1 to 3, characterized in that The external parameter includes at least one of the pitch angle, roll angle, and yaw angle, The calibrating the external parameters of the laser radar according to the marker point cloud and the ground point cloud includes: Calibrate the pitch angle and roll angle of the laser radar according to the ground point cloud; The yaw angle of the laser radar is calibrated according to the marker point cloud.

5. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Get the first line information of the ground point cloud, performing downsampling processing on the ground point cloud according to the first line harness information; and / or, Get the second harness information of the marker point cloud, performing downsampling processing on the marker point cloud according to the second line harness information; Calibrate the external parameters of the laser radar according to the marker point cloud and the ground point cloud, including: The external parameters of the laser radar are calibrated according to the marker point cloud and the ground point cloud after downsampling processing.

6. The method according to any one of claims 1 to 3, characterized in that The acquired point cloud is the point cloud collected by the lidar when the vehicle is traveling in a straight line.

7. The method according to any one of claims 1 to 3, characterized in that The marker includes at least one of a curb, a guardrail, and a building.

8. A laser radar calibration method, characterized in that: The method comprises: Obtaining a point cloud collected by a laser radar when a vehicle passes through a target area; at least one side of the target area is vertically provided with a marker; Extracting a landmark point cloud from the collected point cloud; Determining an initial value of the rotation angle based on the marker point cloud, wherein the initial value of the rotation angle minimizes the projection area of ​​the horizontal plane in the laser radar coordinate system after the marker point cloud is rotated; Rotating the marker point cloud according to the initial value of the rotation angle; Using the rotated marker point cloud, the fitting line information of the marker is obtained; the fitting line information includes the position and direction information of the fitting line; According to the fitting line information, the value of the laser radar external parameter is obtained, and the laser radar external parameter includes at least one of the pitch angle, roll angle, and yaw angle of the laser radar.

9. The method according to claim 8, characterized in that The external parameters include pitch angle; The method further comprises: When the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is greater than a second preset threshold, the external parameters of the laser radar are calibrated according to the marker point cloud, wherein the second preset threshold is determined by the vertical field of view angle of the laser radar.

10. The method according to claim 9, characterized in that The external parameters include yaw angle; The method further comprises: Obtaining the position information of the vehicle and the position information of the laser radar; Determining the heading angle of the vehicle based on the position information of the vehicle and the position information of the laser radar; According to the heading angle, the yaw angle of the calibrated laser radar is optimized.

11. The method according to claim 9 or 10, characterized in that The vehicle is equipped with a master laser radar and a slave laser radar, wherein the master laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the environment on the side and / or rear of the vehicle; The method further comprises: Determining position information of the multiple markers based on the calibrated external parameters of the main laser radar and the multiple marker point clouds collected by the main laser radar; Obtaining a predicted position of a first marker based on the position information of the plurality of markers; Obtaining a measured position of the first marker based on the calibrated external parameters of the laser radar and the first marker point cloud collected from the laser radar; The external parameters of the slave laser radar are optimized through the predicted position and the measured position.

12. The method according to claim 9 or 10, characterized in that The method further comprises: extracting a ground point cloud from the collected point cloud; The calibrating the external parameters of the laser radar according to the marker point cloud also includes: The external parameters of the laser radar are calibrated according to the marker point cloud and the ground point cloud.

13. The method according to claim 9 or 10, characterized in that The method further comprises: When the angle between the laser radar and the vertical upward direction is less than the first preset threshold and the value of the pitch angle is not greater than the second preset threshold, the external parameters of the laser radar are calibrated according to the marker point cloud and the ground point cloud.

14. The method according to any one of claims 8 to 10, characterized in that The external parameter includes at least one of a pitch angle, a roll angle, and a yaw angle, and the method further includes: When the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is not greater than a second preset threshold, the pitch angle and roll angle of the laser radar are calibrated according to the ground point cloud; and the yaw angle of the laser radar is calibrated according to the position information of the fitting line.

15. The method according to any one of claims 8 to 10, characterized in that A plurality of markers are vertically arranged on at least one side of the target area, and intersection points of the plurality of markers and the ground are on the same straight line.

16. A laser radar calibration device, characterized in that: The device comprises: an acquisition module, configured to acquire a point cloud collected by a laser radar when a vehicle passes through a target road, wherein at least one side of the target road is provided with a marker; A screening module, configured to perform preliminary screening of the collected point cloud according to a preset threshold value; the preset threshold value is determined by the installation height of the laser radar; A first extraction module is used to perform multiple fitting processes on the initially screened point cloud to obtain a ground point cloud; a second extraction module configured to filter out the ground point cloud from the collected point cloud; divide the filtered point cloud into a plurality of slices along a direction perpendicular to the vehicle's travel; and extract the marker point cloud, wherein the marker point cloud includes feature points in a set of slices that meet a preset condition, wherein the set of slices includes one or more adjacent target slices, the number of feature points in the target slices exceeds a threshold, and the preset condition includes that the positions of all feature points in the set of slices exceed a length range along the vehicle's travel direction; The calibration module is used to calibrate the external parameters of the laser radar according to the marker point cloud and the ground point cloud.

17. The device according to claim 16, characterized in that The device further comprises: A conversion module is used to obtain the position information of the marker point cloud and the position information of the ground point cloud corresponding to each laser radar based on the calibrated external parameters of multiple laser radars; a third extraction module, configured to obtain an intersection feature point or an intersection domain based on the position information of the landmark point cloud and the position information of the ground point cloud corresponding to each of the laser radars, wherein the intersection domain represents an area parallel to the direction of travel of the vehicle and centered on the intersection feature point; An optimization module is used to optimize the calibrated external parameters of any laser radar among the multiple laser radars according to the intersection feature points or the intersection domain.

18. A laser radar calibration device, characterized in that: The device comprises: An acquisition module, configured to acquire a point cloud collected by a laser radar when a vehicle passes through a target area; at least one side of the target area is vertically provided with a marker; An extraction module, used to extract the landmark point cloud from the collected point cloud; A fitting module is configured to determine an initial rotation angle value based on the marker point cloud, wherein the initial rotation angle value minimizes the projection area of ​​the marker point cloud on the horizontal plane in the laser radar coordinate system after the marker point cloud is rotated; rotate the marker point cloud based on the initial rotation angle value; and obtain fitting line information of the marker using the rotated marker point cloud; the fitting line information includes position and direction information of the fitting line; A calculation module is used to obtain the value of the laser radar external parameter based on the fitting line information, and the laser radar external parameter includes at least one of the pitch angle, roll angle, and yaw angle of the laser radar.

19. The device according to claim 18, characterized in that The external parameters include pitch angle; The device also includes: a calibration module, which is used to calibrate the external parameters of the laser radar according to the marker point cloud when the angle between the laser radar and the vertical upward direction is less than a first preset threshold and the value of the pitch angle is greater than a second preset threshold, wherein the second preset threshold is determined by the vertical field of view angle of the laser radar.

20. The device according to claim 19, characterized in that The external parameters include a yaw angle; the device also includes: an optimization module for obtaining the position information of the vehicle and the position information of the laser radar; determining the heading angle of the vehicle based on the position information of the vehicle and the position information of the laser radar; and optimizing the yaw angle of the calibrated laser radar based on the heading angle.

21. The device according to claim 19 or 20, characterized in that The vehicle is equipped with a master laser radar and a slave laser radar, wherein the master laser radar is used to scan the environment in front of the vehicle, and the slave laser radar is used to scan the side and / or rear environment of the vehicle; the device also includes: a determination module, configured to determine position information of a plurality of markers based on the calibrated external parameters of the main lidar and a plurality of marker point clouds collected by the main lidar; a prediction module, configured to obtain a predicted position of a first marker based on the position information of the plurality of markers; a measurement module, configured to obtain a measured position of the first marker based on the calibrated external parameters of the laser radar and the first marker point cloud collected from the laser radar; A matching module is used to optimize the external parameters of the slave laser radar through the predicted position and the measured position.

22. A laser radar calibration device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 or the method described in any one of claims 8 to 15 when executing the instructions.

23. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 15 is implemented.

24. A computer program product comprising instructions, characterized in that When the method is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7 or implement the method according to any one of claims 8 to 15.

Images