Laser radar external parameter calibration method, product and vehicle

Abstract

The invention provides a laser radar external parameter calibration method, a product and a vehicle, and belongs to the technical field of radar external parameter calibration. The method comprises the following steps: acquiring an initial point cloud data set corresponding to each calibration device through a laser radar; according to the initial point cloud data set corresponding to each calibration device, executing a plane fitting strategy to obtain a fitting plane corresponding to each calibration device; executing a point cloud completion strategy according to the fitting plane corresponding to each calibration device to obtain a target point cloud data set corresponding to each calibration device; determining a second centroid coordinate of each calibration device under the laser radar coordinate system according to the target point cloud data set corresponding to each calibration device; and according to the first centroid coordinates corresponding to the calibration devices and the second centroid coordinates of the calibration devices, external parameters of the laser radar relative to the vehicle coordinate system are determined. The invention aims to improve the external parameter calibration precision of the laser radar.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of radar extrinsic parameter calibration, and specifically, to a lidar extrinsic parameter calibration method, product, and vehicle. Background Art

[0002] In the field of LiDAR extrinsic calibration, accurate extrinsic calibration is crucial for applications such as autonomous driving systems, robot navigation, and three-dimensional reconstruction. LiDAR extrinsic calibration aims to determine the spatial position and posture relationship between the LiDAR sensor and the vehicle or other platform.

[0003] At present, a variety of technologies have been used for external parameter calibration of lidar, but each technology has certain limitations. For example, the common method based on feature point matching has problems such as sensitivity to initial values and strong dependence on the environment. The method based on plane fitting may lead to inaccurate plane fitting in scenarios with sparse point clouds or occlusion problems, thereby affecting the accuracy of the calibration results. Summary of the Invention

[0004] The embodiments of the present application provide a laser radar extrinsic parameter calibration method, product, and vehicle, aiming to improve the accuracy of laser radar extrinsic parameter calibration.

[0005] In a first aspect, an embodiment of the present application provides a laser radar extrinsic parameter calibration method, the method comprising: Obtain the initial point cloud data set corresponding to each calibration device through the laser radar; Executing a plane fitting strategy according to the initial point cloud data set corresponding to each calibration device to obtain a fitting plane corresponding to each calibration device; Executing a point cloud completion strategy according to the fitting planes corresponding to the respective calibration devices to obtain a target point cloud data set corresponding to the respective calibration devices; Determining the second centroid coordinates of each calibration device in the laser radar coordinate system according to the target point cloud data set corresponding to each calibration device; The external parameters of the laser radar relative to the vehicle coordinate system are determined according to the first center of mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system.

[0006] Optionally, the method further includes: Respectively obtaining the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system; The first centroid coordinates corresponding to each calibration device are determined according to the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system.

[0007] Optionally, the initial point cloud data set corresponding to each calibration device is obtained by using a laser radar, including: The original point cloud data obtained by LiDAR; In response to the boundary coordinate selection operation, determining the regional point cloud data set corresponding to each calibration device; In the regional point cloud data sets of each calibration device, all point cloud data with reflection intensity greater than an intensity threshold corresponding to the calibration device are selected to form an initial point cloud data set corresponding to the calibration device.

[0008] Optionally, executing the plane fitting strategy includes: Based on the initial point cloud data set corresponding to the calibration device, executing a target number of iterations; In each iteration, three point cloud data are randomly selected from the initial point cloud data set corresponding to the calibration device, and the plane equation of this iteration process is calculated; based on the plane equation of this iteration process, the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device is determined; After executing the target number of iterations, the plane corresponding to the plane equation with the largest number of valid point clouds is selected as the fitting plane corresponding to the calibration device.

[0009] Optionally, determining the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device according to the plane equation of the current iterative process includes: respectively calculating the distance between the remaining point cloud data in the initial point cloud data set of the calibration device and the plane corresponding to the plane equation of this iterative process; The point cloud data with a distance less than the preset error threshold is regarded as a valid point cloud, and the number of all valid point clouds is counted.

[0010] Optionally, execute a point cloud completion strategy, including: Projecting all valid point clouds corresponding to the fitting plane of the calibration device onto the fitting plane to obtain a projection plane corresponding to the calibration device; In the projection plane corresponding to the calibration device, detecting whether point cloud data is missing at the upper edge and the lower edge respectively; When point cloud data is missing from the upper edge and / or the lower edge, the point cloud is completed at a target interval in the projection plane corresponding to the calibration device, and the two-dimensional coordinates of each completed point cloud in the projection plane are determined respectively; Obtaining three-dimensional point cloud coordinates of each completed point cloud according to the two-dimensional coordinates of each completed point cloud in the projection plane and the plane equation corresponding to the fitting plane of the calibration device; All valid point clouds and completed point clouds corresponding to the fitting plane of the calibration device are used as the target point cloud data set corresponding to the calibration device.

[0011] Optionally, when point cloud data is missing from the upper edge and / or the lower edge, completing the point cloud at a target interval in a projection plane corresponding to the calibration device includes: When point cloud data is missing on the upper edge, the distance between two adjacent point clouds on the upper edge of the projection plane is determined, the smallest distance is selected as the target interval, and the point cloud is completed on the upper edge until the completed point cloud overlaps with any point cloud; And / or, when point cloud data is missing on the lower edge, determine the distance between two adjacent point clouds on the lower edge of the projection plane, select the smallest distance as the target interval to complete the point cloud, and complete the point cloud on the lower edge until the completed point cloud repeats any point cloud and stops completing.

[0012] Optionally, determining the second centroid coordinates of each calibration device in the laser radar coordinate system according to the target point cloud data set corresponding to each calibration device includes: For any calibration device, respectively determine the average x-axis coordinate, the average y-axis coordinate, and the average z-axis coordinate of all point cloud data in the target point cloud data set of the calibration device; The average value of the x-axis coordinate, the average value of the y-axis coordinate, and the average value of the z-axis coordinate are used as the second center of mass coordinates of the calibration device in the lidar coordinate system.

[0013] Optionally, determining the external parameters of the laser radar relative to the vehicle coordinate system according to the first center-of-mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center-of-mass coordinates of each calibration device in the laser radar coordinate system includes: Calculate the transformation matrix of the rigid body transformation in three-dimensional space using the SVD algorithm based on the first center-of-mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center-of-mass coordinates of each calibration device in the lidar coordinate system to obtain a rotation matrix and a translation matrix; The rotation matrix and the translation matrix are used as external parameters of the laser radar relative to the vehicle coordinate system.

[0014] In a second aspect, an embodiment of the present application provides a laser radar extrinsic parameter calibration device, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the laser radar extrinsic parameter calibration method described in the first aspect of the embodiment is executed.

[0015] In a third aspect, an embodiment of the present application provides a non-volatile readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, the laser radar external parameter calibration method described in the first aspect of the embodiment is executed.

[0016] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the laser radar external parameter calibration method described in the first aspect of the embodiment.

[0017] In a fifth aspect, an embodiment of the present application provides a vehicle, which is used to execute the laser radar external parameter calibration method described in the first aspect of the embodiment.

[0018] Beneficial effects: The laser radar extrinsic parameter calibration method provided in this embodiment obtains the initial point cloud data set corresponding to each calibration device through the laser radar, then executes the plane fitting strategy to obtain the fitting plane corresponding to each calibration device, and executes the point cloud completion strategy based on the fitting plane corresponding to each calibration device to obtain the target point cloud data set corresponding to each calibration device; based on the target point cloud data set corresponding to each calibration device, the second center of mass coordinates of each calibration device in the laser radar coordinate system are determined; based on the first center of mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system, the extrinsic parameters of the laser radar relative to the vehicle coordinate system are determined.

[0019] By fitting a plane to the calibration device, abnormal point cloud data can be filtered out, and then point cloud completion can be performed to make the target point cloud data set of each calibration device more accurate, and then a more accurate second center of mass coordinate can be calculated, so that the final calculated external parameter accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flowchart of the steps of the laser radar extrinsic parameter calibration method proposed in one embodiment of the present application; Figure 2 is a schematic diagram of the original point cloud data proposed in one embodiment of the present application; Figure 3 is a schematic diagram of a regional point cloud data set provided by an embodiment of the present application; Figure 4 is a schematic diagram of a fitting plane proposed in one embodiment of the present application; Figure 5 is a schematic diagram of point cloud completion proposed in one embodiment of the present application; Figure 6 is a schematic diagram of point cloud completion provided by an embodiment of the present application; Figure 7 This is a functional module diagram of a laser radar extrinsic parameter calibration device proposed in one embodiment of the present application; Figure 8 Schematic diagram of a laser radar extrinsic calibration device according to an embodiment of the present application; Figure 9 is a schematic diagram of a non-volatile readable storage medium proposed in an embodiment of the present application; Figure 10 It is a schematic diagram of a computer program product proposed in one embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] In the field of LiDAR extrinsic calibration, accurate extrinsic calibration is crucial for applications such as autonomous driving systems, robot navigation, and three-dimensional reconstruction. LiDAR extrinsic calibration aims to determine the spatial position and posture relationship between the LiDAR sensor and the vehicle or other platform. Currently, a variety of technologies have been used for LiDAR extrinsic calibration, but each technology has certain limitations. Common LiDAR extrinsic calibration methods include: feature point matching-based methods, plane fitting-based methods, and point cloud completion technology.

[0024] Among them, the method based on feature point matching is to calibrate the external parameters between the lidar and the camera or other sensors through feature point matching; this method usually relies on feature points extracted from the lidar point cloud and image, such as edges, corners, etc., and solves the external parameters by minimizing the errors between feature points through optimization algorithms; however, this method has the following defects: it is sensitive to initial values, and the performance of the optimization algorithm is highly dependent on the setting of the initial external parameters. Inappropriate initial values may cause the algorithm to fall into a local optimal solution; it has strong environmental dependence, and in complex or dynamically changing environments, the extraction and matching of feature points becomes difficult, affecting the calibration accuracy.

[0025] The method based on plane fitting is to use the plane features in the lidar point cloud to calibrate the external parameters. For example, the ground plane or other planes of known geometric shapes are fitted from the point cloud through RANSAC (Random Sample Consensus), and the plane information is used to solve the external parameters. Although this method has improved the calibration accuracy and robustness to a certain extent, it still has the following shortcomings: point cloud sparsity. The lidar point cloud may be relatively sparse in some areas, resulting in inaccurate plane fitting, which in turn affects the calibration results; occlusion problem. In actual applications, occlusion by obstacles or vehicles may cause some plane features to be unable to be fully extracted, affecting the reliability of the calibration process; single plane limitation. Relying only on a single plane for calibration may not fully reflect the complex spatial relationship between the lidar and the vehicle, especially in unstructured environments.

[0026] Point cloud completion technology aims to overcome the problems of point cloud sparsity and occlusion by generating missing point cloud data through algorithms to improve the accuracy of subsequent processing (such as external parameter calibration). However, existing point cloud completion technologies often have the following defects: limited completion accuracy. Although the algorithm can generate a certain amount of point cloud data, the completion accuracy is often difficult to reach the level of the original data, especially in complex or unknown environments; poor generalization ability. The point cloud characteristics in different scenarios vary greatly. Existing completion algorithms may perform poorly when generalized to new scenarios.

[0027] Therefore, the calibration accuracy of the currently common laser radar extrinsic parameter calibration method is relatively weak, resulting in poor calibration effect. In order to improve the accuracy of the laser radar extrinsic parameter calibration, an embodiment of the present application provides a laser radar extrinsic parameter calibration method.

[0028] Reference Figure 1 , shows a flowchart of the steps of a laser radar extrinsic parameter calibration method in an embodiment of the present application, and the method may specifically include the following steps: S101: Obtaining an initial point cloud data set corresponding to each calibration device through a laser radar.

[0029] During the actual implementation process, multiple calibration devices can be set up in the scene area. The type and number of the calibration devices can be selected according to the needs of the actual application. The placement angles of the multiple calibration devices can be different, and the colors of the multiple calibration devices can also be different. For example, the calibration device can use a calibration plate. Specifically, a diffuse reflection plate can be selected as the calibration device, and a diffuse reflection plate with a reflectivity greater than a preset reflectivity value can be selected as the calibration device. For example, the preset reflectivity value can be 90%. The calibration device with a higher reflectivity can make the point cloud data obtained by the lidar richer and more accurate.

[0030] The non-calibration parts with reflective properties in the calibration device can be covered in advance to prevent the point cloud data of the non-calibration parts collected by the lidar from affecting subsequent processing. For example, if the calibration device has non-calibration parts such as a metal tripod, the metal tripod of the calibration device can be wrapped with foam cotton to prevent the point cloud data of the metal tripod obtained by the lidar from affecting the subsequent identification of the initial point cloud data set corresponding to the calibration device.

[0031] During the actual implementation process, the first center of mass coordinates of each calibration device in the vehicle coordinate system can be first determined. Specifically, the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system can be obtained respectively; according to the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system, the first center of mass coordinates corresponding to each calibration device can be determined.

[0032] Specifically, according to the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system, the calculation formula for determining the first centroid coordinates corresponding to each calibration device is:

[0033] Among them, (X car-mean, Y car-mean, Z car-mean) are the first center of mass coordinates corresponding to the calibration device; (x1, y1, z1) are the corner point coordinates of the first corner point corresponding to the calibration device, (x2, y2, z2) are the corner point coordinates of the second corner point corresponding to the calibration device, (x3, y3, z3) are the corner point coordinates of the third corner point corresponding to the calibration device, and (x4, y4, z4) are the corner point coordinates of the fourth corner point corresponding to the calibration device.

[0034] In the actual implementation process, in addition to being applied to vehicles to determine the external parameters of the lidar on the vehicle relative to the vehicle coordinate system, this method can also be applied to verification scenarios for determining the external parameters of the lidar. In the verification scenario, a total station is used as the origin of the vehicle coordinate system, and the total station is used to measure the coordinates of the four corner points of each calibration device in the vehicle coordinate system.

[0035] In the actual implementation process, if this method is applied to a verification scenario for determining the external parameters of a laser radar, a laser radar is set up in the scene area, and the relative positions of the laser radar and the total station can be set according to the needs of the actual application. For example, the installation position of the laser radar can be set according to the relative position of the laser radar on any real vehicle and the vehicle coordinate system.

[0036] Then, the point cloud data within the scene area obtained by the lidar in the scene area can be obtained through the ROS system (Robot Operating System). For example, the point cloud data obtained by the lidar can be subscribed through the topic mechanism in the ROS system, the bag information can be obtained in the ROS system, and the bag information can be converted into point cloud information in the pcd format that can be read by MATLAB or Python through the ROS system.

[0037] In a feasible implementation, obtaining the initial point cloud data set corresponding to each calibration device through a laser radar includes the following steps: A1: Original point cloud data obtained by LiDAR.

[0038] Reference Figure 2 , shows a schematic diagram of the original point cloud data provided by this embodiment. The original point cloud data obtained by the laser radar is the point cloud data of the scene area where the laser radar is set. The original point cloud data includes all point cloud data corresponding to each calibration device, and the coordinate system of the point cloud data is the laser radar coordinate system.

[0039] A2: In response to the boundary coordinate selection operation, determine the regional point cloud data sets corresponding to the respective calibration devices.

[0040] In order to improve the efficiency of extracting point cloud data corresponding to each calibration device from the original point cloud data, the boundary coordinate selection operation of the calibration device can be performed manually. Specifically, a point cloud data has three-dimensional coordinates. The coordinate range corresponding to all point cloud data of a calibration device can be manually input into MATLAB, such as the coordinate range of the point cloud data corresponding to each corner point or the point cloud data near the corner point, that is, the range of xyz coordinates. The range of the maximum and minimum values of the xyz coordinates is used to preliminarily determine the range of the boundary point cloud data of a single calibration device. By responding to the boundary coordinate selection operation, the regional point cloud data set corresponding to each calibration device is determined according to the coordinate range corresponding to all point cloud data of a calibration device.

[0041] Reference Figure 3 , a schematic diagram of the regional point cloud data set provided by this embodiment is shown. By selecting the boundary coordinates of the calibration device, such as setting the coordinates of the point cloud corresponding to the four corner points of the calibration device, the ROI area (Region of Interest) of the point cloud data of the calibration device can be preliminarily extracted. The point cloud data within the ROI area constitutes the regional point cloud data set of the calibration device.

[0042] A3: In the regional point cloud data sets of each calibration device, all point cloud data with reflection intensity greater than the intensity threshold corresponding to the calibration device are selected to form the initial point cloud data set corresponding to the calibration device.

[0043] Specifically, in the regional point cloud data set corresponding to each calibration device, it is determined whether the point cloud data should be added to the initial point cloud data set corresponding to the calibration device based on the intensity corresponding to each point cloud data, wherein all point cloud data in the regional point cloud data set whose intensity is greater than the intensity threshold corresponding to the calibration device are used as the initial point cloud data set corresponding to the calibration device.

[0044] In the actual implementation process, since the setting positions and setting angles of each calibration device are different, the angles of the received laser radar beams are also different, and thus the intensity of the point cloud data of the calibration device will also be different. Therefore, a corresponding intensity threshold can be set for each calibration device, and the setting of the intensity threshold can be set according to the needs of the actual application.

[0045] Furthermore, for the calibration devices within the non-dense beam range of the laser radar, the point cloud data obtained by the laser radar is relatively sparse, and the point cloud data of these calibration devices can be ignored. The point cloud data of multiple calibration devices within the dense beam range of the laser radar are retained to improve the accuracy of the laser radar external parameter calibration.

[0046] S102: Executing a plane fitting strategy according to the initial point cloud data set corresponding to each calibration device to obtain a fitting plane corresponding to each calibration device.

[0047] In a feasible implementation, a plane fitting strategy may be executed based on a RANSAC algorithm to determine the fitting plane of each calibration device through random sampling and iteration.

[0048] Specifically, executing the plane fitting strategy includes executing a target number of iterative processes based on the initial point cloud data set corresponding to the calibration device; in each iterative process, randomly selecting three point cloud data from the initial point cloud data set corresponding to the calibration device, and calculating the plane equation of this iterative process; according to the plane equation of this iterative process, determining the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device; after executing the target number of iterative processes, selecting the plane corresponding to the plane equation with the largest number of valid point clouds as the fitting plane corresponding to the calibration device.

[0049] In a feasible implementation, when determining the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device based on the plane equation of this iterative process, the distance between each of the remaining point cloud data in the initial point cloud data set of the calibration device and the plane corresponding to the plane equation of this iterative process can be calculated respectively; then, the point cloud data with a distance less than a preset error threshold is regarded as a valid point cloud, and the number of all valid point clouds is counted.

[0050] The target number of iterations and the preset error threshold can be set according to the actual application requirements. For example, the target number of iterations can be set to 20 times and the preset error threshold can be set to 0.1 cm. For example, taking the initial point cloud data set of calibration device A as including 20 point cloud data, the target number of times is 20, and the preset error threshold is 0.1 cm, the process of executing the plane fitting strategy to obtain the fitting plane corresponding to calibration device A is as follows: Three non-collinear points can determine a plane, and the formula for the plane equation is: Ax1+By1+Cz1+D=0 Ax2+By2+Cz2+D=0 Ax3+By3+Cz3+D=0 In the first iteration process, three point cloud data are randomly selected from the initial point cloud data set corresponding to the calibration device A, and then the coordinates of these three point cloud data are substituted into the plane equation to solve the parameters A1, B1, C1 and D1, and then the plane equation of the first iteration process is obtained as A1x+B1y+C1z+D1=0.

[0051] Then, for the remaining 17 point cloud data in the initial point cloud data set of calibration device A, the distance between each point cloud data and the plane corresponding to the plane equation A1x+B1y+C1z+D1=0 is calculated respectively. If the distance between a point cloud data and the plane is less than the preset error threshold of 0.1cm, the point cloud data is a valid point cloud of calibration device A. The number of all valid point clouds in the first iteration process is counted.

[0052] In the second iteration process, three point cloud data are randomly selected from the initial point cloud data set corresponding to the calibration device A, the coordinates of these three point cloud data are substituted into the plane equation, and the parameters A2, B2, C2 and D2 are solved to obtain the plane equation A2x+B2y+C2z+D2=0 of the second iteration process.

[0053] Then, for the remaining 17 point cloud data in the initial point cloud data set of the current calibration device A, the number of all valid point clouds in the second iteration process is determined again based on the distance from the point cloud data to the plane and the preset error threshold of 0.1 cm.

[0054] The iteration process is stopped when the number of iterations reaches 20, and the plane corresponding to the plane equation with the largest number of valid point clouds in the 20 iterations is selected as the fitting plane corresponding to the calibration device A. Assuming that the number of valid point clouds is the largest in the second iteration, the plane equation of the fitting plane corresponding to the calibration device A is A2x+B2y+C2z+D2=0.

[0055] Reference Figure 4 , showing a schematic diagram of the fitting plane provided in an embodiment of the present application. This embodiment determines the plane equation of the calibration device through multiple iterations of the RANSAC algorithm, which can identify and eliminate abnormal point cloud data in the initial point cloud data set corresponding to the calibration device, reduce the noise impact of abnormal point cloud data on the subsequent external parameter calculation process, and improve the accuracy and robustness of the external parameter calibration of the lidar.

[0056] S103: Execute a point cloud completion strategy according to the fitting planes corresponding to the respective calibration devices to obtain a target point cloud data set corresponding to the respective calibration devices.

[0057] Since the edges of the calibration device, especially the upper and lower edges, have a weak reflection effect on the lidar, the upper and lower point cloud data are usually missing, or when the local reflection effect of the calibration device is poor, the intensity of the point cloud data detected by the lidar is weak and is filtered out, resulting in missing point clouds. In order to further improve the accuracy of the external parameter calibration, the point cloud completion strategy can be executed according to the fitting plane corresponding to each calibration device to obtain a target point cloud data set with richer point cloud data corresponding to each calibration device.

[0058] In a feasible implementation, the process of executing the point cloud completion strategy includes: projecting all valid point clouds corresponding to the fitting plane of the calibration device onto the fitting plane respectively to obtain a projection plane corresponding to the calibration device.

[0059] Specifically, the projection algorithm is as follows: The plane equation of a three-dimensional space plane is Ax+By+Cz+D=0 (1) Assume that the coordinates of the three-dimensional space point that is not on the plane are marked as ( x 0, y 0, z 0), the coordinates of its projection point on the plane are marked as ( x p , y p , z p ), because the projection point to the current point is perpendicular to the plane, according to the vertical constraint, y p and z pThe following conditions must be met:

[0060] Substituting formulas (2) and (3) into formula (1), we can obtain:

[0061] Substituting formula (4) into formulas (2) and (3), we can obtain:

[0062] Furthermore, according to the above-mentioned projection algorithm, all valid point clouds corresponding to the fitting plane of the calibration device can be projected onto the fitting plane of the calibration device respectively to obtain the projection plane corresponding to the calibration device.

[0063] Then, in the projection plane corresponding to the calibration device, the upper and lower edges are respectively detected to see whether point cloud data is missing. In addition to detecting whether point cloud data is missing at the upper and lower edges, during the actual implementation process, it is also possible to detect whether point cloud data is missing in other areas of the projection plane. If point cloud data is missing, point cloud completion can be performed.

[0064] Reference Figure 5 , shows a schematic diagram of point cloud completion provided by an embodiment of the present application. This embodiment takes the missing point cloud at the upper and lower edges as an example. When it is detected that the upper edge and / or the lower edge are missing point cloud data, the point cloud is completed at the target interval in the projection plane corresponding to the calibration device, and the two-dimensional coordinates of each completed point cloud in the projection plane are determined respectively.

[0065] The target interval can be selected according to the needs of actual application, but in order to improve the density of point cloud data, in this embodiment, when point cloud data is missing on the upper edge, the distance between two adjacent point clouds in the upper edge of the projection plane is determined, and the minimum distance is selected as the target interval to complete the point cloud on the upper edge until the completed point cloud repeats any point cloud; when point cloud data is missing on the lower edge, the distance between two adjacent point clouds in the lower edge of the projection plane is determined, and the minimum distance is selected as the target interval to complete the point cloud, and the point cloud is completed on the lower edge until the completed point cloud repeats any point cloud.

[0066] After the point cloud data in the projection plane of the point cloud completion and calibration device is completed, the three-dimensional point cloud coordinates of each completed point cloud are obtained according to the two-dimensional coordinates of each completed point cloud in the projection plane and the plane equation corresponding to the fitting plane of the calibration device, and then the three-dimensional coordinates of the completed point cloud data are obtained.

[0067] Finally, all valid point clouds and completed point clouds corresponding to the fitting plane of the calibration device are used as the target point cloud data set corresponding to the calibration device.

[0068] Reference Figure 6 , shows a schematic diagram of point cloud completion provided by an embodiment of the present application. For example, assuming that the upper edge of the projection plane is missing point cloud data, every 100 points on the two-dimensional projection plane is X min Complete a point cloud, X min It is the distance between the two point clouds with the smallest distance in the existing point clouds on the upper edge.

[0069] Assume that the two-dimensional coordinates of the multiple point cloud data added on the upper edge are ( x 1, y 1), ( x 2, y 2), ( x n , y n ); Substituting the two-dimensional coordinates of each supplemented point cloud data into the plane equation of the fitting plane of the calibration device, the z coordinate in the three-dimensional coordinates of the point cloud data can be obtained. Assume that the plane equation of the fitting plane is recorded as: Ax+By+Cz+D=0; but:

[0070] In this embodiment, after identifying and eliminating abnormal point clouds based on the RANSAC algorithm, the best fitting plane corresponding to each calibration device is iteratively determined, and then the valid point cloud corresponding to the fitting plane of the calibration device is projected onto the fitting plane, and the missing point cloud data in the projection plane is completed, effectively filling the missing area of the point cloud, making the calibration process more adaptable to environmental changes and improving the accuracy and robustness of the external parameter calibration.

[0071] S104: Determine the second centroid coordinates of each calibration device in the laser radar coordinate system according to the target point cloud data set corresponding to each calibration device.

[0072] After completing the missing point cloud data of the calibration device, in the process of determining the second center of mass coordinates of each calibration device in the lidar coordinate system according to the completed target point cloud data set corresponding to the calibration device, for any calibration device, the x-axis coordinate average value, y-axis coordinate average value and z-axis coordinate average value of all point cloud data in the target point cloud data set of the calibration device are determined respectively, and the x-axis coordinate average value, y-axis coordinate average value and z-axis coordinate average value are used as the second center of mass coordinates of the calibration device in the lidar coordinate system.

[0073] Specifically, the calculation formula of the second centroid coordinates is:

[0074] in,( X lidar-mean , Y lidar-mean , Z lidar-mean ) is the coordinate of the second center of mass of the calibration device in the laser radar coordinate system, ( x i , y i , z i ) is the first point in the target point cloud data set of the calibration device i The point cloud coordinates of the point cloud data, i =1, 2...n, where n is the total number of point cloud data in the target point cloud data set of the calibration device.

[0075] S105: Determine the external parameters of the laser radar relative to the vehicle coordinate system based on the first center of mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system.

[0076] In a feasible implementation, in the process of determining the external parameters of the laser radar relative to the vehicle coordinate system based on the first center of mass coordinates of each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system, the transformation matrix of the rigid body transformation in the three-dimensional space can be calculated by the SVD (Singular Value Decomposition) algorithm according to the first center of mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system to obtain the rotation matrix and the translation matrix; the rotation matrix R and the translation matrix T are used as the external parameters of the laser radar relative to the vehicle coordinate system.

[0077] The SVD algorithm is a powerful tool in linear algebra that reveals the most essential transformations of matrices. It can decompose a matrix of any shape into the product of an orthogonal matrix, a diagonal matrix (singular value matrix), and the transpose of another orthogonal matrix. Rigid body transformation in three-dimensional space is the rotation and translation of an object in three-dimensional space, so that the shape and size of the object remain unchanged.

[0078] The lidar extrinsic parameter calibration method provided in this embodiment, by executing a plane fitting strategy based on the RANSAC algorithm, can identify and eliminate abnormal points in the plane fitting when fitting the plane of the calibration device, and then execute a point cloud completion strategy to complete the missing point cloud data in the calibration device, so that the target point cloud data set of each calibration device can be made more accurate, and then a more accurate second center of mass coordinate of each calibration device in the lidar coordinate system can be obtained. Therefore, according to the first center of mass coordinate corresponding to each calibration device and the second center of mass coordinate of each calibration device in the lidar coordinate system, the extrinsic parameters of the lidar relative to the vehicle coordinate system can be determined, thereby improving the calibration accuracy and robustness, and making the lidar extrinsic parameter calibration process more adaptable to environmental changes, which can overcome the shortcomings of the current lidar extrinsic parameter calibration method and provide more reliable technical support for autonomous driving systems, robot navigation and other fields.

[0079] Reference Figure 7 , shows a functional module diagram of a laser radar extrinsic parameter calibration device provided in an embodiment of the present application, the device comprising: The point cloud data acquisition module 100 is used to acquire the initial point cloud data set corresponding to each calibration device through the laser radar; A plane fitting module 200 is configured to execute a plane fitting strategy based on the initial point cloud data set corresponding to each calibration device to obtain a fitting plane corresponding to each calibration device; The point cloud completion module 300 is used to execute a point cloud completion strategy according to the fitting planes corresponding to the respective calibration devices to obtain a target point cloud data set corresponding to the respective calibration devices; A centroid coordinate acquisition module 400 is configured to determine the second centroid coordinates of each calibration device in the laser radar coordinate system based on the target point cloud data set corresponding to each calibration device; The external parameter determination module 500 is used to determine the external parameters of the laser radar relative to the vehicle coordinate system based on the first center of mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system.

[0080] Optionally, the device further includes a first centroid coordinate determination module, configured to: Respectively obtaining the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system; The first centroid coordinates corresponding to each calibration device are determined according to the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system.

[0081] Optionally, the point cloud data acquisition module includes: The original point cloud data acquisition unit is used to acquire the original point cloud data obtained by the laser radar; a regional point cloud data set determining unit, configured to determine the regional point cloud data sets corresponding to the respective calibration devices in response to a boundary coordinate selection operation; The initial point cloud data set determining unit is used to select all point cloud data with reflection intensity greater than the intensity threshold corresponding to the calibration device from the regional point cloud data sets of each calibration device to form the initial point cloud data set corresponding to the calibration device.

[0082] Optionally, the plane fitting module includes: An iterative unit, configured to: execute a target number of iterations based on an initial point cloud data set corresponding to the calibration device; In each iteration, three point cloud data are randomly selected from the initial point cloud data set corresponding to the calibration device, and the plane equation of this iteration process is calculated; based on the plane equation of this iteration process, the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device is determined; The fitting plane determination unit is used to select the plane corresponding to the plane equation with the largest number of valid point clouds as the fitting plane corresponding to the calibration device after executing the target number of iterations.

[0083] Optionally, the iteration unit includes: a distance calculation subunit, configured to respectively calculate the distance between each of the remaining point cloud data in the initial point cloud data set of the calibration device and the plane corresponding to the plane equation of this iterative process; The statistical unit is used to take the point cloud data with a distance less than a preset error threshold as a valid point cloud and count the number of all valid point clouds.

[0084] Optionally, the point cloud completion module includes: A projection unit, configured to project all valid point clouds corresponding to the fitting plane of the calibration device onto the fitting plane to obtain a projection plane corresponding to the calibration device; A point cloud missing detection unit is used to detect whether point cloud data is missing at the upper edge and the lower edge in the projection plane corresponding to the calibration device; a point cloud completion unit, configured to complete the point cloud at a target interval in the projection plane corresponding to the calibration device when point cloud data is missing from the upper edge and / or the lower edge, and to determine the two-dimensional coordinates of each completed point cloud in the projection plane; A coordinate calculation unit, configured to obtain the three-dimensional point cloud coordinates of each completed point cloud according to the two-dimensional coordinates of each completed point cloud in the projection plane and the plane equation corresponding to the fitting plane of the calibration device; The target point cloud data set determining unit is used to take all valid point clouds and completed point clouds corresponding to the fitting plane of the calibration device as the target point cloud data set corresponding to the calibration device.

[0085] Optionally, the point cloud completion unit includes: The first completion subunit is used to determine the distance between two adjacent point clouds on the upper edge of the projection plane when point cloud data is missing on the upper edge, select the minimum distance as the target interval, and complete the point cloud on the upper edge until the completed point cloud overlaps with any point cloud; The second completion subunit is used to determine the distance between two adjacent point clouds in the lower edge of the projection plane when point cloud data is missing on the lower edge, select the smallest distance as the target interval to complete the point cloud, and complete the point cloud on the lower edge until the completed point cloud repeats any point cloud.

[0086] Optionally, the centroid coordinate acquisition module includes: a coordinate average value calculation unit, for determining, for any calibration device, an x-axis coordinate average value, a y-axis coordinate average value, and a z-axis coordinate average value of all point cloud data in a target point cloud data set of the calibration device; The second center of mass coordinate determination unit is used to use the average value of the x-axis coordinate, the average value of the y-axis coordinate and the average value of the z-axis coordinate as the second center of mass coordinate of the calibration device in the laser radar coordinate system.

[0087] Optionally, the external parameter determination module includes: a matrix calculation unit, configured to calculate a transformation matrix of a rigid body transformation in a three-dimensional space by using an SVD algorithm based on the first centroid coordinates corresponding to each calibration device in the vehicle coordinate system and the second centroid coordinates of each calibration device in the lidar coordinate system, thereby obtaining a rotation matrix and a translation matrix; The external parameter determination unit is used to use the rotation matrix and the translation matrix as external parameters of the laser radar relative to the vehicle coordinate system.

[0088] Reference Figure 8 , shows a schematic diagram of a laser radar extrinsic parameter calibration device provided in an embodiment of the present application, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein the processor executes the laser radar extrinsic parameter calibration method described in the embodiment when executing the computer program.

[0089] Reference Figure 9 , shows a schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the laser radar extrinsic parameter calibration method described in the embodiment is executed.

[0090] Reference Figure 10 , shows a schematic diagram of a computer program product provided in an embodiment of the present application, including a computer program / instruction, which, when executed by a processor, implements the laser radar extrinsic parameter calibration method described in the embodiment.

[0091] This embodiment also provides a vehicle, which includes the laser radar external parameter calibration device described in this embodiment.

[0092] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0093] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, apparatuses, or computer program products. Therefore, the embodiments of the present application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware. Furthermore, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0097] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0098] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0099] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A laser radar extrinsic parameter calibration method, characterized in that: The method comprises: Obtain the initial point cloud data set corresponding to each calibration device through the laser radar; Executing a plane fitting strategy according to the initial point cloud data set corresponding to each calibration device to obtain a fitting plane corresponding to each calibration device; Executing a point cloud completion strategy according to the fitting planes corresponding to the respective calibration devices to obtain a target point cloud data set corresponding to the respective calibration devices; Determining the second centroid coordinates of each calibration device in the laser radar coordinate system according to the target point cloud data set corresponding to each calibration device; The external parameters of the laser radar relative to the vehicle coordinate system are determined according to the first center of mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center of mass coordinates of each calibration device in the laser radar coordinate system.

2. The method according to claim 1, characterized in that The method further comprises: Respectively obtaining the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system; The first centroid coordinates corresponding to each calibration device are determined according to the corner point coordinates of the four corner points of each calibration device in the vehicle coordinate system.

3. The method according to claim 1, characterized in that The initial point cloud data set corresponding to each calibration device is obtained through the lidar, including: The original point cloud data obtained by LiDAR; In response to the boundary coordinate selection operation, determining the regional point cloud data set corresponding to each calibration device; In the regional point cloud data sets of each calibration device, all point cloud data with reflection intensity greater than an intensity threshold corresponding to the calibration device are selected to form an initial point cloud data set corresponding to the calibration device.

4. The method according to claim 1, wherein Implementation of plane fitting strategies includes: Based on the initial point cloud data set corresponding to the calibration device, executing a target number of iterations; In each iteration, three point cloud data are randomly selected from the initial point cloud data set corresponding to the calibration device, and the plane equation of this iteration process is calculated; based on the plane equation of this iteration process, the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device is determined; After executing the target number of iterations, the plane corresponding to the plane equation with the largest number of valid point clouds is selected as the fitting plane corresponding to the calibration device.

5. The method according to claim 4, characterized in that Determining the number of valid point clouds in the remaining point cloud data in the initial point cloud data set of the calibration device according to the plane equation of the current iterative process includes: respectively calculating the distance between the remaining point cloud data in the initial point cloud data set of the calibration device and the plane corresponding to the plane equation of this iterative process; The point cloud data with a distance less than the preset error threshold is regarded as a valid point cloud, and the number of all valid point clouds is counted.

6. The method according to claim 1, characterized in that Execute point cloud completion strategy, including: Projecting all valid point clouds corresponding to the fitting plane of the calibration device onto the fitting plane to obtain a projection plane corresponding to the calibration device; In the projection plane corresponding to the calibration device, detecting whether point cloud data is missing at the upper edge and the lower edge respectively; When point cloud data is missing from the upper edge and / or the lower edge, the point cloud is completed at a target interval in the projection plane corresponding to the calibration device, and the two-dimensional coordinates of each completed point cloud in the projection plane are determined respectively; Obtaining three-dimensional point cloud coordinates of each completed point cloud according to the two-dimensional coordinates of each completed point cloud in the projection plane and the plane equation corresponding to the fitting plane of the calibration device; All valid point clouds and completed point clouds corresponding to the fitting plane of the calibration device are used as the target point cloud data set corresponding to the calibration device.

7. The method according to claim 6, characterized in that When point cloud data is missing from the upper edge and / or the lower edge, the point cloud is completed at a target interval in the projection plane corresponding to the calibration device, including: When point cloud data is missing on the upper edge, the distance between two adjacent point clouds on the upper edge of the projection plane is determined, the smallest distance is selected as the target interval, and the point cloud is completed on the upper edge until the completed point cloud overlaps with any point cloud; And / or, when point cloud data is missing on the lower edge, determine the distance between two adjacent point clouds on the lower edge of the projection plane, select the smallest distance as the target interval to complete the point cloud, and complete the point cloud on the lower edge until the completed point cloud repeats any point cloud and stops completing.

8. The method according to claim 1, characterized in that Determining the second centroid coordinates of each calibration device in the laser radar coordinate system according to the target point cloud data set corresponding to each calibration device includes: For any calibration device, respectively determine the average x-axis coordinate, the average y-axis coordinate, and the average z-axis coordinate of all point cloud data in the target point cloud data set of the calibration device; The average value of the x-axis coordinate, the average value of the y-axis coordinate, and the average value of the z-axis coordinate are used as the second center of mass coordinates of the calibration device in the lidar coordinate system.

9. The method according to claim 1, characterized in that Determining the external parameters of the laser radar relative to the vehicle coordinate system according to the first center-of-mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center-of-mass coordinates of each calibration device in the laser radar coordinate system, including: Calculate the transformation matrix of the rigid body transformation in three-dimensional space using the SVD algorithm based on the first center-of-mass coordinates corresponding to each calibration device in the vehicle coordinate system and the second center-of-mass coordinates of each calibration device in the lidar coordinate system to obtain a rotation matrix and a translation matrix; The rotation matrix and the translation matrix are used as external parameters of the laser radar relative to the vehicle coordinate system.

10. A laser radar external parameter calibration device, characterized in that: include: At least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, it executes the laser radar extrinsic parameter calibration method according to any one of claims 1 to 9.

11. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein, when the computer program is executed by a processor, the laser radar extrinsic parameter calibration method according to any one of claims 1 to 9 is executed.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the laser radar extrinsic parameter calibration method described in any one of claims 1 to 9 is implemented.

13. A vehicle, characterized in that: The vehicle is used to execute the lidar extrinsic parameter calibration method described in any one of claims 1-9.

Images