Multi-laser radar external parameter self-calibration method and system, terminal device and storage medium

Abstract

The application discloses a kind of multi-laser radar external parameter self-calibration method, system, terminal equipment and storage medium, method includes: obtaining first point cloud data and second point cloud data, wherein, first point cloud data is collected by first laser radar, and second point cloud data is collected by second laser radar;With the center of first laser radar and second laser radar respectively establishes coordinate system;Determine reference coordinate system;Calculate the yaw angle of first laser radar coordinate system and second laser radar coordinate system to reference coordinate system;Calculate the pitch angle and roll angle of first laser radar coordinate system and second laser radar coordinate system to reference coordinate system;According to the yaw angle, pitch angle and roll angle, and the installation setting of back-pack multi-laser radar system, calculate the translation parameter of second laser radar coordinate system to first laser radar coordinate system.The application solves the problems that the existing technology carries out multi-laser radar calibration, the calibration process is complex, depends on external calibration object and leads to high calibration cost.

Description

Technical Field

[0001] This invention relates to a method, system, terminal device, and storage medium for self-calibration of multiple lidar extrinsic parameters, belonging to the field of lidar technology. Background Technology

[0002] Backpack lidar is easy to carry and operate, and has become a major tool for mobile laser measurement in recent years. Lidar emits laser points to sense the surrounding environment and collect its three-dimensional information, featuring a large detection range and strong anti-interference capabilities. Existing backpack lidar systems typically carry one or more lidars, aiming to collect rich point cloud information of the surrounding environment using different field-of-view angles. Therefore, before using a backpack lidar system, it is necessary to calibrate the relative positions of the different lidars, i.e., calculate the rotation and translation parameters between the lidars, and unify the point clouds collected by multiple lidars into the same coordinate system. The accuracy of this calibration has a significant impact on the fusion of point cloud data from multiple lidars.

[0003] There are two main methods for extrinsic parameter calibration of lidar: manual calibration and reflector calibration. Manual calibration involves measuring the coordinate system transformation between lidars using manual or instrumental methods. However, this method requires precise measuring instruments and necessitates re-measuring each time the backpack lidar is deployed, resulting in high labor and time costs and limiting its widespread applicability. Reflector calibration requires a specific calibration environment. Lidars are mutually calibrated by matching reflector control points, but matching these control points is difficult due to the different field of view angles of the lidars. Furthermore, this method relies on external calibration objects, leading to high calibration costs and low efficiency. Therefore, designing an efficient backpack multi-lidar extrinsic parameter calibration method is of great significance. Summary of the Invention

[0004] In view of this, the present invention provides a method, system, terminal device and storage medium for self-calibration of multiple lidar extrinsic parameters. It aims to solve the problems of complex calibration process and high calibration cost due to reliance on external calibration objects when calibrating multiple lidars in the prior art. The present invention gets rid of environmental limitations and realizes high-precision and high-efficiency automated calibration for the backpack rotating lidar.

[0005] The first objective of this invention is to provide a method for self-calibrating extrinsic parameters of multiple lidar systems.

[0006] The second objective of this invention is to provide a multi-laser radar extrinsic parameter self-calibration system.

[0007] The third objective of this invention is to provide a terminal device.

[0008] A fourth objective of this invention is to provide a storage medium.

[0009] The first objective of this invention can be achieved by adopting the following technical solution:

[0010] A self-calibration method for extrinsic parameters of multiple lidar systems is applied to a backpack-based multi-lidar system. The lidar system includes a first lidar and a second lidar. The first lidar is fixed horizontally, and the second lidar rotates relative to the first lidar via a rotating bolt. The rotation angle is adjustable. The method includes:

[0011] Acquire first point cloud data and second point cloud data, wherein the first point cloud data is collected by a first lidar and the second point cloud data is collected by a second lidar;

[0012] Establish coordinate systems with the centers of the first and second lidars respectively;

[0013] Establish a reference coordinate system, where the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the centers of the first and second lidars, and the Z-axis is perpendicular to the ground.

[0014] Based on the first point cloud data and the second point cloud data, calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system;

[0015] Based on the first point cloud data and the second point cloud data, and according to the yaw angle, calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system.

[0016] Based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-LiDAR system, calculate the translation parameters from the second LiDAR coordinate system to the first LiDAR coordinate system.

[0017] Furthermore, the step of calculating the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data includes:

[0018] Based on the corresponding point cloud data, segment the corresponding wall point cloud and calculate the longitudinal mean of the wall point cloud;

[0019] Based on the longitudinal mean, and according to the angle compensation method, the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system is calculated.

[0020] Furthermore, the step of calculating the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the longitudinal mean and according to the angle compensation method includes:

[0021] Set the rotation matrix to contain only the yaw angle, as follows:

[0022]

[0023]

[0024] Where yaw1 represents the yaw angle from the first lidar coordinate system to the reference coordinate system, yaw2 represents the yaw angle from the second lidar coordinate system to the reference coordinate system, and R... yaw1 R represents a rotation matrix containing only the yaw angle yaw1. yaw2 This represents a rotation matrix containing only the yaw angle yaw2;

[0025] Based on the rotation matrix, and through angle compensation, the first variance between each point in the corresponding wall point cloud and its corresponding longitudinal mean is calculated, and then a first optimization function is constructed, wherein the first optimization function is as follows:

[0026]

[0027]

[0028] in, Indicated based on R yaw1 The first optimization function constructed, Indicated based on R yaw 2. The second optimization function constructed, This represents the longitudinal mean of the wall point cloud obtained from the first lidar. X represents the longitudinal mean of the wall point cloud from the second lidar. i Let X represent any point in the wall point cloud of the first lidar. j w1 represents any point in the wall point cloud of the second lidar, w2 represents the number of wall point clouds of the first lidar, and w2 represents the number of wall point clouds of the second lidar.

[0029] Based on the first optimization function, and according to the nonlinear least squares optimization method, the yaw angles from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system are obtained.

[0030] Furthermore, the step of calculating the pitch and roll angles from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data, according to the yaw angle, includes:

[0031] Based on the first point cloud data and the second point cloud data, according to the yaw angle, the ground point cloud of the first lidar and the ground point cloud of the second lidar are extracted, and the average altitude is calculated.

[0032] Based on the average height, and according to the angle compensation method, the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system are calculated.

[0033] Furthermore, the step of extracting the ground point cloud of the first lidar and the ground point cloud of the second lidar based on the first point cloud data and the second point cloud data, according to the yaw angle, includes:

[0034] Based on the corresponding yaw angle, the corresponding point cloud data is rotated around the Z-axis to obtain the first and second point cloud data after rotation, wherein the Z-axis belongs to the reference coordinate system;

[0035] Multiple planes are fitted to the rotated first point cloud data and the segments are extracted. The angle between the normal vector of the fitted plane and the Z-axis is compared. The plane containing the normal vector that meets the angle threshold is taken as the ground point cloud of the first lidar.

[0036] Multiple planes are fitted to the rotated second point cloud data and the segments are extracted. The angle between the normal vector of the fitted plane and the Z-axis is compared, and the mean Z-coordinate of the fitted plane is calculated. The plane containing the normal vector that meets the angle threshold and has a smaller mean Z-coordinate is taken as the ground point cloud of the second lidar.

[0037] Furthermore, the calculation of the pitch and roll angles from the first and second lidar coordinate systems to the reference coordinate system based on the average height and according to the angle compensation method includes:

[0038] Set up a rotation matrix that contains only pitch and roll angles, as follows:

[0039]

[0040]

[0041]

[0042]

[0043] Where roll1 and pitch1 are the roll and pitch angles from the first lidar coordinate system to the reference coordinate system, respectively; roll2 and pitch2 are the roll and pitch angles from the second lidar coordinate system to the reference coordinate system, respectively; R roll1 R represents a rotation matrix containing only the roll angle roll1. roll2 R represents a rotation matrix containing only the roll angle roll2. pitch1 R represents a rotation matrix containing only the pitch angle pitch1. pitch2 This represents a rotation matrix containing only the pitch angle pitch2;

[0044] Based on the rotation matrix, the second variance between each point in the ground point cloud and its corresponding mean height is calculated using angle compensation. This is then used to construct a second optimization function, which is expressed as follows:

[0045]

[0046]

[0047] in, Indicated based on R roll1 R pitch1 The first optimization function constructed, Indicated based on R roll2 R pitch2 The second optimization function constructed, This represents the average height of the ground point cloud from the first lidar sensor. Z represents the average height of the ground point cloud from the second lidar. i Z' represents any point in the ground point cloud of the first lidar. j ′ represents any point in the ground point cloud of the second lidar, g1 represents the number of ground point clouds of the first lidar, and g2 represents the number of ground point clouds of the second lidar.

[0048] Based on the second optimization function, and according to the nonlinear least squares optimization method, the pitch angle and roll angle of the first and second lidar coordinate systems to the reference coordinate system are obtained.

[0049] Furthermore, the step of calculating the translation parameters from the second lidar coordinate system to the first lidar coordinate system based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-lidar system, includes:

[0050] Based on the yaw angle, pitch angle and roll angle, the first lidar coordinate system is aligned with the reference coordinate system, the second lidar coordinate system is parallel to the reference coordinate system, and the rotation matrix from the second lidar coordinate system to the first lidar coordinate system is obtained.

[0051] Based on the rotation matrix from the second lidar coordinate system to the first lidar coordinate system, calculate the rotation angle θ of the second lidar, which is the rotation angle θ generated by the rotating bolt.

[0052] The translation parameters are calculated according to the following formula:

[0053] (tx, ty, tz) T =(R pitch1 R roll1 R yaw1 ) -1 (tx′,ty′,tz′)T

[0054] in:

[0055] tx′=0

[0056] ty′=l1+l2cosθ+(m+n)sinθ

[0057] tz′=m+n+l2sinθ-(m+n)cosθ

[0058] (tx, ty, tz) T The three-dimensional coordinates represent the three translation parameters from the second lidar coordinate system to the first lidar coordinate system; (tx′, ty′, tz′). T R represents the coordinates of the center point of the second lidar coordinate system in the reference coordinate system. pitch R represents a rotation matrix containing only the pitch angle pitch1; roll1 This represents a rotation matrix containing only the roll angle roll1; R yaw This represents a rotation matrix containing only the yaw angle yaw1; l1 represents the distance from the center of the first lidar base to the rotation axis; l2 represents the distance from the center of the second lidar base to the rotation axis; n represents the rotation radius of the rotating bolt; and m represents the height of the lidar center point from the base.

[0059] The second objective of this invention can be achieved by adopting the following technical solution:

[0060] A multi-laser radar extrinsic parameter self-calibration system is applied to a backpack multi-laser radar system. The radar system includes a first laser radar and a second laser radar. The first laser radar is fixedly placed horizontally, and the second laser radar rotates with the first laser radar via a rotating bolt. The rotation angle is adjustable. The system includes:

[0061] The acquisition unit is used to acquire first point cloud data and second point cloud data, wherein the first point cloud data is acquired by a first lidar and the second point cloud data is acquired by a second lidar.

[0062] A coordinate system is established using the centers of the first and second lidars, respectively.

[0063] A defining unit is used to define a reference coordinate system, wherein the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the center of the first lidar and the center of the second lidar, and the Z-axis is perpendicular to the ground.

[0064] The first calculation unit is used to calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data.

[0065] The second calculation unit is used to calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data, according to the yaw angle.

[0066] The third calculation unit is used to calculate the translation parameters from the second lidar coordinate system to the first lidar coordinate system based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-lidar system.

[0067] The third objective of this invention can be achieved by adopting the following technical solution:

[0068] A terminal device includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the above-described multi-laser radar extrinsic parameter self-calibration method.

[0069] The fourth objective of this invention can be achieved by adopting the following technical solution:

[0070] A storage medium storing a program, which, when executed by a processor, implements the above-described multi-laser radar extrinsic parameter self-calibration method.

[0071] The present invention has the following advantages over the prior art:

[0072] In this invention, the yaw angle between the second and first lidars is a fixed value. Each time the rotating bolt is turned to rotate the second lidar, the ground point cloud of both lidars is automatically extracted. Only the ground point cloud is needed to calculate the pitch and yaw angles, as well as three translation parameters, achieving calibration of the extrinsic parameters between multiple lidars. The calibration process is simple, fast, and yields accurate results. Furthermore, this invention is not limited by environmental conditions; only a consistent ground slope is required, making it suitable for calibration in various scenarios such as rooms, roads, and forests. This calibration method eliminates the need for calibration boards and manual calibration. The workflow begins simply by unfolding the backpack and rotating the second lidar, achieving automatic extraction of ground point clouds and high-precision automated calibration of the lidar while rotating the backpack. This improves work efficiency and solves the problems of high cost and complexity in existing calibration technologies. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0074] Figure 1This is a flowchart illustrating the multi-laser radar extrinsic parameter self-calibration method of Embodiment 1 of the present invention.

[0075] Figure 2 This is a simplified flowchart of the multi-lidar extrinsic parameter self-calibration method of Embodiment 1 of the present invention.

[0076] Figure 3 This is a schematic diagram of the placement of the backpack-mounted multi-laser radar during data acquisition in Embodiment 1 of the present invention.

[0077] Figure 4 This is a schematic diagram of the initial acquisition results of Embodiment 1 of the present invention.

[0078] Figure 5 This is an explanatory diagram illustrating the positional relationship of multiple lidar sensors when calculating rotational parameters in Embodiment 1 of the present invention.

[0079] Figure 6 This is an explanatory diagram illustrating the positional relationship of multiple lidar sensors when calculating translation parameters in Embodiment 1 of the present invention.

[0080] Figure 7 This is a schematic diagram of the final calibration result of Embodiment 1 of the present invention.

[0081] Figure 8 This is a structural block diagram of the multi-laser radar extrinsic parameter self-calibration system of Embodiment 2 of the present invention.

[0082] Figure 9 This is a structural block diagram of the computer device according to Embodiment 3 of the present invention.

[0083] Figure 3 In the diagram, L1 is the first lidar, S is the rotating bolt, and L2 is the second lidar that rotates by rotating the rotating bolt.

[0084] Figure 4 In the image, P1 and P2 are two point cloud frames simultaneously acquired by the first and second lidar sensors, respectively.

[0085] Figure 5 In the diagram, l1 is the distance from the center of the first lidar base to the pivot, l2 is the distance from the center of the second lidar base to the pivot, n is the pivot radius of the rotating bolt, and m is the height of the lidar center point from the base. All of these are fixed values ​​when the backpack lidar is shipped from the factory.

[0086] Figure 7 In the diagram, P1 is the point cloud acquired by the first lidar, and P2 is the point cloud acquired by the second lidar, which is then transformed into the coordinate system of the first lidar after calibration. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0088] Example 1:

[0089] like Figure 1 and Figure 2 As shown, this embodiment provides a self-calibration method for extrinsic parameters of multiple lidar systems, which is applied to a backpack-based multiple lidar system.

[0090] like Figure 3 As shown, the backpack rotating multi-laser radar (backpack multi-laser radar system) mainly includes a first laser radar L1, a second laser radar L2, and a rotating bolt S; the first laser radar is fixed horizontally, and the second laser radar rotates with the first laser radar through the rotating bolt. The rotation angle can be adjusted until the field of view of the two laser radars meets the working requirements, and the second laser radar cannot block the scanning rays of the first laser radar.

[0091] The multi-lidar extrinsic parameter self-calibration method in this embodiment includes the following steps:

[0092] S101. Acquire first point cloud data and second point cloud data, wherein the first point cloud data is collected by a first lidar and the second point cloud data is collected by a second lidar.

[0093] Specifically, acquire a frame of point cloud data collected by the first lidar at the same time. A frame of point cloud data acquired by the second lidar refer to Figure 4 .

[0094] S102. Establish coordinate systems with the centers of the first and second lidars respectively.

[0095] In this embodiment, the first lidar is a horizontally placed lidar when the backpack is unfolded, and the coordinate system is C. L1 The second lidar is a lidar that tilts due to rotation relative to the first lidar when the rotating bolt is turned, with coordinate system C. L2 .

[0096] S103. Determine the reference coordinate system, where the origin is the center of the first lidar, and the X-axis is perpendicular to the line connecting the center of the first lidar and the center of the second lidar.

[0097] In this embodiment, the positive direction of the Y-axis points to the straight line connecting the center point of the second lidar to the center point of the first lidar, the X-axis is parallel to the ground and perpendicular to the Y-axis, and the Z-axis is perpendicular to the ground, forming a right-handed coordinate system.

[0098] S104. Based on the first point cloud data and the second point cloud data, calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system. (Refer to...) Figure 5 .

[0099] S1041. Based on the corresponding point cloud data, segment the corresponding wall point cloud.

[0100] In this step, a frame of point cloud data acquired by the first lidar is used. A frame of point cloud data acquired by the second lidar The wall point cloud is manually segmented to obtain the wall point cloud of the first lidar and the wall point cloud of the second lidar.

[0101] S1042. Calculate the longitudinal mean of the wall point cloud of the first lidar and the wall point cloud of the second lidar.

[0102] In this step, the vertical mean is the mean of the vertical distance, that is, the mean of the X-coordinate. and

[0103] S1043. Based on the longitudinal mean, calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system according to the angle compensation method.

[0104] S10431. Set the rotation matrix containing only the yaw angle, as follows:

[0105]

[0106]

[0107] Where yaw1 represents the yaw angle from the first lidar coordinate system to the reference coordinate system, yaw2 represents the yaw angle from the second lidar coordinate system to the reference coordinate system, and R... yaw1 R represents a rotation matrix containing only the yaw angle yaw1. yaw2 This represents a rotation matrix containing only the yaw angle yaw2.

[0108] S10432. Based on the rotation matrix, using angle compensation, calculate the first variance between each point in the corresponding wall point cloud and the corresponding longitudinal mean, and then construct a first optimization function, wherein the first optimization function is as follows:

[0109]

[0110]

[0111] in, Indicated based on R yaw1 The first optimization function constructed, Indicated based on R yaw2 The second optimization function constructed, This represents the longitudinal mean of the wall point cloud obtained from the first lidar. X represents the longitudinal mean of the wall point cloud from the second lidar. i Let X represent any point in the wall point cloud of the first lidar. j w1 represents any point in the wall point cloud of the second lidar, w2 represents the number of wall point clouds of the first lidar, and w2 represents the number of wall point clouds of the second lidar.

[0112] In this step, the variance of the longitudinal distance X of the wall point cloud is minimized by using angle compensation.

[0113] S10433. Based on the first optimization function, and according to the nonlinear least squares optimization method, the yaw angles from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system are obtained.

[0114] In this step, the Levenberg-Marquardt algorithm is used to solve the first optimization function, which yields the optimal values ​​of yaw1 and yaw2. This allows us to obtain the yaw angles from the first and second lidar coordinate systems to the reference coordinate system, as shown in the following equation:

[0115]

[0116]

[0117] It should be noted that the yaw angle yaw1 from the first lidar coordinate system to the reference coordinate system and the yaw angle yaw2 from the second lidar coordinate system to the reference coordinate system obtained by the above formula are fixed values. If the second lidar is rotated again, steps S1041-S1043 do not need to be executed again.

[0118] S105. Based on the first point cloud data and the second point cloud data, calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system according to the yaw angle.

[0119] S1051. Based on the first point cloud data and the second point cloud data, according to the yaw angle, extract the ground point cloud of the first lidar and the ground point cloud of the second lidar, and calculate the average altitude.

[0120] In this step, the average height is the average height distance, which is the average Z-coordinate. and

[0121] In this step, based on the first point cloud data and the second point cloud data, and according to the yaw angle, the ground point cloud of the first lidar and the ground point cloud of the second lidar are extracted, specifically including:

[0122] S21. Based on the corresponding yaw angle, rotate the corresponding point cloud data around the Z-axis to obtain the rotated first point cloud data and second point cloud data, wherein the Z-axis belongs to the reference coordinate system.

[0123] In this step, the initial calibration of the first lidar to the second lidar is achieved using the yaw angles of the first lidar to the reference coordinate system and the second lidar to the reference coordinate system. Specifically, this involves using the first point cloud data collected by the first lidar... Second point cloud data acquired by the second lidar Rotate the reference coordinate system around the Z-axis by angles yaw1 and yaw2 to obtain the initially calibrated point cloud. and These are also referred to as the first point cloud data and the second point cloud data after rotation, respectively.

[0124] Point cloud and As shown in the following formula:

[0125]

[0126]

[0127] S22. Fit multiple planes to the rotated first point cloud data and extract the segments. Compare the angle between the normal vector of the fitted plane and the Z-axis. Take the plane containing the normal vector that meets the angle threshold as the ground point cloud of the first lidar.

[0128] Specifically, for the first lidar, when deployed using a backpack, it is approximately parallel to the ground, and the RANSAC method (existing technology) is used to... Multiple planes are fitted and segmented, and the normal vectors of the fitted planes are recorded. Since the first LiDAR is approximately parallel to the ground when the backpack is deployed, if the angle between the normal vector of the plane and the Z-axis of the reference coordinate system is less than 15°, then the plane is a candidate ground. If there are multiple candidate grounds, the angles between their normal vectors and the Z-axis of the reference coordinate system are compared. If the angles are within 2°, the candidate grounds are merged. If the angles differ by more than 2°, the candidate ground with the smaller angle is the real ground automatically segmented by the first LiDAR.

[0129] S23. Fit multiple planes to the rotated second point cloud data and extract the segments. Compare the angle between the normal vector of the fitted plane and the Z-axis, and calculate the mean Z-coordinate of the fitted plane. The plane containing the normal vector that meets the angle threshold and has a smaller mean Z-coordinate is taken as the ground point cloud of the second lidar.

[0130] Specifically, for the second lidar, when deployed using a backpack, the rotation angle of the second lidar is between 30° and 90° due to the rotating bolts. The RANSAC method (existing technology) is used to... Multiple planes are fitted and segmented, and the normal vectors of the fitted planes are recorded. If the angle between the normal vector of a plane and the Z-axis of the reference coordinate system is greater than 30° and less than 90°, then the plane is a candidate ground. Due to the installation characteristics of the second lidar, outdoors, there is usually only one plane normal vector that meets this threshold. If indoors, in addition to the ground, the roof also meets this threshold. In this case, the mean Z-coordinate of the plane containing the normal vector that meets the above angle threshold is calculated. The mean Z-coordinate of the ground is less than that of the roof, so the candidate ground with the smaller mean Z-coordinate is the real ground automatically segmented by the second lidar.

[0131] S1052. Based on the aforementioned average height, and according to the angle compensation method, calculate the pitch and roll angles from the first and second lidar coordinate systems to the reference coordinate system. (Refer to...) Figure 5 .

[0132] S10521. Set the rotation matrix containing only pitch angle and roll angle, as follows:

[0133]

[0134]

[0135]

[0136]

[0137] Where roll1 and pitch1 are the roll and pitch angles from the first lidar coordinate system to the reference coordinate system, respectively; roll2 and pitch2 are the roll and pitch angles from the second lidar coordinate system to the reference coordinate system, respectively; R roll1 R represents a rotation matrix containing only the roll angle roll1. rool2 R represents a rotation matrix containing only the roll angle roll2. pitch1 R represents a rotation matrix containing only the pitch angle pitch1. pitch2 This represents a rotation matrix containing only the pitch angle pitch2.

[0138] S10522. Based on the rotation matrix, using angle compensation, calculate the second variance between each point in the corresponding ground point cloud and its corresponding mean height, and then construct a second optimization function, wherein the second optimization function is as follows:

[0139]

[0140]

[0141] in, Indicated based on R roll1 R pitch1 The first optimization function constructed, Indicated based on R roll2 R pitch2 The second optimization function constructed, This represents the average height of the ground point cloud from the first lidar sensor. Z represents the average height of the ground point cloud from the second lidar. i Z' represents any point in the ground point cloud of the first lidar. j ′ represents any point in the ground point cloud of the second lidar, g1 represents the number of ground point clouds of the first lidar, and g2 represents the number of ground point clouds of the second lidar.

[0142] In this step, the variance of the height distance Z of the ground point cloud is minimized by using angle compensation.

[0143] S10523. Based on the second optimization function, and according to the nonlinear least squares optimization method, the pitch angle and roll angle of the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system are obtained.

[0144] In this step, the Levenberg-Marquardt algorithm is used to solve the second optimization function, which yields the optimal values ​​of roll and pitch. This allows us to obtain the pitch and roll angles from the first and second lidar coordinate systems to the reference coordinate system, as shown in the following formula:

[0145]

[0146]

[0147] The pitch angle (roll1) and roll angle (pitch1) of the first lidar coordinate system to the reference coordinate system after initial calibration are obtained from the above formula, and the pitch angle (roll2) and roll angle (pitch2) of the second lidar coordinate system to the reference coordinate system after initial calibration are obtained from the above formula.

[0148] S106. Based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-laser radar system, calculate the translation parameters from the second laser radar coordinate system to the first laser radar coordinate system.

[0149] S1061. Based on the yaw angle, pitch angle and roll angle, the first lidar coordinate system is made to coincide with the reference coordinate system (transformation process), the second lidar coordinate system is made to be parallel with the reference coordinate system (transformation process), and the rotation matrix from the second lidar coordinate system to the first lidar coordinate system is obtained.

[0150] In this step, the transformation process is achieved using the following formula:

[0151]

[0152]

[0153] in, This represents the transformed first lidar coordinate system. This represents the transformed second lidar coordinate system, with all axes parallel to each other.

[0154] In this step, the rotation matrix from the second lidar coordinate system to the first lidar coordinate system is as follows:

[0155]

[0156] S1062. Calculate the rotation angle θ of the second lidar based on the rotation matrix from the second lidar coordinate system to the first lidar coordinate system.

[0157] In this step, the rotation angle θ of the second lidar is the angle between the Z-axis of the first lidar coordinate system and the Z-axis of the second lidar coordinate system, and it is also the rotation angle of the rotating bolt. Specifically, the direction vector on the Z-axis of the first lidar coordinate system is given as (0 0 1). T Then the Z-axis direction vector of the second lidar coordinate system is a rotation matrix. The last column vector. (Reference) Figure 6 The Z-axis direction vector of the second lidar coordinate system intersects with the vector (0 0 1). T The included angle is θ.

[0158] S1063. Calculate the translation parameters according to the following formula:

[0159] (tx, ty, tz) T =(R pitch1 R roll1 R yaw1 ) -1 (tx', ty', tz') T

[0160] in:

[0161] tx'=0

[0162] ty′=l1+l2cosθ+(m+n)sinθ

[0163] tz′=m+n+l2sinθ-(m+n)cosθ

[0164] (tx, ty, tz) T The three-dimensional coordinates represent the three translation parameters from the second lidar coordinate system to the first lidar coordinate system; (tx′, ty′, tz′). T R represents the coordinates of the center point of the second lidar coordinate system in the reference coordinate system. pitch R represents a rotation matrix containing only the pitch angle pitch1; roll1 This represents a rotation matrix containing only the roll angle roll1; R yaw This represents a rotation matrix containing only the yaw angle yaw1; l1 represents the distance from the center of the first lidar base to the rotation axis; l2 represents the distance from the center of the second lidar base to the rotation axis; n represents the rotation radius of the rotating bolt; and m represents the height of the lidar center point from the base.

[0165] In this step, the positive direction of the Y-axis of the reference coordinate system points to the straight line connecting the center point of the second lidar to the center point of the first lidar. Therefore, the X-coordinate tx′ of the center point of the second lidar is 0 in the reference coordinate system. In this backpack lidar system, the distance l1 from the center of the first lidar base to the pivot and the distance l2 from the center of the second lidar base to the pivot are both 6cm, the pivot radius n of the rotating bolt is 1cm, and the height m of the lidar center point from the base is 3.6cm. These are all fixed values ​​when the backpack lidar is manufactured. Figure 6 .

[0166] It is worth noting that this embodiment uses the known installation distance of the lidar and the calculated rotation angle θ generated by the rotating bolt to calculate the Y coordinates ty′ and tz′ of the center point of the second lidar in the reference coordinate system; this embodiment is based on the rotation matrix from the first lidar coordinate system to the reference coordinate system obtained above. And the coordinates (tx′, ty′, tz′) of the center point of the second lidar coordinate system in the reference coordinate system. T The three-dimensional coordinates (tx, ty, tz) of the center point of the second lidar coordinate system in the first lidar coordinate system are obtained by inverse calculation. T .

[0167] Using this method, each time the backpack is unfolded and the axis is rotated to rotate the second lidar, the coordinate transformation relationship between the second lidar and the first lidar can be easily obtained, i.e., the six degrees of freedom parameters between the two lidars. This allows the point cloud collected by the second lidar to be unified into the coordinate system of the first lidar. The final calibration results can be referenced. Figure 7 .

[0168] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the corresponding program can be stored in a computer-readable storage medium.

[0169] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the order of execution of the described steps may be changed. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0170] Example 2:

[0171] like Figure 8 As shown, this embodiment provides a multi-LiDAR extrinsic parameter self-calibration system applied to a backpack multi-LiDAR system. The LiDAR system includes a first LiDAR and a second LiDAR. The first LiDAR is fixed horizontally, and the second LiDAR rotates with the first LiDAR via a rotating bolt. The rotation angle is adjustable. The system includes an acquisition unit 801, an establishment unit 802, a determination unit 803, a first calculation unit 804, a second calculation unit 805, and a third calculation unit 806. The specific functions of each unit are as follows:

[0172] The acquisition unit 801 is used to acquire first point cloud data and second point cloud data, wherein the first point cloud data is acquired by a first lidar and the second point cloud data is acquired by a second lidar.

[0173] Establishment unit 802 is used to establish coordinate systems with the centers of the first lidar and the second lidar respectively;

[0174] The determining unit 803 is used to determine the reference coordinate system, wherein the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the center of the first lidar and the center of the second lidar, and the Z-axis is perpendicular to the ground.

[0175] The first calculation unit 804 is used to calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data.

[0176] The second calculation unit 805 is used to calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data, according to the yaw angle.

[0177] The third calculation unit 806 is used to calculate the translation parameters from the second lidar coordinate system to the first lidar coordinate system based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-lidar system.

[0178] Example 3:

[0179] like Figure 9 As shown, this embodiment provides a terminal device, which includes a processor 902, a memory, an input device 903, a display device 904, and a network interface 905 connected via a system bus 901. The processor 902 provides computing and control capabilities. The memory includes a non-volatile storage medium 906 and internal memory 907. The non-volatile storage medium 906 stores an operating system, computer programs, and a database. The internal memory 907 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 906. When the computer program is executed by the processor 902, it implements the multi-laser radar extrinsic parameter self-calibration method of Embodiment 1 above, as follows:

[0180] Acquire first point cloud data and second point cloud data, wherein the first point cloud data is collected by a first lidar and the second point cloud data is collected by a second lidar;

[0181] Establish coordinate systems with the centers of the first and second lidars respectively;

[0182] Establish a reference coordinate system, where the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the centers of the first and second lidars, and the Z-axis is perpendicular to the ground.

[0183] Based on the first point cloud data and the second point cloud data, calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system;

[0184] Based on the first point cloud data and the second point cloud data, and according to the yaw angle, calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system.

[0185] Based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-LiDAR system, calculate the translation parameters from the second LiDAR coordinate system to the first LiDAR coordinate system.

[0186] Example 4:

[0187] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, it implements the multi-LiDAR extrinsic parameter self-calibration method of Embodiment 1 above, as follows:

[0188] Acquire first point cloud data and second point cloud data, wherein the first point cloud data is collected by a first lidar and the second point cloud data is collected by a second lidar;

[0189] Establish coordinate systems with the centers of the first and second lidars respectively;

[0190] Establish a reference coordinate system, where the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the centers of the first and second lidars, and the Z-axis is perpendicular to the ground.

[0191] Based on the first point cloud data and the second point cloud data, calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system;

[0192] Based on the first point cloud data and the second point cloud data, and according to the yaw angle, calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system.

[0193] Based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-LiDAR system, calculate the translation parameters from the second LiDAR coordinate system to the first LiDAR coordinate system.

[0194] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0195] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0196] The computer-readable storage medium described above can be used to write computer programs for executing this embodiment in one or more programming languages ​​or combinations thereof. These programming languages ​​include object-oriented programming languages—such as Java, Python, and C++—and conventional procedural programming languages—such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0197] In summary, in this invention, the yaw angle between the second and first lidars is a fixed value. Each time the rotating bolt is turned to rotate the second lidar, the ground point cloud of both lidars is automatically extracted. Only the ground point cloud is needed to calculate the transformed pitch and yaw angles, as well as three translation parameters, achieving calibration of the extrinsic parameters between multiple lidars. The calibration process is simple, fast, and yields accurate results. Furthermore, this invention is not limited by environmental conditions; only a consistent ground slope is required, making it suitable for calibration in various scenarios such as rooms, roads, and forests. This calibration method eliminates the need for calibration boards and manual calibration. The workflow begins simply by unfolding the backpack and rotating the second lidar, achieving automatic extraction of ground point clouds and high-precision automated calibration of the lidar while rotating the backpack. This improves work efficiency and solves the problems of high cost and complexity in existing calibration technologies.

[0198] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A multi-lidar extrinsic calibration method, applied to a backpack multi-lidar system, the lidar system comprising a first lidar and a second lidar, characterized in that, The first lidar is fixed horizontally, and the second lidar rotates with the first lidar via a rotating bolt, the rotation angle of which is adjustable. The method includes: Acquire first point cloud data and second point cloud data, wherein the first point cloud data is collected by a first lidar and the second point cloud data is collected by a second lidar; Establish coordinate systems with the centers of the first and second lidars respectively; Establish a reference coordinate system, where the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the centers of the first and second lidars, and the Z-axis is perpendicular to the ground. Based on the first point cloud data and the second point cloud data, calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system; Based on the first point cloud data and the second point cloud data, and according to the yaw angle, calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system. Based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-LiDAR system, calculate the translation parameters from the second LiDAR coordinate system to the first LiDAR coordinate system. The calculation of the pitch and roll angles from the first and second lidar coordinate systems to the reference coordinate system based on the first and second point cloud data and the yaw angle includes: Based on the first point cloud data and the second point cloud data, according to the yaw angle, the ground point cloud of the first lidar and the ground point cloud of the second lidar are extracted, and the average altitude is calculated. Based on the average altitude, and according to the angle compensation method, the pitch and roll angles from the first and second lidar coordinate systems to the reference coordinate system are calculated; including: Set up a rotation matrix that contains only pitch and roll angles, as follows: in, These are the roll angle and pitch angle from the first lidar coordinate system to the reference coordinate system, respectively. These are the roll angle and pitch angle from the second lidar coordinate system to the reference coordinate system, respectively. Indicates that only the roll angle is included. The rotation matrix, Indicates that only the roll angle is included. The rotation matrix, Indicates only pitch angle The rotation matrix, Indicates only pitch angle rotation matrix; Based on the rotation matrix, the second variance between each point in the ground point cloud and its corresponding mean height is calculated using angle compensation. This is then used to construct a second optimization function, which is expressed as follows: in, Indicates based on The first optimization function constructed, Indicates based on The second optimization function constructed, This represents the average height of the ground point cloud from the first lidar sensor. This represents the average height of the ground point cloud from the second lidar. This represents any point in the ground point cloud of the first lidar. This represents any point in the ground point cloud of the second lidar. This indicates the number of ground point clouds represented by the first lidar. This indicates the number of ground point clouds represented by the second lidar sensor; Based on the second optimization function, and according to the nonlinear least squares optimization method, the pitch angle and roll angle of the first and second lidar coordinate systems to the reference coordinate system are obtained.

2. The method of claim 1, wherein, The step of calculating the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data includes: Based on the corresponding point cloud data, segment the corresponding wall point cloud and calculate the longitudinal mean of the wall point cloud; Based on the longitudinal mean, and according to the angle compensation method, the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system is calculated.

3. The method of claim 2, wherein, The step of calculating the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the longitudinal mean and according to the angle compensation method includes: Set the rotation matrix to contain only the yaw angle, as follows: wherein yaw1 represents the yaw angle of the first lidar coordinate system to the reference coordinate system, yaw2 represents the yaw angle of the second lidar coordinate system to the reference coordinate system, represents a rotation matrix containing only the yaw angle yaw1, represents a rotation matrix containing only the yaw angle yaw2; Based on the rotation matrix, and through angle compensation, the first variance between each point in the corresponding wall point cloud and its corresponding longitudinal mean is calculated, and then a first optimization function is constructed, wherein the first optimization function is as follows: in, Indicates based on The first optimization function constructed, Indicates based on The second optimization function constructed, This represents the longitudinal mean of the wall point cloud obtained from the first lidar. This represents the longitudinal mean of the wall point cloud from the second lidar. This represents any point in the wall point cloud of the first lidar. w1 represents any point in the wall point cloud of the second lidar, w2 represents the number of wall point clouds of the first lidar, and w2 represents the number of wall point clouds of the second lidar. Based on the first optimization function, and according to the nonlinear least squares optimization method, the yaw angles from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system are obtained.

4. The method according to claim 1, characterized in that, The step of extracting the ground point cloud of the first lidar and the ground point cloud of the second lidar based on the first point cloud data and the yaw angle includes: Based on the corresponding yaw angle, the corresponding point cloud data is rotated around the Z-axis to obtain the first and second point cloud data after rotation, wherein the Z-axis belongs to the reference coordinate system; Multiple planes are fitted to the rotated first point cloud data and the segments are extracted. The angle between the normal vector of the fitted plane and the Z-axis is compared. The plane containing the normal vector that meets the angle threshold is taken as the ground point cloud of the first lidar. Multiple planes are fitted to the rotated second point cloud data and the segments are extracted. The angle between the normal vector of the fitted plane and the Z-axis is compared, and the mean Z-coordinate of the fitted plane is calculated. The plane containing the normal vector that meets the angle threshold and has a smaller mean Z-coordinate is taken as the ground point cloud of the second lidar.

5. The method according to claim 1, characterized in that, The step of calculating the translation parameters from the second lidar coordinate system to the first lidar coordinate system based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-lidar system, includes: Based on the yaw angle, pitch angle and roll angle, the first lidar coordinate system is aligned with the reference coordinate system, the second lidar coordinate system is parallel to the reference coordinate system, and the rotation matrix from the second lidar coordinate system to the first lidar coordinate system is obtained. Calculate the rotation angle of the second lidar based on the rotation matrix from the second lidar coordinate system to the first lidar coordinate system. That is, the rotation angle produced by the rotating bolt. ; The translation parameters are calculated according to the following formula: in: The three-dimensional coordinates represent the three translation parameters from the second lidar coordinate system to the first lidar coordinate system; This indicates the coordinates of the center point of the second lidar coordinate system in the reference coordinate system; This represents a rotation matrix containing only the pitch angle pitch1; This represents a rotation matrix containing only the roll angle roll1; This represents a rotation matrix containing only the yaw angle yaw1; This indicates the distance from the center of the first lidar base to the pivot point; This indicates the distance from the center of the second lidar base to the pivot point; Indicates the radius of the rotating bolt's shaft; This indicates the height of the center point of the lidar from the base.

6. A multi-laser radar extrinsic parameter self-calibration system, used to implement the method described in any one of claims 1-5, applied to a backpack multi-laser radar system, wherein the radar system includes a first laser radar and a second laser radar, characterized in that, The first lidar is fixed horizontally, and the second lidar rotates with the first lidar via a rotating bolt; the rotation angle is adjustable. The system includes: The acquisition unit is used to acquire first point cloud data and second point cloud data, wherein the first point cloud data is acquired by a first lidar and the second point cloud data is acquired by a second lidar. A coordinate system is established using the centers of the first and second lidars, respectively. A defining unit is used to define a reference coordinate system, wherein the origin is the center of the first lidar, the X-axis is perpendicular to the line connecting the center of the first lidar and the center of the second lidar, and the Z-axis is perpendicular to the ground. The first calculation unit is used to calculate the yaw angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data. The second calculation unit is used to calculate the pitch angle and roll angle from the first lidar coordinate system and the second lidar coordinate system to the reference coordinate system based on the first point cloud data and the second point cloud data, according to the yaw angle. The third calculation unit is used to calculate the translation parameters from the second lidar coordinate system to the first lidar coordinate system based on the yaw angle, pitch angle, and roll angle, as well as the factory installation settings of the backpack multi-lidar system.

7. A terminal device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the method according to any one of claims 1-5.

8. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1-5.

Images