Point cloud calibration method based on laser radar and IMU (Inertial Measurement Unit) and related equipment

Abstract

The invention discloses a point cloud calibration method based on a laser radar and an IMU (Inertial Measurement Unit) and related equipment. The method comprises the following steps: extracting each frame of ground point cloud of pre-acquired laser radar point cloud data; a plane corresponding to the IMU is set, and for each frame of ground point cloud, the vertical translation amount of the frame of ground point cloud relative to the plane in the vertical direction, the roll angle of the frame of ground point cloud in the rotation direction around the X axis and the pitch angle of the frame of ground point cloud in the rotation direction around the Y axis are calculated; transforming each frame of ground point cloud according to the vertical translation amount, the roll angle and the pitch angle to obtain a first point cloud; fitting the first point cloud, and rotating the first point cloud to a preset perfect plane to obtain a second point cloud; and calculating a yaw angle of the second point cloud relative to the plane in the rotation direction around the Z axis, a first translation amount in the horizontal X-axis direction and a second translation amount in the horizontal Y-axis direction, thereby performing calibration. According to the invention, accurate calibration can be realized, accurate point cloud data can be obtained, and subsequent target analysis is facilitated.

Description

Technical Field

[0001] This application relates to the field of point cloud calibration technology, specifically to a point cloud calibration method and related equipment based on lidar and IMU. Background Technology

[0002] IMUs are typically installed on the target's chassis or inside the vehicle, making it difficult to obtain the target's precise position. However, they are crucial in the field of autonomous driving, so high-precision automatic calibration is very important. IMUs generally output displacement information of the target itself, such as rotation angles, so their single-frame accuracy is relatively limited.

[0003] Currently, the conversion between lidar and IMU typically relies on manual subjective adjustment and conversion, which has very low accuracy. Especially when it comes to fine precision, it is difficult for the human eye to judge, which is not conducive to obtaining accurate point cloud data of the target and affects subsequent analysis of the target. Summary of the Invention

[0004] In view of this, this application provides a point cloud calibration method and related equipment based on lidar and IMU, which solves the problem that the current method usually relies on manual subjective adjustment and conversion, which has very low accuracy. Especially when it comes to fine precision, it is difficult for the human eye to judge, which is not conducive to obtaining accurate point cloud data of the target and affects the subsequent analysis of the target.

[0005] To achieve the above objectives, the following solution is proposed:

[0006] Firstly, a point cloud calibration method based on lidar and IMU includes:

[0007] Extract each frame of ground point cloud from the pre-acquired lidar point cloud data;

[0008] Define a plane corresponding to the IMU, and for each frame of ground point cloud, calculate the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis.

[0009] The ground point cloud in each frame is transformed according to the vertical translation, roll angle and pitch angle to obtain the first point cloud;

[0010] The first point cloud is fitted and rotated to a preset perfect plane to obtain the second point cloud;

[0011] Calculate the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction;

[0012] The lidar data is calibrated based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud.

[0013] Preferably, calculating the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction about the X-axis, and the pitch angle in the rotation direction about the Y-axis includes:

[0014] For each of the vertical direction, the rotation direction around the X-axis, and the rotation direction around the Y-axis, a first initial value is set in that direction;

[0015] Calculate the first distance from each point in the ground point cloud of this frame to the plane;

[0016] The overall thickness in this direction is calculated based on the first spacing and the first initial value;

[0017] Optimize in this direction to calculate the optimal thickness;

[0018] Determine the vertical translation amount corresponding to the optimal thickness in the vertical direction, the roll angle corresponding to the optimal thickness in the rotation direction around the X-axis, and the pitch angle corresponding to the optimal thickness in the rotation direction around the Y-axis.

[0019] Preferably, the optimization in this direction to calculate the optimal thickness includes:

[0020] Set the first initial step size;

[0021] The thickness is adjusted multiple times in this direction according to the first initial value and the first initial step size to calculate the comprehensive thickness after each adjustment, and the optimal thickness is determined from these adjustments.

[0022] Preferably, the step of making multiple adjustments in this direction according to the first initial value and the first initial step size to calculate the comprehensive thickness corresponding to each adjustment, and determining the optimal thickness therefrom, includes:

[0023] Define the first positive direction and the first negative direction for this direction;

[0024] Adjustments are made in the first positive direction according to the first initial value and the first initial step size;

[0025] After each adjustment, the overall thickness is recalculated and compared with the previously calculated overall thickness.

[0026] If the current calculated overall thickness is less than the previous calculated overall thickness, then continue to adjust in the first positive direction;

[0027] If the current calculated overall thickness is greater than the previous calculated overall thickness, then the first initial step size is reduced, and in the next calculation, the first initial step size is adjusted in the opposite direction according to the first initial value and the reduced first initial step size until the first initial step size is reduced to a preset step size threshold. The smallest of the obtained overall thicknesses is taken as the optimal thickness.

[0028] Preferably, calculating the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction includes:

[0029] For each of the rotational direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction, a second initial value is set in that direction;

[0030] Calculate the second distance from each point in the second point cloud to the plane;

[0031] Calculate the total area in this direction based on the second spacing and the second initial value;

[0032] Optimize in this direction to calculate the optimal area;

[0033] Determine the yaw angle corresponding to the optimal area in the rotation direction around the Z-axis, the first translation amount corresponding to the optimal area in the horizontal X-axis direction, and the second translation amount corresponding to the optimal area in the horizontal Y-axis direction.

[0034] Preferably, the optimization in this direction to calculate the optimal area includes:

[0035] Set a second initial step size;

[0036] The area is adjusted multiple times in this direction according to the second initial value and the second initial step size to calculate the comprehensive area corresponding to each adjustment, and the optimal area is determined from it.

[0037] Preferably, the step of making multiple adjustments in this direction according to the second initial value and the second initial step size to calculate the comprehensive area corresponding to each adjustment, and determining the optimal area from there, includes:

[0038] Define the second positive direction and the second negative direction of this direction;

[0039] Adjustments are made in the second positive direction according to the second initial value and the second initial step size;

[0040] After each adjustment, the total area is recalculated and compared with the total area calculated previously.

[0041] If the total area calculated in the current step is less than the total area calculated in the previous step, then continue to adjust in the second positive direction;

[0042] If the current calculated area is greater than the previous calculated area, the second initial step size is reduced, and in the next calculation, the second initial step size is adjusted in the opposite direction according to the second initial value and the reduced second initial step size until the second initial step size is reduced to a preset step size threshold. The smallest of the resulting areas is then taken as the optimal area.

[0043] Secondly, a point cloud calibration device based on lidar and IMU includes:

[0044] The ground point cloud extraction module is used to extract each frame of ground point cloud from the pre-acquired lidar point cloud data;

[0045] The first relative calculation module is used to set the plane corresponding to the IMU, and for each frame of ground point cloud, calculates the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis.

[0046] The transformation module is used to transform the ground point cloud in each frame according to the vertical translation, roll angle and pitch angle to obtain the first point cloud;

[0047] The second point cloud determination module is used to fit the first point cloud and rotate the first point cloud to a preset perfect plane to obtain the second point cloud.

[0048] The second relative calculation module is used to calculate the yaw angle of the second point cloud relative to the plane in the rotation direction around the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction;

[0049] The calibration module is used to calibrate the lidar data based on the vertical translation, roll angle, pitch angle, yaw angle, first translation amount, and second translation amount of each frame of ground point cloud.

[0050] Thirdly, a point cloud calibration device based on lidar and IMU, including a memory and a processor;

[0051] The memory is used to store programs;

[0052] The processor is configured to execute the program to implement the various steps of the point cloud calibration method based on lidar and IMU as described in any of the first aspects.

[0053] Fourthly, a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the point cloud calibration method based on lidar and IMU as described in any of the first aspects.

[0054] As can be seen from the above technical solution, this application extracts each frame of ground point cloud data from pre-acquired lidar point cloud data; sets a plane corresponding to the IMU; for each frame of ground point cloud, calculates the vertical translation, roll angle, and pitch angle of the ground point cloud relative to the plane in the vertical direction, the roll angle around the X-axis, and the pitch angle around the Y-axis; transforms each frame of ground point cloud according to the vertical translation, roll angle, and pitch angle to obtain a first point cloud; fits the first point cloud and rotates it to a preset perfect plane to obtain a second point cloud; calculates the yaw angle, first translation in the horizontal X-axis direction, and second translation in the horizontal Y-axis direction of the second point cloud relative to the plane; and calibrates the lidar data based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud. This application calculates relative translation or angle information in the vertical direction, the rotation direction around the X-axis, and the rotation direction around the Y-axis by setting a plane corresponding to the IMU, and then transforms it to obtain a first point cloud. Then, the first point cloud is transformed on a perfect plane to form a second point cloud. Then, relative translation or angle information is calculated in the rotation direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction for the second point cloud to determine the offset data required for calibrating the lidar point cloud, thereby achieving accurate calibration. Compared with existing methods, it can obtain accurate point cloud data, which is beneficial for subsequent target analysis. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 An optional flowchart of a point cloud calibration method based on lidar and IMU provided for an embodiment of this application;

[0057] Figure 2 A schematic diagram of a point cloud calibration device based on lidar and IMU provided for embodiments of this application;

[0058] Figure 3This is a schematic diagram of a point cloud calibration device based on lidar and IMU, provided for an embodiment of this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] L2IMU refers to the unification of the lidar coordinate system and the IMU coordinate system (inertial measurement unit coordinate system). IMUs are typically installed on the target's chassis or internally, making it difficult to obtain the target's precise position. However, they are crucial in the field of autonomous driving, so high-precision automatic calibration is essential. In autonomous driving, IMUs generally output displacement information such as rotation angles, thus limiting their single-frame accuracy. L2IMU calibration typically relies on positioning information, which in turn depends on the positioning point of reference (POS) output to complete full point cloud imaging. The classic manual calibration method is the figure-eight method, where the target vehicle travels back and forth along an S-shaped path, and the resulting point cloud data is stored in real-time. Motion compensation is applied to the point cloud at each timestamp, and the L2IMU transformation relationship is manually adjusted to minimize the fused volume of the full point cloud. The smallest POS is the final result. This method relies on subjective adjustments and transformations by humans, resulting in low accuracy, especially at finer precision levels where the human eye struggles to judge the difference, hindering the acquisition of accurate point cloud data and impacting subsequent target analysis.

[0061] Therefore, to overcome the shortcomings of the prior art, this invention provides a point cloud calibration method based on lidar and IMU. This method can be applied to various computer terminals or smart terminals, and its execution entity can be the processor or server of the computer terminal or smart terminal. This method can also be used in many general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0062] The flowchart of the method is as follows: Figure 1 As shown, it specifically includes:

[0063] S1: Extract each frame of ground point cloud from the pre-acquired lidar point cloud data.

[0064] In this application, considering that ground point clouds usually have planar features and high stability, they can be selected as a reference benchmark for calibration. Ground point clouds can be extracted by filtering or deep learning methods, thereby reducing interference from obstacles (such as other vehicles, people, other objects, etc.), improving calibration accuracy, and enhancing calibration effect.

[0065] Among these methods, filtering can be achieved using plane fitting methods, while deep learning methods can include semantic segmentation, etc.

[0066] S2: Set the plane corresponding to the IMU, and for each frame of ground point cloud, calculate the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis.

[0067] This step involves offset data in the vertical direction (Z-axis direction), the rotation direction around the axis (Roll), and the rotation direction around the Y-axis (Pitch). The vertical translation reflects the offset between the lidar point cloud data and the IMU coordinate system in the Z-axis direction, the roll angle reflects the rotational deviation between the lidar point cloud data and the IMU coordinate system in the rotation direction around the X-axis, and the pitch angle reflects the pitch angle between the lidar point cloud data and the IMU coordinate system in the rotation direction around the Y-axis.

[0068] S3: Transform the ground point cloud in each frame according to the vertical translation, roll angle and pitch angle to obtain the first point cloud.

[0069] Once the vertical translation, roll angle, and pitch angle are obtained, the ground point cloud in each frame of the lidar point cloud data can be initially aligned to the IMU coordinate system based on these data. This can eliminate the height and tilt errors between the lidar point cloud data and the IMU, and retain the residuals in the horizontal plane for subsequent processing. This distributed calculation can improve the calibration accuracy and calculation accuracy.

[0070] S4: Fit the first point cloud and rotate the first point cloud to a preset perfect plane to obtain the second point cloud.

[0071] Fitting the first point cloud can optimize the plane parameters and reduce the impact of noise. Then, rotating it to a preset perfect plane can ensure that it is completely parallel to the IMU coordinate system.

[0072] For a plane equation In general, a perfect plane refers to the plane where the γ parameter is 0.

[0073] S5: Calculate the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction.

[0074] This step involves offset data in the rotation direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction. The yaw angle in the rotation direction around the Z-axis can reflect the rotational deviation between the lidar point cloud data and the IMU coordinate system in the Z-axis direction. The first translation amount can reflect the horizontal deviation between the lidar point cloud data and the IMU coordinate system in the horizontal X-axis direction, and the second translation amount can reflect the horizontal deviation between the lidar point cloud data and the IMU coordinate system in the horizontal Y-axis direction.

[0075] S6: The lidar data is calibrated based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud.

[0076] After obtaining the above data, point cloud registration and alignment can be performed. By combining all the data, the calibrated lidar point cloud data can be accurately converted to the IMU coordinate system for subsequent precise target analysis and evaluation, applicable to various scenarios.

[0077] As can be seen from the above technical solution, this application extracts each frame of ground point cloud data from pre-acquired lidar point cloud data; sets a plane corresponding to the IMU; for each frame of ground point cloud, calculates the vertical translation, roll angle, and pitch angle of the ground point cloud relative to the plane in the vertical direction, the roll angle around the X-axis, and the pitch angle around the Y-axis; transforms each frame of ground point cloud according to the vertical translation, roll angle, and pitch angle to obtain a first point cloud; fits the first point cloud and rotates it to a preset perfect plane to obtain a second point cloud; calculates the yaw angle, first translation in the horizontal X-axis direction, and second translation in the horizontal Y-axis direction of the second point cloud relative to the plane; and calibrates the lidar data based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud. This application calculates relative translation or angle information in the vertical direction, the rotation direction around the X-axis, and the rotation direction around the Y-axis by setting a plane corresponding to the IMU, and then transforms it to obtain a first point cloud. Then, the first point cloud is transformed on a perfect plane to form a second point cloud. Then, relative translation or angle information is calculated in the rotation direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction for the second point cloud to determine the offset data required for calibrating the lidar point cloud, thereby achieving accurate calibration. Compared with existing methods, it can obtain accurate point cloud data, which is beneficial for subsequent target analysis.

[0078] In addition, this application can achieve automatic calibration without relying on manual subjective adjustments and conversions, thus avoiding problems such as high error rates caused by human intervention.

[0079] The method provided in this embodiment of the invention includes a process for calculating the vertical translation of the ground point cloud frame relative to the plane in the vertical direction, the roll angle in the rotation direction about the X-axis, and the pitch angle in the rotation direction about the Y-axis. The specific steps are as follows:

[0080] For each of the vertical direction, the rotation direction around the X-axis, and the rotation direction around the Y-axis, a first initial value is set in that direction;

[0081] Calculate the first distance from each point in the ground point cloud of this frame to the plane;

[0082] The overall thickness in this direction is calculated based on the first spacing and the first initial value;

[0083] Optimize in this direction to calculate the optimal thickness;

[0084] Determine the vertical translation amount corresponding to the optimal thickness in the vertical direction, the roll angle corresponding to the optimal thickness in the rotation direction around the X-axis, and the pitch angle corresponding to the optimal thickness in the rotation direction around the Y-axis.

[0085] Specifically, the above process adopts independent optimization in each direction, which can improve the optimization accuracy and focus on small and precise optimization. In the vertical direction, the height difference between the LiDAR point cloud data and the IMU coordinate system is optimized; in the X-axis direction, the left and right tilt of the LiDAR point cloud data is optimized; and in the Y-axis direction, the forward and backward tilt of the LiDAR data is optimized. Thus, geographic optimization in each direction simplifies the high-dimensional nonlinear optimization problem.

[0086] When calculating the overall thickness, the initial plane equation can be fitted first. Then, the distance from each point in the ground point cloud of that frame to the plane is calculated as the first spacing. :

[0087] ;

[0088] Then calculate the overall thickness based on the first spacing. :

[0089] ;

[0090] To achieve high-precision calibration, the optimal thickness needs to be determined, requiring optimization in every direction to calculate the optimal thickness. The vertical translation, roll angle, and pitch angle corresponding to the optimal thickness are the optimal calibration data.

[0091] The process of optimizing in this direction to calculate the optimal thickness will be explained in detail below.

[0092] Set the first initial step size;

[0093] The thickness is adjusted multiple times in this direction according to the first initial value and the first initial step size to calculate the comprehensive thickness after each adjustment, and the optimal thickness is determined from these adjustments.

[0094] The step of making multiple adjustments in this direction according to the first initial value and the first initial step size to calculate the comprehensive thickness corresponding to each adjustment and to determine the optimal thickness is explained in detail below.

[0095] Define the first positive direction and the first negative direction for this direction;

[0096] Adjustments are made in the first positive direction according to the first initial value and the first initial step size;

[0097] After each adjustment, the overall thickness is recalculated and compared with the previously calculated overall thickness.

[0098] If the current calculated overall thickness is less than the previous calculated overall thickness, then continue to adjust in the first positive direction;

[0099] If the current calculated overall thickness is greater than the previous calculated overall thickness, then the first initial step size is reduced, and in the next calculation, the first initial step size is adjusted in the opposite direction according to the first initial value and the reduced first initial step size until the first initial step size is reduced to a preset step size threshold. The smallest of the obtained overall thicknesses is taken as the optimal thickness.

[0100] Specifically, the above process balances accuracy and robustness through adaptive step size adjustment and forward and reverse adjustments in each direction, greatly improving calibration efficiency.

[0101] If the overall thickness has multiple minimum values, the number of iterations can be limited to find the optimal thickness. The optimal thickness directly reflects the alignment between the point cloud and the plane, and the optimization objective is clear.

[0102] Furthermore, the process of calculating the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction may include the following steps:

[0103] For each of the rotational direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction, a second initial value is set in that direction;

[0104] Calculate the second distance from each point in the second point cloud to the plane;

[0105] Calculate the total area in this direction based on the second spacing and the second initial value;

[0106] Optimize in this direction to calculate the optimal area;

[0107] Determine the yaw angle corresponding to the optimal area in the rotation direction around the Z-axis, the first translation amount corresponding to the optimal area in the horizontal X-axis direction, and the second translation amount corresponding to the optimal area in the horizontal Y-axis direction.

[0108] Specifically, similar to the process in this application of determining the vertical translation corresponding to the optimal thickness in the vertical direction, the roll angle corresponding to the optimal thickness in the rotation direction around the X-axis, and the pitch angle corresponding to the optimal thickness in the rotation direction around the Y-axis, this step also optimizes the other three directions independently: the rotation direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction. The yaw angle is used to correct the horizontal orientation deviation between the lidar point cloud data and the IMU coordinate system, the first translation amount is used to compensate for the X-direction offset between the lidar point cloud data and the IMU on the horizontal plane, and the second translation amount is used to compensate for the Y-direction offset between the lidar point cloud data and the IMU on the horizontal plane. This reduces the complexity of the problem and avoids the problem of local optima and global poor performance caused by the simultaneous deviation or offset calculation of multiple parameters.

[0109] The specific calculation method for optimizing this direction in the above process to calculate the optimal area can be referred to as follows:

[0110] Set a second initial step size;

[0111] The area is adjusted multiple times in this direction according to the second initial value and the second initial step size to calculate the comprehensive area corresponding to each adjustment, and the optimal area is determined from it.

[0112] Optionally, the steps for making multiple adjustments in this direction according to the second initial value and the second initial step size to calculate the comprehensive area corresponding to each adjustment, and determining the optimal area from them, are as follows:

[0113] Define the second positive direction and the second negative direction of this direction;

[0114] Adjustments are made in the second positive direction according to the second initial value and the second initial step size;

[0115] After each adjustment, the total area is recalculated and compared with the total area calculated previously.

[0116] If the total area calculated in the current step is less than the total area calculated in the previous step, then continue to adjust in the second positive direction;

[0117] If the current calculated area is greater than the previous calculated area, the second initial step size is reduced, and in the next calculation, the second initial step size is adjusted in the opposite direction according to the second initial value and the reduced second initial step size until the second initial step size is reduced to a preset step size threshold. The smallest of the resulting areas is then taken as the optimal area.

[0118] Specifically, similar to the process described above, by adaptive step size adjustment and forward and reverse adjustments in each of the rotational directions around the Z-axis, the horizontal X-axis, and the horizontal Y-axis, accuracy and robustness are balanced, improving calibration efficiency. Optimal data calibration in these three directions refers to the overall area, reflecting the degree of fit between the LiDAR point cloud data and the plane. Minimizing this area is equivalent to maximizing the alignment accuracy, achieving high-precision calibration of the LiDAR point cloud data relative to the IMU in the horizontal direction with high reliability.

[0119] and Figure 1 Corresponding to the method described above, this embodiment of the invention also provides a point cloud calibration device based on lidar and IMU, used for calibration of... Figure 1 In the specific implementation of the method, the point cloud calibration device based on lidar and IMU provided in this embodiment of the invention can be used in computer terminals or various mobile devices, combined with Figure 2 This paper introduces a point cloud calibration device based on lidar and IMU, such as... Figure 2 As shown, the device may include:

[0120] The ground point cloud extraction module 10 is used to extract each frame of ground point cloud from the pre-acquired lidar point cloud data.

[0121] The first relative calculation module 20 is used to set the plane corresponding to the IMU, and for each frame of ground point cloud, calculate the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis.

[0122] Transformation module 30 is used to transform the ground point cloud in each frame according to the vertical translation, roll angle and pitch angle to obtain the first point cloud;

[0123] The second point cloud determination module 40 is used to fit the first point cloud and rotate the first point cloud to a preset perfect plane to obtain the second point cloud.

[0124] The second relative calculation module 50 is used to calculate the yaw angle of the second point cloud relative to the plane in the rotation direction around the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction;

[0125] The calibration module 60 is used to calibrate the lidar data based on the vertical translation, roll angle, pitch angle, yaw angle, first translation amount, and second translation amount of each frame of ground point cloud.

[0126] As can be seen from the above technical solution, this application extracts each frame of ground point cloud data from pre-acquired lidar point cloud data; sets a plane corresponding to the IMU; for each frame of ground point cloud, calculates the vertical translation, roll angle, and pitch angle of the ground point cloud relative to the plane in the vertical direction, the roll angle around the X-axis, and the pitch angle around the Y-axis; transforms each frame of ground point cloud according to the vertical translation, roll angle, and pitch angle to obtain a first point cloud; fits the first point cloud and rotates it to a preset perfect plane to obtain a second point cloud; calculates the yaw angle, first translation in the horizontal X-axis direction, and second translation in the horizontal Y-axis direction of the second point cloud relative to the plane; and calibrates the lidar data based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud. This application calculates relative translation or angle information in the vertical direction, the rotation direction around the X-axis, and the rotation direction around the Y-axis by setting a plane corresponding to the IMU, and then transforms it to obtain a first point cloud. Then, the first point cloud is transformed on a perfect plane to form a second point cloud. Then, relative translation or angle information is calculated in the rotation direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction for the second point cloud to determine the offset data required for calibrating the lidar point cloud, thereby achieving accurate calibration. Compared with existing methods, it can obtain accurate point cloud data, which is beneficial for subsequent target analysis.

[0127] Furthermore, embodiments of this application provide a point cloud calibration device based on lidar and IMU. Optionally, Figure 3 The hardware structure block diagram of the point cloud calibration device based on lidar and IMU is shown. (Refer to...) Figure 3 The hardware structure of a point cloud calibration device based on lidar and IMU may include: at least one processor 01, at least one communication interface 02, at least one memory 03 and at least one communication bus 04.

[0128] In this embodiment of the application, the number of processor 01, communication interface 02, memory 03 and communication bus 04 is at least one, and processor 01, communication interface 02 and memory 03 communicate with each other through communication bus 04.

[0129] Processor 01 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0130] Memory 03 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0131] The memory stores a program that the processor can call. The program executes the following point cloud calibration method based on lidar and IMU, including:

[0132] Extract each frame of ground point cloud from the pre-acquired lidar point cloud data;

[0133] Define a plane corresponding to the IMU, and for each frame of ground point cloud, calculate the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis.

[0134] The ground point cloud in each frame is transformed according to the vertical translation, roll angle and pitch angle to obtain the first point cloud;

[0135] The first point cloud is fitted and rotated to a preset perfect plane to obtain the second point cloud;

[0136] Calculate the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction;

[0137] The lidar data is calibrated based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud.

[0138] Optionally, the refined and extended functions of the program can be found in the description of the point cloud calibration method based on lidar and IMU in the method embodiments.

[0139] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor. When the program runs, it controls the device where the storage medium is located to execute the following point cloud calibration method based on lidar and IMU, including:

[0140] Extract each frame of ground point cloud from the pre-acquired lidar point cloud data;

[0141] Define a plane corresponding to the IMU, and for each frame of ground point cloud, calculate the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis.

[0142] The ground point cloud in each frame is transformed according to the vertical translation, roll angle and pitch angle to obtain the first point cloud;

[0143] The first point cloud is fitted and rotated to a preset perfect plane to obtain the second point cloud;

[0144] Calculate the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction;

[0145] The lidar data is calibrated based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud.

[0146] Specifically, the storage medium can be a computer-readable storage medium, which can be an electronic storage device such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM.

[0147] Optionally, the refined and extended functions of the program can be found in the description of the point cloud calibration method based on lidar and IMU in the method embodiments.

[0148] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a live streaming device, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this disclosure.

[0149] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0151] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A point cloud calibration method based on lidar and IMU, characterized in that, include: Extract each frame of ground point cloud from the pre-acquired lidar point cloud data; Define a plane corresponding to the IMU, and for each frame of ground point cloud, calculate the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis. The ground point cloud in each frame is transformed according to the vertical translation, roll angle and pitch angle to obtain the first point cloud; The first point cloud is fitted and rotated to a preset perfect plane to obtain the second point cloud; Calculate the yaw angle of the second point cloud relative to the plane in the direction of rotation about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction; The lidar data is calibrated based on the vertical translation, roll angle, pitch angle, yaw angle, first translation, and second translation of each frame of ground point cloud.

2. The method according to claim 1, characterized in that, The calculation of the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction about the X-axis, and the pitch angle in the rotation direction about the Y-axis includes: For each of the vertical direction, the rotation direction around the X-axis, and the rotation direction around the Y-axis, a first initial value is set in that direction; Calculate the first distance from each point in the ground point cloud of this frame to the plane; The overall thickness in this direction is calculated based on the first spacing and the first initial value; Optimize in this direction to calculate the optimal thickness; Determine the vertical translation amount corresponding to the optimal thickness in the vertical direction, the roll angle corresponding to the optimal thickness in the rotation direction around the X-axis, and the pitch angle corresponding to the optimal thickness in the rotation direction around the Y-axis.

3. The method according to claim 2, characterized in that, The optimization in this direction to calculate the optimal thickness includes: Set the first initial step size; The thickness is adjusted multiple times in this direction according to the first initial value and the first initial step size to calculate the comprehensive thickness after each adjustment, and the optimal thickness is determined from these adjustments.

4. The method according to claim 3, characterized in that, The process of making multiple adjustments in this direction according to the first initial value and the first initial step size to calculate the comprehensive thickness corresponding to each adjustment, and determining the optimal thickness from these adjustments, includes: Define the first positive direction and the first negative direction for this direction; Adjustments are made in the first positive direction according to the first initial value and the first initial step size; After each adjustment, the overall thickness is recalculated and compared with the previously calculated overall thickness. If the current calculated overall thickness is less than the previous calculated overall thickness, then continue to adjust in the first positive direction; If the current calculated overall thickness is greater than the previous calculated overall thickness, then the first initial step size is reduced, and in the next calculation, the first initial step size is adjusted in the opposite direction according to the first initial value and the reduced first initial step size until the first initial step size is reduced to a preset step size threshold. The smallest of the obtained overall thicknesses is taken as the optimal thickness.

5. The method according to any one of claims 1 to 4, characterized in that, The calculation of the yaw angle of the second point cloud relative to the plane in the rotation direction about the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction includes: For each of the rotational direction around the Z-axis, the horizontal X-axis direction, and the horizontal Y-axis direction, a second initial value is set in that direction; Calculate the second distance from each point in the second point cloud to the plane; Calculate the total area in this direction based on the second spacing and the second initial value; Optimize in this direction to calculate the optimal area; Determine the yaw angle corresponding to the optimal area in the rotation direction around the Z-axis, the first translation amount corresponding to the optimal area in the horizontal X-axis direction, and the second translation amount corresponding to the optimal area in the horizontal Y-axis direction.

6. The method according to claim 5, characterized in that, The optimization in this direction to calculate the optimal area includes: Set a second initial step size; The area is adjusted multiple times in this direction according to the second initial value and the second initial step size to calculate the comprehensive area corresponding to each adjustment, and the optimal area is determined from it.

7. The method according to claim 6, characterized in that, The process of making multiple adjustments in this direction according to the second initial value and the second initial step size to calculate the comprehensive area corresponding to each adjustment, and determining the optimal area from these adjustments, includes: Define the second positive direction and the second negative direction of this direction; Adjustments are made in the second positive direction according to the second initial value and the second initial step size; After each adjustment, the total area is recalculated and compared with the total area calculated previously. If the total area calculated in the current step is less than the total area calculated in the previous step, then continue to adjust in the second positive direction; If the current calculated area is greater than the previous calculated area, the second initial step size is reduced, and in the next calculation, the second initial step size is adjusted in the opposite direction according to the second initial value and the reduced second initial step size until the second initial step size is reduced to a preset step size threshold. The smallest of the resulting areas is then taken as the optimal area.

8. A point cloud calibration device based on lidar and IMU, characterized in that, include: The ground point cloud extraction module is used to extract each frame of ground point cloud from the pre-acquired lidar point cloud data; The first relative calculation module is used to set the plane corresponding to the IMU, and for each frame of ground point cloud, calculates the vertical translation of the ground point cloud relative to the plane in the vertical direction, the roll angle in the rotation direction around the X-axis, and the pitch angle in the rotation direction around the Y-axis. The transformation module is used to transform the ground point cloud in each frame according to the vertical translation, roll angle and pitch angle to obtain the first point cloud; The second point cloud determination module is used to fit the first point cloud and rotate the first point cloud to a preset perfect plane to obtain the second point cloud. The second relative calculation module is used to calculate the yaw angle of the second point cloud relative to the plane in the rotation direction around the Z-axis, the first translation in the horizontal X-axis direction, and the second translation in the horizontal Y-axis direction; The calibration module is used to calibrate the lidar data based on the vertical translation, roll angle, pitch angle, yaw angle, first translation amount, and second translation amount of each frame of ground point cloud.

9. A point cloud calibration device based on lidar and IMU, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the point cloud calibration method based on lidar and IMU as described in any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the point cloud calibration method based on lidar and IMU as described in any one of claims 1-7.

Images