Laser radar and imu joint calibration method and device for mine

Abstract

The present disclosure relates to a laser radar and IMU joint calibration method and device for a mine, wherein the method comprises: acquiring laser point cloud data collected by the laser radar and IMU data collected by the IMU to calculate the attitude change information of the laser radar and the IMU, performing point cloud distortion calibration on the laser point cloud data based on the initial external parameters in the attitude change information to obtain target laser point cloud data; acquiring key frame point cloud data from the target laser point cloud data, constructing a target number group hand-eye calibration equation based on the key frame point cloud data and the attitude change information to determine a first residual error; performing clustering processing on the key frame point cloud data to obtain line feature points of each key frame point cloud and processing the line feature points to obtain a second residual error, determining a target optimization function based on the first residual error and the second residual error and solving the target optimization function to obtain target external parameters of the laser radar and the IMU. Thus, the external parameters of the laser radar and the IMU are automatically calibrated without specifying a marker, and the calibration accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a laser radar and IMU joint calibration method and device for a mine. BACKGROUND

[0002] In recent years, intelligent driving develops rapidly, a large number of sensors are installed on vehicles, the advantages of various sensors are combined to make up for the limitations of a single sensor, improve system performance, and realize real-time perception, positioning, planning and decision-making of the current environment. In the rapid development of intelligent driving, the concept of smart mine is also put forward. The mine environment is harsh, and the intelligent driving technology is migrated to the mine truck transportation scene, which can greatly reduce the accident rate.

[0003] Generally, the mine is located in a remote place, and the GPS (Global Positioning System) positioning signal is weak. High-precision positioning is the key to intelligent driving of the mine truck, which needs to be realized by multiple sensors. At present, laser radar and IMU (Inertial Measurement Unit) are mainly used for fusion positioning. The laser radar can obtain three-dimensional information in the environment in real time, and the IMU can obtain the displacement and rotation information of itself at a high frequency based on Newton's law. The premise of realizing accurate fusion of laser and IMU information is to know the pose relationship between the two, so it is necessary to calibrate the laser and IMU to obtain the pose relationship between the two.

[0004] In the related art, a calibration board is set, and then the data of the laser radar, the IMU and the camera are collected under the space of the calibration board. The extrinsic parameters between the laser radar and the IMU are obtained by constructing a graph optimization model. This method needs to specify a calibration object, and the operation process is relatively cumbersome, which has certain limitations. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a laser radar and IMU joint calibration method and device for a mine.

[0006] The present disclosure provides a laser radar and IMU joint calibration method for a mine, the relative position of the laser radar and the inertial measurement unit (IMU) remains unchanged, and the method comprises the following steps: acquiring laser point cloud data collected by the laser radar and IMU data collected by the IMU; calculating based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU; performing point cloud distortion calibration on the laser point cloud data based on initial external parameters in the attitude change information to obtain target laser point cloud data; acquiring key frame point cloud data from the target laser point cloud data, constructing a target number group hand-eye calibration equation based on the key frame point cloud data and the attitude change information, and determining a first residual based on the target number group hand-eye calibration equation; performing clustering processing on the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud, and performing processing based on the line feature points of each key frame point cloud to obtain a second residual; determining a target optimization function based on the first residual and the second residual, and solving the target optimization function based on a preset solving algorithm to obtain target external parameters of the laser radar and the IMU.

[0007] In an optional embodiment of the present disclosure, the processing based on the laser point cloud data and the IMU data to obtain the attitude change information of the laser radar and the IMU comprises: synchronizing the time stamps of the laser point cloud data and the IMU data, and calculating the pose change of each frame of laser point cloud in the laser point cloud data based on a normal distribution transformation registration algorithm to construct a pose group of the laser point cloud data; constructing a pose group of the IMU data based on an extended Kalman filter algorithm; calculating a pose transformation matrix of the laser radar and the IMU based on the pose group of the laser point cloud data and the pose group of the IMU data; and constructing a hand-eye calibration equation based on the pose transformation matrix and the initial external parameters of the laser radar and the IMU to obtain the attitude change information of the laser radar and the IMU.

[0008] In an optional embodiment of the present disclosure, the point cloud distortion calibration of the laser point cloud data based on the initial external parameters in the attitude change information to obtain the target laser point cloud data comprises: determining the rotation increment of each point relative to the frame header moment based on the initial external parameters and the laser point cloud data; and compensating each frame of laser point cloud based on the rotation increment to obtain the target laser point cloud data.

[0009] In an optional embodiment of the present disclosure, the acquisition of the key frame point cloud data from the target laser point cloud data comprises: after filtering the target laser point cloud data, acquiring a point cloud rotation angle corresponding to each frame of target laser point cloud; and taking the target laser point cloud with a point cloud rotation angle greater than a preset rotation angle threshold as a key frame point cloud to obtain the key frame point cloud data.

[0010] In optional embodiments of the present disclosure, the key frame point cloud data is obtained from the target laser point cloud data, including: after filtering the target laser point cloud data, a translation distance corresponding to each frame of target laser point cloud is obtained; the target laser point cloud with a translation distance greater than a preset distance threshold is taken as a key frame point cloud, and the key frame point cloud data is obtained.

[0011] In optional embodiments of the present disclosure, the second residual is obtained by processing the line feature points of each key frame point cloud, including: a line feature point cloud map is obtained by mapping the line feature points of each key frame point cloud; the line feature point cloud map is divided to obtain a plurality of point cloud clusters; a line feature representation of each point cloud cluster is obtained, a straight line distance formula of the laser point cloud of each point cloud cluster to the line feature representation is constructed, and a distance residual is determined based on the straight line distance formula as the second residual.

[0012] In optional embodiments of the present disclosure, the target optimization function is determined based on the first residual and the second residual, including: a first optimization function is constructed based on the first residual; a second optimization function is constructed based on the second residual; and the target optimization function is generated based on the first optimization function and the second optimization function.

[0013] The present disclosure provides a laser radar and IMU joint calibration device for a mine, the relative position of the laser radar and the inertial measurement unit (IMU) remains unchanged, the device comprises: an acquisition module for acquiring laser point cloud data collected by the laser radar and IMU data collected by the IMU; a calculation module for calculating based on the laser point cloud data and the IMU data to obtain the attitude change information of the laser radar and the IMU; a calibration module for performing point cloud distortion calibration on the laser point cloud data based on the initial external parameters in the attitude change information to obtain target laser point cloud data; an acquisition module for obtaining key frame point cloud data from the target laser point cloud data; a construction and determination module for constructing a target number group hand-eye calibration equation based on the key frame point cloud data and the attitude change information, and determining a first residual based on the target number group hand-eye calibration equation; a clustering module for clustering the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud; a processing module for processing the line feature points of each key frame point cloud to obtain a second residual; a determination and solving module for determining a target optimization function based on the first residual and the second residual, and solving the target optimization function based on a preset solving algorithm to obtain target external parameters of the laser radar and the IMU.

[0014] An electronic device is provided according to an embodiment of the present disclosure, and the electronic device includes a processor, a memory for storing executable instructions of the processor, and the processor is configured to read the executable instructions from the memory and execute the instructions to implement a method for joint calibration of a laser radar and an IMU for a mine according to an embodiment of the present disclosure.

[0015] A computer-readable storage medium is provided according to an embodiment of the present disclosure, and the storage medium stores a computer program for executing a method for joint calibration of a laser radar and an IMU for a mine according to an embodiment of the present disclosure.

[0016] An embodiment of the present disclosure further provides a computer program product comprising a computer program for executing a method for joint calibration of a laser radar and an IMU for a mine according to an embodiment of the present disclosure when executed by a processor.

[0017] The technical solution provided by the embodiments of the present disclosure has the following advantages compared with the prior art.

[0018] The present disclosure provides a joint calibration scheme for a laser radar and an IMU for a mine, and the relative position of the laser radar and the inertial measurement unit (IMU) remains unchanged. The method comprises: acquiring laser point cloud data collected by the laser radar and IMU data collected by the IMU; calculating based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU; performing point cloud distortion calibration on the laser point cloud data based on initial external parameters in the attitude change information to obtain target laser point cloud data; acquiring key frame point cloud data from the target laser point cloud data, constructing a target number group hand-eye calibration equation based on the key frame point cloud data and the attitude change information, and determining a first residual based on the target number group hand-eye calibration equation; performing clustering processing on the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud, and performing processing based on the line feature points of each key frame point cloud to obtain a second residual; determining a target optimization function based on the first residual and the second residual, and solving the target optimization function based on a preset solving algorithm to obtain target external parameters of the laser radar and the IMU. Thus, the technical problems of relying on developing markers in the laser radar and IMU calibration process in the prior art and point cloud distortion of the laser radar in high-speed motion are solved, effective information of the laser radar and the IMU can be extracted, and line features are introduced to improve the calibration accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0021] Figure 1 A flowchart example of a laser radar and IMU joint calibration method for a mine provided by the embodiments of the present disclosure is shown in the figure.

[0022] Figure 2 A flowchart example of another laser radar and IMU joint calibration method for a mine provided by the embodiments of the present disclosure is shown in the figure.

[0023] Figure 3 A structural schematic diagram of a laser radar and IMU joint calibration device for a mine provided by the embodiments of the present disclosure is shown in the figure. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the above-mentioned purposes, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0025] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, not all the embodiments.

[0026] Based on the foregoing background technology description, in the intelligent driving system, the laser radar can obtain the three-dimensional information in the environment in real time, but the laser point cloud data will have distortion problems, especially in high-speed motion scenarios. The IMU can obtain its three-axis acceleration and three-axis angular velocity at a high frequency, and can estimate the laser radar scanning motion, greatly reducing the distortion of the point cloud. By combining the advantages of laser radar and IMU and fusing the information of the two, it is very beneficial to the mapping and positioning of intelligent driving. Before fusing the laser and IMU information, the relative position and attitude of the two need to be calibrated to ensure the accuracy of the fused information. In the current calibration method of laser radar and IMU, first, most of them rely on specified markers, and the calibration is relatively cumbersome; second, for objects that are blocked or lack large flat surfaces in the environment, the calibration accuracy is relatively low.

[0027] That is, for the joint calibration method of the laser radar and the IMU, one is a calibration based on no specified marker, which solves the geometric constraint of a line plane to obtain the external parameter by using the natural environment of the scene. This method has low calibration accuracy and poor robustness in the environment without large flat surfaces. One is a calibration based on a specified marker, which collects point cloud and IMU data by using a designed calibration board, constructs a graph optimization model to obtain the external parameter between the laser and the IMU. This method needs to specify the marker, and the operation is more complicated and has limitations.

[0028] To solve the above problems, the present disclosure provides a laser radar and IMU joint calibration method for a mine, which does not need to specify the marker to automatically calibrate the external parameter of the laser radar and the IMU, and still has good calibration accuracy under the condition that there is a lack of large flat surfaces in the environment (such as a mine). In addition, the point cloud is deformed during calibration, and the line feature is introduced to improve the accuracy of the calibration.

[0029] Figure 1 A flowchart example of a laser radar and IMU joint calibration method for a mine provided by the embodiment of the present disclosure is shown in the figure, the relative position of the laser radar and the inertial measurement unit (IMU) remains unchanged, comprising:

[0030] Step 101, acquiring laser point cloud data collected by the laser radar and IMU data collected by the IMU.

[0031] Specifically, the number of installations of the laser radar and the IMU is not specifically limited, and the laser radar and the IMU are usually installed on the same vehicle. In the embodiment of the present disclosure, the laser radar and the IMU are fixed to form a whole, and it is confirmed that the relative position of the laser radar and the IMU will not change. At the same time, the laser point cloud data and the IMU data can form a closed loop; wherein the laser point cloud data refers to multiple frames of laser point cloud in the laser coordinate system, and the IMU data refers to three-axis attitude angle (or angle degree) and acceleration data in the IMU coordinate system.

[0032] Step 102, based on the laser point cloud data and the IMU data, the attitude change information of the laser radar and the IMU is obtained by calculation.

[0033] In the embodiment of the present disclosure, after synchronizing the laser point cloud data and the IMU data according to the time stamp, the pose group of the laser point cloud data and the pose group of the IMU data are obtained, and the pose transformation matrix of the laser point cloud data is determined according to the pose group of the laser point cloud data, and the pose transformation matrix of the IMU data is determined according to the pose group of the IMU data. Finally, the hand-eye calibration equation can be constructed according to the pose transformation matrix and the initial external parameter of the laser radar and the IMU, so as to obtain the attitude change information of the laser radar and the IMU.

[0034] In one embodiment, based on laser point cloud data and IMU data, the pose change information of the lidar and the IMU is obtained through processing, including: synchronizing the time stamps of the laser point cloud data and the IMU data, and calculating the pose change of each frame of laser point cloud in the laser point cloud data based on the registration algorithm of the Normal Distributions Transform (NDT), to construct a pose group of the laser point cloud data; based on the extended Kalman filter algorithm, a pose group of the IMU data is constructed; based on the pose group of the laser point cloud data and the pose group of the IMU data, a pose transformation matrix of the lidar and the IMU is calculated; based on the pose transformation matrix and the initial extrinsic parameters of the lidar and the IMU, a hand-eye calibration equation is constructed, to obtain the pose change information of the lidar and the IMU.

[0035] In step 103, based on the initial extrinsic parameters in the pose change information, point cloud distortion calibration is performed on the laser point cloud data, to obtain target laser point cloud data.

[0036] Specifically, there is a point cloud distortion problem of the lidar in high-speed motion, therefore, it is necessary to perform point cloud distortion calibration on the laser point cloud data. Specifically, the rotation increment of each point relative to the frame header moment is determined based on the initial extrinsic parameters and the laser point cloud data, and each frame of laser point cloud is compensated based on the rotation increment, to obtain target laser point cloud data.

[0037] In step 104, key frame point cloud data is obtained from the target laser point cloud data, a target number group hand-eye calibration equation is constructed based on the key frame point cloud data and the pose change information, and a first residual is determined based on the target number group hand-eye calibration equation.

[0038] Specifically, in order to further improve the calibration accuracy, it is necessary to obtain key frame point cloud data from the target laser point cloud data. In some embodiments, after filtering the target laser point cloud data, the point cloud rotation angle corresponding to each frame of target laser point cloud is obtained, and the target laser point cloud with a point cloud rotation angle greater than a preset rotation angle threshold is taken as a key frame point cloud, to obtain key frame point cloud data; in another embodiment, after filtering the target laser point cloud data, the translation distance corresponding to each frame of target laser point cloud is obtained, and the target laser point cloud with a translation distance greater than a preset distance threshold is taken as a key frame point cloud, to obtain key frame point cloud data.

[0039] Further, the pose change information of the IMU motion at the moment corresponding to the key frame point cloud is obtained, such as the target number group key frame point cloud data, a target number group hand-eye calibration equation can be constructed, a least squares optimization problem is constructed for the target number group hand-eye calibration equation, and the residual of each item is calculated, i.e. the first residual is obtained.

[0040] In step 105, the key frame point cloud data is clustered based on a preset clustering algorithm to obtain line feature points of each key frame point cloud, and the line feature points of each key frame point cloud are processed to obtain a second residual error.

[0041] In step 106, a target optimization function is determined based on the first residual error and the second residual error, and the target optimization function is solved based on a preset solving algorithm to obtain the target extrinsic parameters of the laser radar and the IMU.

[0042] Specifically, the key frame point cloud data is clustered by a preset clustering algorithm such as the LeGO-LOAM algorithm to obtain line feature points of each key frame point cloud, and the line feature points of each key frame point cloud are processed to obtain a second residual error. Specifically, the line feature points of each key frame point cloud are mapped to obtain a line feature point cloud map, the line feature point cloud map is divided to obtain a plurality of point cloud clusters, the line feature representation of each point cluster is obtained, a straight line distance formula of the laser point cloud to the line feature representation of each point cluster is constructed, and the distance residual error is determined based on the straight line distance formula as the second residual error.

[0043] Finally, a target optimization function is determined based on the first residual error and the second residual error, and the target optimization function is solved based on a preset solving algorithm to obtain the target extrinsic parameters of the laser radar and the IMU. Specifically, a first optimization function is constructed based on the first residual error, a second optimization function is constructed based on the second residual error, the target optimization function is generated based on the first optimization function and the second optimization function, and the target optimization function is solved by a solving algorithm such as the Gauss-Newton algorithm to obtain the target extrinsic parameters of the laser radar and the IMU.

[0044] In summary, the method for joint calibration of a laser radar and an IMU for a mine according to the embodiments of the present disclosure acquires laser point cloud data collected by the laser radar and IMU data collected by the IMU; calculates based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU; performs point cloud distortion calibration on the laser point cloud data based on initial extrinsic parameters in the attitude change information to obtain target laser point cloud data; acquires key frame point cloud data from the target laser point cloud data, constructs a target number group hand-eye calibration equation based on the key frame point cloud data and the attitude change information, and determines a first residual error based on the target number group hand-eye calibration equation; performs clustering processing on the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud, and processes based on the line feature points of each key frame point cloud to obtain a second residual error; determines a target optimization function based on the first residual error and the second residual error, and solves the target optimization function based on a preset solving algorithm to obtain target extrinsic parameters of the laser radar and the IMU. In this way, the technical problems in the prior art that the calibration process of the laser radar and the IMU relies on the development of markers and the laser radar in high-speed motion point cloud distortion are solved, effective information of the laser radar and the IMU can be extracted, line features are introduced, and the calibration accuracy is improved.

[0045] Specifically, Figure 2 Another flowchart example of a method for joint calibration of a laser radar and an IMU for a mine is provided for the embodiments of the present disclosure. The present embodiment further optimizes the method for joint calibration of a laser radar and an IMU for a mine described above. As shown in the figure, the method comprises the following steps: Figure 2

[0046] Step 201: Acquire laser point cloud data collected by a laser radar and IMU data collected by an IMU.

[0047] Specifically, the laser radar and the IMU are fixed to become one whole, and the relative positions of the two are confirmed not to change. Define the laser radar coordinate system O L , hereinafter referred to as O L , the IMU coordinate system O I , hereinafter referred to as O I . Collect laser point cloud and IMU data that can form a closed loop at the same time.

[0048] Step 202: Synchronize the time stamps of the laser point cloud data and the IMU data, calculate the pose change of each frame of laser point cloud in the laser point cloud data based on a normal distribution transformation registration algorithm, construct a pose group of the laser point cloud data, and construct a pose group of the IMU data based on an extended Kalman filter algorithm.

[0049] ​Step 203, based on the pose group of the laser point cloud data and the pose group of the IMU data, a pose transformation matrix of the laser radar and the IMU is calculated, based on the pose transformation matrix and the initial external parameter of the laser radar and the IMU, a hand-eye calibration equation is constructed, and the attitude change information of the laser radar and the IMU is obtained.

[0050] Specifically, the collected laser point cloud data and the IMU data are time-stamped synchronized, and then the pose change of each frame of laser point cloud is calculated by the NDT algorithm to construct the pose group of the laser point cloud data The pose group of the IMU data is constructed by extending the Kalman filter algorithm Among them, is the pose of the K-th laser point cloud in O L , is the pose of the K-th IMU in O I . The pose transformation of the laser radar and the IMU from K-1 time to K time is calculated respectively Then the hand-eye calibration equation is constructed, because the laser radar and the IMU are fixedly connected, so the external parameters of the two do not change with time, and the following can be obtained:

[0051] Step 204, based on the initial external parameter and the laser point cloud data, the rotation increment of each frame of laser point cloud relative to the frame header time is determined, and each frame of laser point cloud is compensated based on the rotation increment to obtain target laser point cloud data.

[0052] Specifically, based on the external parameter of the laser radar and the IMU obtained in the above step 203, the rotation increment of each point in each frame of laser point cloud relative to the frame header time is estimated: In the formula, indicates the rotation increment of the i-th point of the K-th frame of laser point cloud relative to the frame header, indicates the frame header time of the K-th frame, indicates the time corresponding to the i-th point of the K-th frame. Through the rotation increment, the points of each frame are compensated, and the distortion processing of each frame of point cloud of the laser radar is realized.

[0053] Step 205, after filtering the target laser point cloud data, the point cloud rotation angle or the translation distance corresponding to each frame of target laser point cloud is obtained, and the target laser point cloud with the point cloud rotation angle greater than the preset rotation angle threshold or the translation distance greater than the preset distance threshold is taken as the key frame point cloud, and the key frame point cloud data is obtained.

[0054] Step 206, based on the key frame point cloud data and the attitude change information, a target number group hand-eye calibration equation is constructed, and a first residual error is determined based on the target number group hand-eye calibration equation.

[0055] ​Specifically, the deformed point cloud obtained in step 204 is filtered to remove noise points and occluded points. In the laser motion process, laser point clouds with a rotation angle greater than a preset rotation angle threshold (which can be selected and set according to the actual application scenario), such as 0.5°, or a translation distance greater than a preset distance threshold (which can be selected and set according to the actual application scenario), such as 0.1 m, are selected as key frame point clouds, and the pose information of the IMU motion at this moment is recorded. Assuming that there are n groups of target key frame data, n groups of hand-eye calibration equations are constructed as follows:

[0056] Specifically, a least squares optimization problem is constructed for the n groups of calibration equations, and the residual of each term is calculated denoted as the first residual r1.

[0057] Step 207, based on a preset clustering algorithm, the key frame point cloud data is clustered to obtain the line feature points of each key frame point cloud, and the line feature point cloud map is obtained by mapping each key frame point cloud line feature point. The line feature point cloud map is divided to obtain a plurality of point cloud clusters, the line feature representation of each point cluster is obtained, and a straight line distance formula from the laser point cloud to the line feature representation of each point cluster is constructed. The distance residual is determined based on the straight line distance formula as the second residual.

[0058] Specifically, for the key frame point cloud data obtained in step 205, the LeGO-LOAM algorithm is used to cluster the laser point cloud to obtain the line feature points of each frame of laser point cloud, and then the line feature point cloud map is obtained by mapping the clustered line feature points. The line feature point cloud map is divided into a plurality of point cloud clusters, and the line feature representation of each point cloud cluster is denoted as [O, n0], O is the coordinate of a point on the line feature straight line, and n0 is the normal vector on the straight line. For the laser points of the line feature cluster, the distance d L is calculated from the point to the line feature cluster point cloud straight line. where P M The surface feature laser point is P L The coordinates of the key frame point cloud.

[0059] Specifically, for the point to straight line distance d L , the distance residual of the point to the straight line is denoted as the second residual r2.

[0060] Step 208, based on the first residual, a first optimization function is constructed, based on the second residual, a second optimization function is constructed, based on the first optimization function and the second optimization function, a target optimization function is generated, and based on a preset solving algorithm, the target optimization function is solved to obtain the target external parameter of the laser radar and the IMU.

[0061] Specifically, the Mahalanobis norm of n groups of residuals is accumulated to construct a least square optimization objective function, i.e., a first optimization function: Similarly, a second optimization function is obtained, so that a final objective function can be constructed in combination with r1 and r2 The external parameters after optimization can be obtained by minimizing the objective function using the Gauss-Newton method wherein ρ u is a kernel function, is the constructed objective function.

[0062] Therefore, the method for jointly calibrating a laser radar and an IMU for a mine can not need to specify a marker, can perform distortion correction on point clouds during calibration, and introduces line features to improve the accuracy of calibration.

[0063] Corresponding to the method for jointly calibrating a laser radar and an IMU for a mine, an embodiment of the present disclosure provides a device for jointly calibrating a laser radar and an IMU for a mine, Figure 3 A structural schematic diagram of a device for jointly calibrating a laser radar and an IMU for a mine provided by an embodiment of the present disclosure, which can be implemented by software and / or hardware, and can generally be integrated in an electronic device, such as Figure 3 As shown in the figure, the mine road network includes a plurality of loading areas, each of which has a plurality of electric shovels, the relative positions of the laser radar and the inertial measurement unit (IMU) remain unchanged, and the device for jointly calibrating a laser radar and an IMU for a mine 300 includes the following modules:

[0064] The acquisition module 301 is configured to acquire laser point cloud data collected by the laser radar and IMU data collected by the IMU;

[0065] The calculation module 302 is configured to perform calculation based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU;

[0066] The calibration module 303 is configured to perform point cloud distortion calibration on the laser point cloud data based on initial external parameters in the attitude change information to obtain target laser point cloud data;

[0067] The acquisition module 304 is configured to acquire key frame point cloud data from the target laser point cloud data;

[0068] The construction and determination module 305 is configured to construct a target number of groups of hand-eye calibration equations based on the key frame point cloud data and the attitude change information, and determine a first residual based on the target number of groups of hand-eye calibration equations;

[0069] The clustering module 306 is configured to perform clustering processing on the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud.

[0070] process the line feature points of each key frame point cloud to obtain a second residual;

[0071] determine a target optimization function based on the first residual and the second residual, and solve the target optimization function based on a preset solving algorithm to obtain a target extrinsic parameter of the lidar and the IMU.

[0072] In some embodiments, the computing module 302 is specifically configured to: synchronize time stamps of the laser point cloud data and the IMU data, and calculate a pose change of each frame of laser point cloud in the laser point cloud data based on a registration algorithm of normal distribution transformation to construct a pose group of the laser point cloud data; construct a pose group of the IMU data based on an extended Kalman filtering algorithm; calculate a pose transformation matrix of the lidar and the IMU based on the pose group of the laser point cloud data and the pose group of the IMU data; and construct a hand-eye calibration equation based on the pose transformation matrix and an initial extrinsic parameter of the lidar and the IMU to obtain attitude change information of the lidar and the IMU.

[0073] In some embodiments, the calibration module 303 is specifically configured to: determine a rotation increment of each point relative to a frame header moment based on the initial extrinsic parameter and the laser point cloud data; and compensate each frame of laser point cloud based on the rotation increment to obtain the target laser point cloud data.

[0074] In some embodiments, the acquisition module 304 is specifically configured to: after filtering the target laser point cloud data, acquire a point cloud rotation angle corresponding to each frame of target laser point cloud; and take a target laser point cloud with a point cloud rotation angle greater than a preset rotation angle threshold as a key frame point cloud to obtain the key frame point cloud data.

[0075] In some embodiments, the acquisition module 304 is specifically configured to: after filtering the target laser point cloud data, acquire a translation distance corresponding to each frame of target laser point cloud; and take a target laser point cloud with a translation distance greater than a preset distance threshold as a key frame point cloud to obtain the key frame point cloud data.

[0076] In some embodiments, the processing module 307 is specifically configured to: map the line feature points of each key frame point cloud to obtain a line feature point cloud map; divide the line feature point cloud map to obtain a plurality of point cloud clusters; acquire a line feature representation of each point cloud cluster; construct a straight line distance formula of laser point cloud to the line feature representation of each point cloud cluster; and determine a distance residual based on the straight line distance formula as the second residual.

[0077] In some embodiments, the determining module 308 is specifically configured to: construct a first optimization function based on the first residual; construct a second optimization function based on the second residual; generate the target optimization function based on the first optimization function and the second optimization function, and solve the target optimization function based on a preset solving algorithm to obtain the target extrinsic parameter of the laser radar and the IMU.

[0078] The laser radar and IMU joint calibration device for mines provided by the embodiments of the present application can perform the laser radar and IMU joint calibration method for mines provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0079] According to the embodiments of the present application, an electronic device is provided, which includes: a processor; a memory for storing executable instructions of the processor; and the processor is configured to read the executable instructions from the memory and execute the instructions to implement the laser radar and IMU joint calibration method for mines provided by the embodiments of the present application.

[0080] The embodiments of the present application also provide a storage medium containing computer executable instructions, which are used to implement the laser radar and IMU joint calibration method for mines provided by the embodiments of the present application when executed by a computer processor.

[0081] Of course, the computer executable instructions of the storage medium provided by the embodiments of the present application are not limited to the method operations as described above, but can also perform the related operations in the laser radar and IMU joint calibration method for mines provided by any of the embodiments of the present application.

[0082] The embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is used to implement the laser radar and IMU joint calibration method for mines provided by the embodiments of the present application when executed by a processor.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary universal hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH, a hard disk, or an optical disc, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0084] It is worth noting that in the above embodiments of the search device, each unit and module included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy mutual distinction, and do not limit the protection scope of the present application.

[0085] It should be noted that in this paper, relationship terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the sentence "including a…" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0086] The above description is only a specific embodiment of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for joint calibration of LiDAR and IMU for mines, characterized in that, The laser radar and an inertial measurement unit (IMU) have a relative position that remains unchanged, and the method comprises: acquiring laser point cloud data collected by the laser radar and IMU data collected by the IMU; calculating based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU; performing point cloud distortion calibration on the laser point cloud data based on initial extrinsic parameters in the attitude change information to obtain target laser point cloud data; acquiring key frame point cloud data from the target laser point cloud data, constructing a target number group hand-eye calibration equation based on the key frame point cloud data and the attitude change information, and determining a first residual based on the target number group hand-eye calibration equation; performing clustering processing on the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud, and performing processing based on the line feature points of each key frame point cloud to obtain a second residual; determining a target optimization function based on the first residual and the second residual, and solving the target optimization function based on a preset solving algorithm to obtain target extrinsic parameters of the laser radar and the IMU; the processing based on the line feature points of each key frame point cloud to obtain a second residual comprises: mapping the line feature points of each key frame point cloud to obtain a line feature point cloud map; dividing the line feature point cloud map to obtain a plurality of point cloud clusters; acquiring a line feature representation of each point cloud cluster, constructing a straight line distance formula of laser point cloud to the line feature representation of each point cloud cluster, and determining a distance residual as the second residual based on the straight line distance formula.

2. The method for joint calibration of LiDAR and IMU for mines according to claim 1, characterized in that, the calculation based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU comprises: synchronizing time stamps of the laser point cloud data and the IMU data, and calculating a pose change of each frame of laser point cloud in the laser point cloud data based on a normal distribution transformation registration algorithm to construct a pose group of the laser point cloud data; constructing a pose group of the IMU data based on an extended Kalman filter algorithm; calculating a pose transformation matrix of the laser radar and the IMU based on the pose group of the laser point cloud data and the pose group of the IMU data; constructing a hand-eye calibration equation based on the pose transformation matrix and initial extrinsic parameters of the laser radar and the IMU to obtain attitude change information of the laser radar and the IMU.

3. The method for joint calibration of LiDAR and IMU for mines as claimed in claim 1 wherein, the point cloud distortion calibration on the laser point cloud data based on the initial extrinsic parameters in the attitude change information to obtain target laser point cloud data comprises: determining a rotation increment of each point relative to a frame header time based on the initial extrinsic parameters and the laser point cloud data; compensating each frame of laser point cloud based on the rotation increment to obtain the target laser point cloud data.

4. The method for joint calibration of LiDAR and IMU for mines according to claim 1, characterized in that, the acquisition of key frame point cloud data from the target laser point cloud data comprises: after filtering the target laser point cloud data, acquiring a point cloud rotation angle corresponding to each frame of target laser point cloud; The target laser point cloud with a rotation angle greater than a preset rotation angle threshold is taken as a key frame point cloud, and key frame point cloud data is obtained.

5. The method for joint calibration of LiDAR and IMU for mines according to claim 1, characterized in that, The key frame point cloud data is obtained from the target laser point cloud data, and includes: After filtering the target laser point cloud data, a translation distance corresponding to each frame of target laser point cloud is obtained. The target laser point cloud with a translation distance greater than a preset distance threshold is taken as a key frame point cloud, and the key frame point cloud data is obtained.

6. The method for joint calibration of LiDAR and IMU for mines as claimed in claim 1 wherein, The target optimization function is determined based on the first residual and the second residual, and includes: A first optimization function is constructed based on the first residual; A second optimization function is constructed based on the second residual; The target optimization function is generated based on the first optimization function and the second optimization function.

7. A combined laser radar and IMU calibration device for a mine, characterized by, The relative position between the laser radar and the inertial measurement unit (IMU) remains unchanged, and the device includes: An acquisition module is configured to acquire laser point cloud data collected by the laser radar and IMU data collected by the IMU; A calculation module is configured to calculate based on the laser point cloud data and the IMU data to obtain attitude change information of the laser radar and the IMU; A calibration module is configured to perform point cloud distortion calibration on the laser point cloud data based on initial external parameters in the attitude change information to obtain target laser point cloud data; An acquisition module is configured to acquire key frame point cloud data from the target laser point cloud data; A construction and determination module is configured to construct a target number of group hand-eye calibration equations based on the key frame point cloud data and the attitude change information, and determine a first residual based on the target number of group hand-eye calibration equations; A clustering module is configured to perform clustering processing on the key frame point cloud data based on a preset clustering algorithm to obtain line feature points of each key frame point cloud; A processing module is configured to process based on the line feature points of each key frame point cloud to obtain a second residual; A determination and solving module is configured to determine a target optimization function based on the first residual and the second residual, and solve the target optimization function based on a preset solving algorithm to obtain target external parameters of the laser radar and the IMU. The processing based on the line feature points of each key frame point cloud to obtain a second residual includes: Mapping the line feature points of each key frame point cloud to obtain a line feature point cloud map; Dividing the line feature point cloud map to obtain a plurality of point cloud clusters; Obtaining a line feature representation of each point cloud cluster, constructing a straight line distance formula of the laser point cloud of each point cloud cluster to the line feature representation, and determining a distance residual as the second residual based on the straight line distance formula.

8. An electronic device, comprising: The electronic device includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the laser radar and IMU joint calibration method for a mine according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the laser radar and IMU joint calibration method for a mine according to any one of claims 1-6.

Images