Vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration method and device

Abstract

The application provides a kind of vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration method and device, the method is by digging round hole as laser radar recognizable feature on calibration board, the coordinates of four corner points of calibration board under laser radar coordinate system are calculated, the same point pairing with WGS-84 control point on calibration board is realized in combination with the absolute pose information of vehicle-mounted GPS / IMU, and finally the external parameter matrix of laser radar and IMU is solved by Gauss Newton method. Through the scheme, the calibration scene requirement can be reduced, the automatic external parameter calibration can be realized, the calibration efficiency can be effectively improved, the influence of human factors can be avoided, and the accuracy and reliability of calibration result can be ensured.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of automatic driving, and particularly relates to a vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration method and device. BACKGROUND

[0002] As a commonly used sensor device, the multi-line laser radar has the characteristics of high measurement position accuracy, large measurement range, and less external environment interference, and can be used to accurately measure the position and distance information of obstacles in the surrounding environment. The fusion positioning of GPS and IMU (Inertial Measurement Unit) can provide accurate pose information for the carrier. The data fusion of laser radar and GPS / IMU is widely used in many fields such as surveying, automatic driving, and robots. Since the measurement accuracy of the multi-line laser radar is closely related to its own internal parameters and external parameters relative to the IMU, it is necessary to calibrate the internal parameters and external parameters of the multi-line laser radar to improve the measurement accuracy of the multi-line laser radar. However, under normal circumstances, the internal parameters of the laser radar have been calibrated before leaving the factory.

[0003] The common external parameter calibration of laser radar and IMU mainly includes manual measurement calibration and calibration field control point calibration: manual calibration adopts manual or instrument measurement to measure the transformation relationship between the laser radar coordinate system and the IMU coordinate system. First, the positions of the laser radar coordinate system origin and the IMU coordinate system origin are determined, and the lateral, longitudinal and vertical offset distances between the laser radar coordinate system origin and the IMU origin are measured by using a caliper or a tape measure. Then, the directions of the laser radar coordinate axis and the IMU coordinate axis are determined, and the roll angle, pitch angle and yaw angle of the laser radar in the IMU coordinate system are measured by using a special test instrument such as a three-dimensional angle measuring instrument.

[0004] The calibration field control point calibration method selects an open area with good GPS signal as the calibration field, selects regular house or window corner points as control points, and measures the WGS-84 coordinates of each control point by using a total station. During calibration, the point cloud data of the multi-line laser radar at each control point position in the calibration scene is manually selected, and the point cloud data collected at the control point position is matched with the calibration control point coordinates in WGS-84, and a strict positioning equation of the vehicle-mounted laser radar system is established. The least square adjustment is used to calculate the Rodrigues parameters a, b and c, and the rotation matrix of the laser radar relative to the IMU is calculated. Then, the offset vector between the two sensors is calculated according to the least square adjustment.

[0005] The artificial measurement calibration method has simple principle, but has high requirement for operation precision of operator and precision of measuring instrument, and it is difficult to determine the origin position of the IMU coordinate system in measurement, so that the measurement result has large error and uncertainty, and the measurement precision varies with the level of the operator. The calibration field control point calibration method can theoretically achieve high calibration precision, but the movement of the vehicle-mounted system causes the coordinate origin and the coordinate axis direction of the laser radar to change constantly, and the point cloud of the multi-line laser radar is sparse, so it is difficult to ensure that the light beams can accurately scan the corner points of the house / window. At the same time, the artificial selection of the point cloud coordinates corresponding to the control points in the laser radar coordinate system also has certain error.

[0006] It can be seen that the two calibration methods both need manual operation, the process is complex and low in efficiency, and it is difficult to be applied to batch calibration of the sensor external parameters of the vehicle-mounted system. SUMMARY

[0007] Therefore, the embodiments of the present application provide a vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration method and device, which are used to solve the problems of the existing calibration methods that need manual participation, complex process and low efficiency.

[0008] In a first aspect of the embodiments of the present application, a vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration method is provided, comprising:

[0009] Collecting laser radar original point cloud data of a calibration board and absolute pose of a vehicle in a predetermined route in a calibration field;

[0010] Extracting point cloud data of a region of interest in the original point cloud data, and separating out spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance clustering;

[0011] Extracting plane point cloud from the target cluster by using a RANSAC plane detection algorithm, and extracting edge point cloud according to discontinuity of depth information of radar echo points, the edge point cloud being edge points of hollowed-out circles on the calibration board and edge points on left and right sides of the calibration board;

[0012] After removing the edge point cloud on the left and right sides of the calibration board, a RANSAC sphere fitting algorithm is used to fit the best candidate sphere to the edge point cloud of the hollowed-out circle hole, and the center coordinates of the sphere are calculated;

[0013] According to the three-dimensional coordinates of the inner circle center of the calibration board, the physical size information of the calibration board is known, and the coordinates of four corner points of the calibration board in the laser radar coordinate system are calculated;

[0014] The absolute pose in the IMU coordinate system corresponding to the time stamp of the current frame point cloud is calculated, and all WGS-84 control point coordinates are converted to the IMU coordinate system according to the absolute pose.

[0015] Set the initial values ​​of the external parameters of the lidar and IMU, transform the coordinates of the four corner points of the calibration board from the lidar coordinate system to the IMU coordinate system, and find the control points corresponding to the four corner points of the calibration board based on the KNN nearest neighbor matching algorithm to form pairs of points with the same name.

[0016] A certain number of pairs of identical points are obtained, and the external parameters of the laser radar to the IMU are obtained by solving multiple pairs of identical points using the Gauss-Newton method.

[0017] In a second aspect of the present invention, an automatic calibration device for the extrinsic parameters of a vehicle-mounted multi-line lidar and IMU is provided, comprising:

[0018] The data acquisition module is used to collect raw point cloud data of the lidar on the calibration board and the absolute pose of the vehicle in the calibration field along a predetermined route;

[0019] The clustering separation module is used to extract the point cloud data of the region of interest from the original point cloud data, and to separate the spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance.

[0020] The edge point cloud extraction module is used to extract planar point clouds from target clusters using the RANSAC planar detection algorithm, and to extract edge point clouds based on the discontinuity of depth information of radar echo points. The edge point clouds are the edge points of the hollow circle on the calibration plate and the edge points on the left and right sides of the calibration plate.

[0021] The spherical fitting module is used to remove the edge point clouds on the left and right sides of the calibration board, and then use the RANSAC spherical fitting algorithm to fit the best candidate sphere to the edge point cloud of the hollowed-out circular hole, and calculate the coordinates of the sphere center.

[0022] The corner coordinate calculation module is used to calculate the coordinates of the four corner points of the calibration plate in the lidar coordinate system based on the three-dimensional coordinates of the center of the inner circle of the calibration plate and the known physical size information of the calibration plate.

[0023] The coordinate transformation module is used to calculate the absolute pose in the IMU coordinate system corresponding to the timestamp of the current frame point cloud, and transform the coordinates of all WGS-84 control points to the IMU coordinate system based on the absolute pose.

[0024] The control point acquisition module is used to set the initial values ​​of the external parameters of the lidar and IMU, transform the coordinates of the four corner points of the calibration board from the lidar coordinate system to the IMU coordinate system, and find the control points corresponding to the four corner points of the calibration board based on the KNN nearest neighbor matching algorithm to form pairs of points with the same name.

[0025] The solving calibration module is configured to obtain a certain number of homonym point pairs, and solve a plurality of groups of homonym point pairs by using a Gauss-Newton method to obtain the external parameters of the laser radar to the IMU.

[0026] In a third aspect of the embodiments of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable by the processor, and the processor implements the steps of the method according to the first aspect of the embodiments of the present application when executing the computer program.

[0027] In a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program is executable by a processor to implement the steps of the method according to the first aspect of the embodiments of the present application.

[0028] In the embodiments of the present application, it is not necessary to specially search for a calibration field with obvious regular building corner points, and the corner point coordinate measurement of the calibration plate can be completed in a region with good GPS / IMU signals, thereby reducing the requirement for the calibration scene, automatically extracting the coordinates of the WGS-84 control points in the laser radar coordinate system, completing the external parameter solving and calibration of the laser radar and the IMU, thereby simplifying the external parameter calibration process, realizing automatic calibration processing, improving the calibration efficiency, avoiding errors caused by human factors, and meeting the current batched external parameter calibration requirements of vehicle-mounted sensors. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0030] Figure 1 The design schematic diagram of the calibration plate provided by one embodiment of the present application is shown in FIG. 1.

[0031] Figure 2 The design schematic diagram of the calibration plate base provided by one embodiment of the present application is shown in FIG. 2.

[0032] Figure 3 The flowchart of the automatic calibration method of the external parameters of the vehicle-mounted multi-line laser radar and the IMU provided by one embodiment of the present application is shown in FIG. 3.

[0033] Figure 4 The feature point cloud schematic diagram of the calibration plate provided by one embodiment of the present application is shown in FIG. 4.

[0034] Figure 5 The structure schematic diagram of the automatic calibration device of the external parameters of the vehicle-mounted multi-line laser radar and the IMU provided by one embodiment of the present application is shown in FIG. 5.

[0035] Figure 6 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the application purpose, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the embodiments described below are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0037] It should be understood that the terms “comprise” and other similar expressions in the specification or claims of the present application and the above drawings mean covering non-exclusive inclusion, such as a process, method or system, device comprising a series of steps or units, which is not limited to the listed steps or units. In addition, “first” and “second” are used to distinguish different objects, and are not used to describe a specific order.

[0038] It can be understood that, unlike the traditional calibration method, in the embodiments of the present application, a calibration field with obvious regular building corner points does not need to be specially found, and the deployment of the calibration board and the measurement of the corner point coordinates of the total station can be completed in an area with good GPS / IMU signals. The coordinates of the same-named points of the WGS-84 control points in the laser radar coordinate system can be automatically extracted during calibration, which can solve the problem that the laser beam is difficult to scan the house / window corner point in the traditional calibration field control point calibration method, and eliminate the error generated when the laser point cloud same-named point coordinates are manually selected.

[0039] Before point cloud data collection and calibration, a calibration board device needs to be made, including a calibration board base support, and the calibration field needs to be deployed. Specifically as follows:

[0040] Calibration device making

[0041] Square hollow circular hole calibration board making: design and make a square hardwood board with accurate size (such as 160 cm long, 160 cm wide, and 2 cm thick), and symmetrically dig four hollow circular holes with the same radius (such as 20 cm radius) at the center positions of the half axes of the two diagonal lines of the wood board. The shape and size parameters of the calibration board are shown in the attached Figure 1

[0042] Base support making: according to prior information such as the installation position of the vehicle-mounted laser radar, design and make a base support to fix the calibration board on the support. The design height of the support needs to ensure that there are enough points of a single frame of laser radar point cloud falling on the calibration board. The design of the calibration board base is shown in the attached Figure 2 ​as shown.

[0043] Calibration field deployment

[0044] Select a relatively flat ground, around the open, GPS signal good closed / semi-closed section as a calibration field. In order to facilitate the calibration of the laser point cloud more easily from the calibration board, the calibration field is preferably set in the road section without dense vegetation and traffic signs and other planar interference.

[0045] Set the driving route of the vehicle in the calibration field, and reasonably place multiple calibration board devices combined with actual road condition information. The position should meet: on the vehicle driving route, at least one frame of laser point cloud data exists, and more than two laser beams can scan the edge of each hollow circle, that is, at least four radar reflection points are guaranteed on each circle of the calibration board plane. The position deployment of the calibration board can be assisted by point cloud visualization tools (such as CloudCompare, pcl_viewer, etc.).

[0046] Among them, after the calibration board is deployed, the WGS-84 coordinates corresponding to the four corner points of all calibration boards are measured by the total station as control point coordinates.

[0047] The automatic calibration process of the laser radar to the IMU external parameter, please refer to Figure 3 A flowchart of a vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration method provided by the embodiment of the present application, comprising:

[0048] S101, collect laser radar original point cloud data and vehicle absolute pose of the calibration board in the calibration field according to the predetermined route;

[0049] Collect the point cloud data of the calibration board placed in a specific position by the vehicle according to the set route, and record the absolute pose of the vehicle in real time based on the fusion of GPS / IMU.

[0050] Exemplarily, the vehicle drives at a speed of 5km / h, and the laser radar original point cloud data (each echo point information includes: three-dimensional coordinates x / y / z in the radar coordinate system, reflection intensity intensity, and ring number) is collected.

[0051] In order to ensure that more laser point clouds fall on the calibration board, the horizontal resolution of the laser radar can be improved by reducing the scanning frequency of the laser radar (such as reducing to the lowest 5Hz).

[0052] Among them, the calibration board is fixed on the support, the calibration board plane is perpendicular to the ground, four hollow circular holes with the same radius are dug on the calibration board, and the two centers on the same diagonal line divide the diagonal line of the calibration board into four equal parts.

[0053] The calibration board is arranged on a vehicle driving route, and at least one frame of laser point cloud data has more than two laser beams capable of scanning the edge of each hollow circle of the calibration board; after the calibration board is deployed, the WGS-84 coordinates of the four corner points of all the calibration boards are measured by a total station as control point coordinates.

[0054] Further, the point cloud data is time-stamped in the manner of PPS and GPRMC.

[0055] S102, extracting point cloud data of a region of interest in the original point cloud data, and separating spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance clustering;

[0056] The region of interest is the region point cloud data of the calibration board, which can be extracted according to the calibration board feature information, such as non-ground point cloud, installation height, etc.

[0057] Specifically, the CSF (Cloth Simulation Filter) algorithm is used to segment the ground point cloud and the non-ground point cloud for single-frame point cloud data; the non-ground point cloud is filtered based on prior information (installation distance, height, altitude, etc. of the calibration board) to extract the point cloud data of the region of interest.

[0058] The spatially independent target cluster is an independent object point cloud cluster in three-dimensional space, such as a calibration board and a street lamp. The clustering method of Euclidean distance can directly, simply and quickly search for feature points to obtain the point cloud of the target cluster.

[0059] S103, extracting plane point cloud from the target cluster by RANSAC plane detection algorithm, and extracting edge point cloud according to the discontinuity of the depth information of the radar echo point;

[0060] The edge point cloud is the edge point of the hollow circle on the calibration board and the edge point on the left and right sides of the calibration board.

[0061] The RANSAC plane detection algorithm is a random sample consensus algorithm, which estimates model parameters by iteration in a data set containing outliers. In this embodiment, the RANSAC algorithm is used for plane detection to estimate the plane point cloud.

[0062] Further, the size of the calibration board and the plane normal vector and other information are combined to accurately obtain the point cloud data located on the plane of the calibration board, as shown in (a). Figure 4

[0063] ​For each calibration plate planar point cloud data, the depth information discontinuity of each bundle of rays on the radar echo point is used to extract the edge point cloud that meets the conditions. This edge point cloud is divided into the edge points of the hollowed-out circle and the edge points on the left and right sides of the calibration plate, as shown in FIG. 8. Figure 4 (b) shown.

[0064] After removing the edge point cloud on the left and right sides of the calibration plate, S104, the RANSAC spherical surface fitting algorithm is used to fit the best candidate spherical surface for the edge point cloud of the hollowed-out circular hole, and the spherical center coordinates are calculated.

[0065] Among them, for the extracted edge point cloud, two three-dimensional straight lines are fitted by the RANSAC three-dimensional straight line detection algorithm, and according to the distance information of the point to the three-dimensional straight line, the edge point cloud on the left and right sides of the calibration plate is removed, and the edge point cloud of the internal hollowed-out circle is retained. As shown in FIG. 8. Figure 4 c shown.

[0066] Four best candidate spherical surfaces are fitted for the edge point cloud of the hollowed-out circular hole by using the RANSAC spherical surface fitting algorithm, and four spherical center coordinates are calculated. According to the prior information (the radius of the four hollowed-out circles of the calibration plate is 20 cm, and the relative distance of the four circle centers), it is judged whether the spherical surface fitting result is valid, if valid, continue to execute, otherwise, the point cloud extraction, spherical surface fitting and spherical center calculation and other operations are performed on other frames of point cloud.

[0067] S105, according to the three-dimensional coordinates of the center of the calibration plate, the known physical size information of the calibration plate is used to calculate the coordinates of the four corner points of the calibration plate in the laser radar coordinate system;

[0068] According to the three-dimensional coordinates of the center of the fitted spherical body in the calibration plate, combined with the known size information (length, width and height information) of the calibration plate, the coordinates of the four corner points of the calibration plate in the laser radar coordinate system are calculated.

[0069] S106, the absolute pose in the IMU coordinate system corresponding to the time stamp of the current frame of point cloud is calculated, and all WGS-84 control point coordinates are converted to the IMU coordinate system according to the absolute pose;

[0070] Specifically, the absolute pose in the IMU coordinate system corresponding to the time stamp of the current frame of point cloud is calculated by linear interpolation, that is, the vehicle pose corresponding to the current frame of point cloud is determined.

[0071] According to the absolute pose of the vehicle, the WGS-84 control point coordinates (geocentric coordinate system) are converted to the IMU (inertial measurement unit) coordinate system. The WGS-84 control point is the WGS-84 coordinate of the corner point of the calibration plate measured by the total station.

[0072] S107, set the initial value of the external parameter of the laser radar and the IMU, convert the coordinates of the four corner points of the calibration board from the laser radar coordinate system to the IMU coordinate system, find the control points corresponding to the four corner points of the calibration board based on the KNN nearest neighbor matching algorithm, and form the homonymous point pairs;

[0073] Reasonably set the initial value of the external parameter of the laser radar and the IMU, and convert the coordinates of the corner points of the calibration board from the laser radar coordinate system to the IMU coordinate system.

[0074] The KNN nearest neighbor matching algorithm is used to find the control points corresponding to the four corner points of the calibration board, and form the homonymous point pairs (lidar_axis_xyz, imu_axis_xyz) (that is, the coordinates of the same point in the LiDAR coordinate system and the IMU coordinate system). The KNN (K-Nearest Neighbor) nearest neighbor matching algorithm is a classification algorithm, each sample can be represented by its K nearest neighbors, and the control points can be determined by KNN.

[0075] S108, a certain number of homonymous point pairs are obtained, and the external parameters of the laser radar and the IMU are obtained by solving a plurality of homonymous point pairs by the Gauss-Newton method.

[0076] For all frame point cloud data obtained on the vehicle collection route, the corresponding homonymous point pairs can be calculated, and a plurality of homonymous point pairs can be obtained. The external parameters of the laser radar and the IMU are obtained by solving a plurality of homonymous point pairs by the improved Gauss-Newton method, and the external parameter calibration is performed. The improved Gauss-Newton method can refer to the article "Nonlinear Three-Dimensional Rectangular Coordinate Conversion Method Based on Improved Gauss-Newton Method" (Lu Changlin et al., Wuhan University School of Geomatics).

[0077] In order to avoid the problem of iterative divergence caused by the strong dependence of the external parameter initial value in the similar ICP registration algorithm, the improved Gauss-Newton method is used to solve the external parameters of the laser radar and the IMU, which can achieve high calibration accuracy without depending on the external parameter initial value.

[0078] In this embodiment, a round hole calibration board is designed, that is, four symmetrical round holes are dug on the plane of the calibration board as obvious features visible to the laser radar. The coordinates of the four corner points of the calibration board in the laser radar coordinate system are automatically calculated, the absolute pose information of the vehicle-mounted GPS / IMU is combined, the homonymous point pairing with the WGS-84 control points on the calibration board is realized, and finally the external parameter matrix of the laser radar and the IMU is solved by the improved Gauss-Newton method. The automatic calibration of the external parameters of the laser radar and the IMU can be realized, the calibration efficiency is improved, the requirements for the target corner points and the site are reduced, the calibration process is simplified, and the calibration accuracy is guaranteed.

[0079] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0080] Figure 5 A structure diagram of a vehicle-mounted multi-line laser radar and IMU external parameter automatic calibration device provided by an embodiment of the present application is shown in the figure, and the device comprises:

[0081] The data acquisition module 510 is configured to collect laser radar raw point cloud data and vehicle absolute pose of the calibration board in the calibration field along a predetermined route.

[0082] The clustering separation module 520 is configured to extract point cloud data of a region of interest in the raw point cloud data, and separate spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance clustering.

[0083] The clustering separation module 520 comprises:

[0084] The region point cloud extraction module is configured to segment ground point cloud and non-ground point cloud by using a CSF algorithm on single-frame point cloud data, filter the non-ground point cloud based on prior information, and extract point cloud data of a region of interest.

[0085] The edge point cloud extraction module 530 is configured to extract plane point cloud from the target cluster by using a RANSAC plane detection algorithm, and extract edge point cloud according to the discontinuity of depth information of radar echo points, wherein the edge point cloud is edge points of hollowed-out circles on the calibration board and edge points on the left and right sides of the calibration board.

[0086] The spherical surface fitting module 540 is configured to remove the edge point cloud on the left and right sides of the calibration board, fit the edge point cloud of the hollowed-out circle hole with the best candidate spherical surface by using a RANSAC spherical surface fitting algorithm, and calculate the spherical center coordinates.

[0087] The removal of the edge point cloud on the left and right sides of the calibration board comprises:

[0088] The edge point cloud is fitted with a three-dimensional straight line by using a RANSAC three-dimensional straight line detection algorithm, and the edge point cloud on the left and right sides of the calibration board is removed according to the distance information of the points to the three-dimensional straight line, and the edge point cloud of the internal hollowed-out circle is retained.

[0089] The corner point coordinate calculation module 550 is configured to calculate the coordinates of four corner points of the calibration board in the laser radar coordinate system according to the three-dimensional coordinates of the inner circle center of the calibration board and the known physical size information of the calibration board.

[0090] The coordinate conversion module 560 is configured to calculate the absolute pose in the IMU coordinate system corresponding to the point cloud timestamp of the current frame, and convert all WGS-84 control point coordinates to the IMU coordinate system according to the absolute pose.

[0091] The absolute pose in the IMU coordinate system corresponding to the point cloud timestamp of the current frame is calculated by linear interpolation of the poses.

[0092] The control point acquisition module 570 is configured to set the initial value of the extrinsic parameter of the laser radar and the IMU, convert the coordinates of the four corner points of the calibration board from the laser radar coordinate system to the IMU coordinate system, find the control points corresponding to the four corner points of the calibration board based on the KNN nearest neighbor matching algorithm, and form homonymous point pairs.

[0093] The calibration module 580 is configured to obtain a certain number of homonymous point pairs, and solve the extrinsic parameter of the laser radar and the IMU by the Gauss-Newton method based on the multiple homonymous point pairs.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and module can refer to the corresponding process in the foregoing method embodiment, and will not be described here.

[0095] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device is used for automatic calibration of the extrinsic parameter of a vehicle-mounted multi-line laser radar and an IMU. As shown in the figure, Figure 6 The electronic device 6 of this embodiment includes a memory 610, a processor 420 and a system bus 630, the memory 610 includes a program 6101 stored thereon which can be run, and those skilled in the art can understand that, Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0096] The following will be specifically introduced in combination with Figure 6 The various constituent components of the electronic device will be specifically introduced as follows:

[0097] The memory 610 can be used to store software programs and modules, and the processor 420 executes various function applications and data processing of the electronic device by running the software programs and modules stored in the memory 610. The memory 610 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data (such as cache data) created according to the use of the electronic device, etc. In addition, the memory 610 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage device.

[0098] The executable program 6101 containing the network request method on the memory 610 can be divided into one or more modules / units stored in the memory 610 and executed by the processor 6420 to realize automatic calibration of laser radar and IMU external parameters, etc. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 6101 in the electronic device 6. For example, the computer program 6101 can be divided into a data acquisition module, a clustering separation module, an edge point cloud extraction module, a spherical fitting module, etc.

[0099] The processor 620 is the control center of the electronic device, connects all parts of the electronic device through various interfaces and lines, executes the software programs and / or modules stored in the memory 610 and calls the data stored in the memory 610, executes various functions and processes data of the electronic device, and thus monitors the overall state of the electronic device. Optionally, the processor 620 can include one or more processing units; preferably, the processor 620 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 620.

[0100] The system bus 630 is used to connect the internal functional components of the computer, and can transmit data information, address information, and control information, and its type can be, for example, a PCI bus, an ISA bus, a VESA bus, etc. The instructions of the processor 620 are transmitted to the memory 610 through the bus, and the memory 610 feeds back data to the processor 620, and the system bus 630 is responsible for the data and instruction interaction between the processor 620 and the memory 610. Of course, the system bus 630 can also access other devices, such as a network interface, a display device, etc.

[0101] In the embodiment of the present application, the executable program executed by the processing 620 included in the electronic device comprises:

[0102] Collecting laser radar original point cloud data and vehicle absolute position of the calibration board in the calibration field along a predetermined route;

[0103] Extracting point cloud data of a region of interest in the original point cloud data, and separating spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance clustering;

[0104] Extracting plane point cloud from the target cluster by a RANSAC plane detection algorithm, and extracting edge point cloud according to the discontinuity of the depth information of the radar echo point, the edge point cloud being edge points of the hollowed-out circle on the calibration board and edge points on the left and right sides of the calibration board;

[0105] After removing the edge point cloud on the left and right sides of the calibration board, a RANSAC sphere fitting algorithm is used to fit the edge point cloud of the hollowed-out circle hole to obtain a best candidate sphere, and the center coordinates of the sphere are calculated;

[0106] According to the three-dimensional coordinates of the center of the inner circle of the calibration board, the physical size information of the calibration board is used to calculate the coordinates of the four corner points of the calibration board in the laser radar coordinate system;

[0107] An absolute position in the IMU coordinate system corresponding to the time stamp of the current frame point cloud is calculated, and all WGS-84 control point coordinates are converted to the IMU coordinate system according to the absolute position;

[0108] The initial values of the extrinsic parameters of the laser radar and the IMU are set, the four corner point coordinates of the calibration board are converted from the laser radar coordinate system to the IMU coordinate system, and based on the KNN nearest neighbor matching algorithm, the control points corresponding to the four corner points of the calibration board are found to form homonymous point pairs;

[0109] A certain number of homonymous point pairs are obtained, and the extrinsic parameters of the laser radar and the IMU are solved by the Gauss-Newton method based on multiple homonymous point pairs.

[0110] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0111] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatic calibration of extrinsic parameters of a vehicle-mounted multi-line lidar and IMU, characterized in that, include: In the calibration field, raw point cloud data of the lidar on the calibration board and the absolute pose of the vehicle are collected along a predetermined route; The calibration plate is fixed on the bracket, and the plane of the calibration plate is perpendicular to the ground. The calibration plate has four hollow circular holes with the same radius. The two centers of the holes on the same diagonal divide the diagonal of the calibration plate into four equal parts. Extract the point cloud data of the region of interest from the original point cloud data, and separate the spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance clustering; Planar point clouds are extracted from target clusters using the RANSAC planar detection algorithm, and edge point clouds are extracted based on the discontinuity of depth information of radar echo points. The edge point clouds are the edge points of the hollow circle on the calibration plate and the edge points on the left and right sides of the calibration plate. After removing the edge point clouds on the left and right sides of the calibration plate, the RANSAC spherical fitting algorithm is used to fit the best candidate sphere to the edge point cloud of the hollow circular hole, and the coordinates of the sphere center are calculated. Based on the three-dimensional coordinates of the center of the calibration plate, and using the known physical dimensions of the calibration plate, the coordinates of the four corner points of the calibration plate in the lidar coordinate system are calculated. Calculate the absolute pose in the IMU coordinate system corresponding to the current frame point cloud timestamp, and transform the coordinates of all WGS-84 control points to the IMU coordinate system based on the absolute pose; Among them, the absolute pose in the IMU coordinate system corresponding to the current frame point cloud timestamp is calculated by using pose linear interpolation; Set the initial values ​​of the external parameters of the lidar and IMU, transform the coordinates of the four corner points of the calibration board from the lidar coordinate system to the IMU coordinate system, and find the control points corresponding to the four corner points of the calibration board based on the KNN nearest neighbor matching algorithm to form pairs of points with the same name. A certain number of pairs of identical points are obtained, and the external parameters of the laser radar to the IMU are obtained by solving multiple pairs of identical points using the Gauss-Newton method.

2. The method according to claim 1, characterized in that, Before collecting the raw point cloud data of the lidar on the calibration board in the calibration field along a predetermined route, the following steps are included: A certain number of calibration plates are placed on the vehicle's driving route, and there is at least one frame of laser point cloud data, with more than two laser beams capable of scanning the edge of each hollow circle on the calibration plate. After the calibration boards are deployed, the WGS-84 coordinates of the four corner points of all calibration boards are measured using a total station as the coordinates of the control points.

3. The method according to claim 1, characterized in that, The process of collecting raw point cloud data of the lidar on the calibration board along a predetermined route in the calibration field also includes: Point cloud data is timed using PPS and GPRMC.

4. The method according to claim 1, characterized in that, The extraction of point cloud data from the region of interest in the original point cloud data includes: The CSF algorithm is used to segment ground point cloud and non-ground point cloud from single frame point cloud data; Filtering of non-ground point clouds based on prior information to extract point cloud data of regions of interest.

5. The method according to claim 1, characterized in that, The removal of edge point clouds from the left and right sides of the calibration board includes: The RANSAC 3D line detection algorithm is used to fit 3D lines to the edge point clouds. Based on the distance information from the points to the 3D lines, the edge point clouds on the left and right sides of the calibration plate are removed, while the edge point clouds of the inner hollow circle are retained.

6. An automatic calibration system for external parameters of a vehicle-mounted multi-line lidar and IMU, characterized in that, include: The data acquisition module is used to collect raw point cloud data of the lidar on the calibration board and the absolute pose of the vehicle in the calibration field along a predetermined route; The calibration plate is fixed on the bracket, and the plane of the calibration plate is perpendicular to the ground. The calibration plate has four hollow circular holes with the same radius. The two centers of the holes on the same diagonal divide the diagonal of the calibration plate into four equal parts. The clustering separation module is used to extract the point cloud data of the region of interest from the original point cloud data, and to separate the spatially independent target clusters in the point cloud data of the region of interest based on Euclidean distance. The edge point cloud extraction module is used to extract planar point clouds from target clusters using the RANSAC planar detection algorithm, and to extract edge point clouds based on the discontinuity of depth information of radar echo points. The edge point clouds are the edge points of the hollow circle on the calibration plate and the edge points on the left and right sides of the calibration plate. The spherical fitting module is used to remove the edge point clouds on the left and right sides of the calibration board, and then use the RANSAC spherical fitting algorithm to fit the best candidate sphere to the edge point cloud of the hollowed-out circular hole, and calculate the coordinates of the sphere center. The corner coordinate calculation module is used to calculate the coordinates of the four corner points of the calibration plate in the lidar coordinate system based on the three-dimensional coordinates of the center of the inner circle of the calibration plate and the known physical size information of the calibration plate. The coordinate transformation module is used to calculate the absolute pose in the IMU coordinate system corresponding to the timestamp of the current frame point cloud, and transform the coordinates of all WGS-84 control points to the IMU coordinate system based on the absolute pose. Among them, the absolute pose in the IMU coordinate system corresponding to the current frame point cloud timestamp is calculated by using pose linear interpolation; The control point acquisition module is used to set the initial values ​​of the external parameters of the lidar and IMU, transform the coordinates of the four corner points of the calibration board from the lidar coordinate system to the IMU coordinate system, and find the control points corresponding to the four corner points of the calibration board based on the KNN nearest neighbor matching algorithm to form pairs of points with the same name. The calibration module is used to obtain a certain number of pairs of corresponding points and solve multiple pairs of corresponding points using the Gauss-Newton method to obtain the external parameters of the laser radar to the IMU.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the automatic calibration method for extrinsic parameters of a vehicle-mounted multi-line lidar and IMU as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of the automatic calibration method for extrinsic parameters of a vehicle-mounted multi-line lidar and IMU as described in any one of claims 1 to 5.

Images