Multi-lidar joint calibration device and method

Abstract

The application relates to a multi-laser-radar joint calibration device and method; a multi-laser-radar joint calibration method, which comprises the following steps: step 1, obtaining initial point cloud data by scanning through multiple laser radars respectively; step 2, based on the spatial positions of the multiple laser radars, converting the initial point cloud data of the multiple laser radars to a main coordinate system; step 3, based on the field of view overlap of the multiple laser radars, dividing the point cloud data of each laser radar into non-overlapping regions and several overlapping regions; step 4, directly selecting the point cloud data in each non-overlapping region as the final data of the region in the main coordinate system; and step 5, performing superposition processing on the point cloud data in each overlapping region. The application has the beneficial effect of high calibration accuracy. The processing of the overlapping regions improves the calibration accuracy. The ground removal processing of the initial data reduces the data processing amount in subsequent processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of laser radar calibration, in particular to a multi-laser radar joint calibration method and device. BACKGROUND

[0002] In the process of driving, the existing automatic driving usually uses laser radar to effectively identify the obstacles around the vehicle. However, since each laser radar has its own independent coordinate system, the coordinate systems of multiple laser radars need to be unified and calibrated during joint use. The precision of traditional multi-laser radar joint calibration is relatively low.

[0003] In addition, CN202110926221.1 provides a multi-laser radar joint calibration method based on non-overlapping field of view, which cannot solve the joint calibration processing of overlapping fields of view and cannot improve the calibration precision based on overlapping fields of view. SUMMARY

[0004] The present application aims to provide a multi-laser radar joint calibration device and method to solve the problems raised in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0006] A multi-laser radar joint calibration method, comprising the following steps:

[0007] Step 1: obtaining initial point cloud data by scanning through multiple laser radars respectively;

[0008] Step 2: based on the spatial positions of multiple laser radars, converting the initial point cloud data of multiple laser radars to a master coordinate system;

[0009] Step 3: based on the overlapping fields of view of multiple laser radars, dividing the point cloud data of each laser radar into non-overlapping regions and several overlapping regions;

[0010] Step 4: directly selecting the point cloud data in each non-overlapping region as the final data of the region in the master coordinate system;

[0011] Step 5: performing superposition processing on the point cloud data in each overlapping region;

[0012] Step 6: splicing each non-overlapping region and the overlapping region after superposition processing to obtain complete composite point cloud data based on the master coordinate system;

[0013] Wherein, the method for performing superposition processing on the point cloud data of each overlapping region comprises the following steps:

[0014] Step 5.1, for the point cloud data of multiple laser radars in the overlapping area, by analyzing the point cloud data, the laser radar with the minimum distance to the nearest object in the overlapping area is selected from the multiple laser radars as the main laser radar of the area, and the other laser radars in the area are the supplementary laser radars;

[0015] Step 5.2, the point cloud data of the main laser radar is the basic data of the overlapping area in the main coordinate system, and the point cloud data of the supplementary laser radar is the supplementary data of the overlapping area in the main coordinate system for optimizing the data of the object in the overlapping area;

[0016] Step 5.3, steps 5.1 and 5.2 are performed for each overlapping area to complete the superposition processing of all overlapping areas.

[0017] As a further scheme of the present application: after obtaining the initial point cloud data of multiple laser radars, first perform ground removal processing, and then convert the initial point cloud data to the main coordinate system.

[0018] As a further scheme of the present application: the ground removal processing of the initial point cloud data includes the following steps:

[0019] Step 1, divide the initial point cloud data into multiple grids;

[0020] Step 2, calculate the average height of the initial points in each grid;

[0021] Step 3, calculate the maximum height difference of the initial points in each grid within a preset range; if the maximum height difference is greater than a preset threshold, all initial points in the grid are identified as non-ground point clouds; if the maximum height difference is not greater than the preset threshold, all initial points in the grid are identified as ground point clouds;

[0022] Step 4, delete the ground point clouds in all initial point cloud data.

[0023] As a further scheme of the present application: the joint calibration method of multiple laser radars further includes the following steps:

[0024] Step 7, convert the composite point cloud data into an aerial view image;

[0025] The method for converting the composite point cloud data into an aerial view image includes the following steps:

[0026] Step 7.1, convert the composite point cloud data in the main coordinate system into a three-dimensional image of the composite point cloud data;

[0027] Step 7.2, project the three-dimensional image of the composite point cloud data into a two-dimensional plane as a two-dimensional image of the composite point cloud data;

[0028] Step 7.3, based on the spatial positions of the multiple laser radars, positioning the position of the vehicle itself in a two-dimensional plane;

[0029] Step 7.4, merging the vehicle overhead picture and the two-dimensional image of the composite point cloud data to generate a two-dimensional display image.

[0030] As a further aspect of the present application, before the vehicle overhead picture and the two-dimensional image of the composite point cloud data are merged, first, the two-dimensional image of the composite point cloud data is denoised to remove outliers.

[0031] As a further aspect of the present application, the vehicle overhead picture is divided into multiple body regions.

[0032] When the distance between the body region and the external object is less than a preset distance value, the body region is highlighted.

[0033] As a further aspect of the present application, when the distance between the body region and the external object is less than a preset distance value, the method for highlighting the body region includes the following steps:

[0034] Step 1, each body region is defined with multiple contour points;

[0035] Step 2, the coordinate position of each contour point of each body region on the two-dimensional display image is obtained;

[0036] Step 3, the coordinate position of the point on the contour of the two-dimensional image of the composite point cloud data on the two-dimensional display image is obtained;

[0037] Step 4, the distance between the coordinate position of each point on the contour of the two-dimensional image on the two-dimensional display image and the coordinate position of each contour point of each body region on the two-dimensional display image is calculated;

[0038] Step 5, each distance obtained in step 4 is compared with a preset distance value one by one; if the distance obtained in step 4 is less than the preset distance value, the body region in which the contour point of the body region corresponding to the distance is highlighted.

[0039] As a further aspect of the present application, the minimum distance from the highlighted body region to the external object is marked on the two-dimensional display image.

[0040] The minimum distance from the highlighted body region to the external object is displayed in the highlighted body region on the two-dimensional display image.

[0041] As a further aspect of the present application, the calculation method of the minimum distance from the highlighted body region to the external object includes the following steps:

[0042] The distance corresponding to the plurality of contour points of the highlighted vehicle body region is obtained;

[0043] The value of any one distance is selected as the minimum distance value;

[0044] The minimum distance value is compared with the values of other distances in turn, and if the value of other distance is smaller than the minimum distance value, the minimum distance value is updated;

[0045] After the comparison of all distances, the final minimum distance value is taken as the minimum distance from the highlighted vehicle body region to the external object.

[0046] A multi-laser radar joint calibration device adopts the above-mentioned multi-laser radar joint calibration method for joint calibration.

[0047] The present application has the advantages of high calibration accuracy.

[0048] The processing of the overlapping region improves the calibration accuracy. The ground removal processing of the initial data reduces the data processing amount of subsequent processing.

[0049] Other features and advantages of the present application will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of a multi-laser radar joint calibration method of the present application;

[0051] Figure 2 is a flowchart of a method for superimposing the point cloud data of each overlapping region in a multi-laser radar joint calibration method of the present application;

[0052] Figure 3 is a flowchart of the ground removal processing of the initial point cloud data in a multi-laser radar joint calibration method of the present application;

[0053] Figure 4 is a flowchart of a method for converting the composite point cloud data into an aerial view image in a multi-laser radar joint calibration method of the present application;

[0054] Figure 5 is a schematic diagram of the overlapping of the fields of view of a plurality of laser radars in a multi-laser radar joint calibration method of the present application;

[0055] Figure 6 is a schematic diagram of a vehicle overhead view image in a multi-laser radar joint calibration method of the present application, showing a schematic diagram of the highlighted vehicle body region, wherein the solid part identifies the highlighted vehicle body region, and the cross-section line part identifies other vehicle body regions not highlighted. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0057] As shown in the figure, a multi-laser radar joint calibration method comprises the following steps: Figures 1 to 6

[0058] Step 1, respectively obtaining initial point cloud data by scanning through multiple laser radars;

[0059] Step 2, based on the spatial positions of the multiple laser radars, converting the initial point cloud data of the multiple laser radars to a master coordinate system;

[0060] Step 3, based on the field of view overlap of the multiple laser radars, dividing the point cloud data of each laser radar into non-overlapping regions and several overlapping regions;

[0061] Step 4, directly selecting the point cloud data in each non-overlapping region as the final data of the region in the master coordinate system;

[0062] Step 5, performing superposition processing on the point cloud data in each overlapping region;

[0063] Step 6, splicing each non-overlapping region and the overlapping region after superposition processing to obtain complete composite point cloud data based on the master coordinate system;

[0064] Step 7, converting the composite point cloud data into an aerial view image.

[0065] As a specific implementation, for step 3, referring to Figure 5 , three laser radars are divided into six regions A, B, C, D, E and F. Among them, A, D and F are non-overlapping regions, and B, C and E are overlapping regions.

[0066] As a specific implementation, for step 5, the method of superposition processing on the point cloud data in each overlapping region comprises the following steps:

[0067] Step 5.1, for the point cloud data of multiple laser radars in the overlapping region, by analyzing the point cloud data, selecting the laser radar with the minimum distance to the nearest object in the overlapping region from the multiple laser radars as the master laser radar of the region, and the other laser radars in the region as supplementary laser radars;

[0068] ​Step 5.2, the point cloud data of the main lidar is taken as the basic data of the overlapping area in the main coordinate system, and the point cloud data of the supplementary lidar is taken as the supplementary data of the overlapping area in the main coordinate system for optimizing the data of the object in the overlapping area;

[0069] Step 5.3, steps 5.1 and 5.2 are respectively performed for each overlapping area to complete the superposition processing of all overlapping areas.

[0070] As a preferred embodiment, after obtaining the initial point cloud data of the plurality of lidars, the ground removal processing is first performed, and then the initial point cloud data is converted into the main coordinate system. Through the ground removal processing, the data amount can be reduced, and the efficiency of subsequent data processing can be improved.

[0071] Specifically, the ground removal processing of the initial point cloud data includes the following steps:

[0072] Step 1, the initial point cloud data is divided into a plurality of grids;

[0073] Step 2, the average height of the initial points in each grid is calculated;

[0074] Step 3, the maximum height difference of the initial points in each grid within a preset range is calculated; if the maximum height difference is greater than a preset threshold, all the initial points in the grid are identified as non-ground point clouds; if the maximum height difference is not greater than the preset threshold, all the initial points in the grid are identified as ground point clouds;

[0075] Step 4, the ground point clouds in all the initial point cloud data are deleted.

[0076] As a specific embodiment, for step 7, the method for converting the composite point cloud data into the bird's eye view image includes the following steps:

[0077] Step 7.1, the composite point cloud data is converted into a three-dimensional image of the composite point cloud data in the main coordinate system;

[0078] Step 7.2, the three-dimensional image of the composite point cloud data is projected into a two-dimensional plane as a two-dimensional image of the composite point cloud data;

[0079] Step 7.3, the position of the vehicle itself is located in the two-dimensional plane based on the spatial positions of the plurality of lidars;

[0080] Step 7.4, the bird's eye view picture of the vehicle is placed in the two-dimensional plane, and the two-dimensional image of the composite point cloud data is merged to generate a two-dimensional display image.

[0081] As a specific embodiment, before the bird's eye view picture of the vehicle and the two-dimensional image of the composite point cloud data are merged, the two-dimensional image of the composite point cloud data is first subjected to noise reduction processing to remove the noise points.

[0082] As a specific embodiment, the vehicle top view image is divided into multiple body regions; when the distance between a body region and the external object is less than a preset distance value, the body region is highlighted. Referring to Figure 6 , the block represents the external object, and the solid section represents the highlight.

[0083] As a specific embodiment, the method of highlighting a body region when the distance between the body region and the external object is less than a preset distance value includes the following steps:

[0084] Step 1: Each body region is defined with multiple contour points;

[0085] Step 2: Obtain the coordinate position of each contour point of each body region on the two-dimensional display image;

[0086] Step 3: Obtain the coordinate position of the point on the contour of the two-dimensional image that meets the point cloud data on the two-dimensional display image;

[0087] Step 4: Calculate the distance between the coordinate position of each point on the contour of the two-dimensional image on the two-dimensional display image and the coordinate position of each contour point of each body region on the two-dimensional display image;

[0088] Step 5: Compare each distance obtained in Step 4 with a preset distance value one by one; if the distance obtained in Step 4 is less than the preset distance value, highlight the body region in which the contour point corresponding to the distance is located.

[0089] As a specific embodiment, the minimum distance from the highlighted body region to the external object is marked on the two-dimensional display image;

[0090] The minimum distance from the highlighted body region to the external object is displayed as distance data within the highlighted body region on the two-dimensional display image.

[0091] Not only the area that may have a close collision is displayed by highlighting, but also the distance is displayed by numerical value.

[0092] As a specific embodiment, the calculation method of the minimum distance from the highlighted body region to the external object includes the following steps:

[0093] Obtain the calculated distance corresponding to the multiple contour points of the highlighted body region;

[0094] Select the value of any one distance as the minimum distance;

[0095] The distance minimum value is compared with the values of other distances in turn, and if the value of other distance is less than the distance minimum value, the distance minimum value is updated;

[0096] After comparison of all distances, the final distance minimum value is taken as the minimum distance from the highlighted vehicle body region to the external object.

[0097] A multi-laser radar joint calibration device adopts the multi-laser radar joint calibration method to perform joint calibration.

[0098] It is apparent for a person skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, the scope of the present application being defined by the appended claims rather than the above description, and it is intended to embrace all changes and modifications that fall within the meaning and scope of the equivalent elements of the claims. Any reference signs in the claims should not be considered as limiting the claims involved.

[0099] Furthermore, it should be understood that, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the present specification is described in this way only for the sake of clarity, and a person skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be combined appropriately to form other embodiments that can be understood by a person skilled in the art.

Claims

1. A method for joint calibration of multiple lidars, characterized in that, The method comprises the following steps: Step 1, obtaining initial point cloud data by scanning through multiple laser radars respectively; Step 2, based on the spatial positions of the multiple laser radars, converting the initial point cloud data of the multiple laser radars to a master coordinate system; Step 3, based on the field of view overlap of the multiple laser radars, dividing the point cloud data of each laser radar into non-overlapping regions and several overlapping regions; Step 4, directly selecting the point cloud data in each non-overlapping region as the final data of the non-overlapping region in the master coordinate system; Step 5, performing superposition processing on the point cloud data in each overlapping region; Step 6, splicing each non-overlapping region and the overlapping region after superposition processing to obtain complete composite point cloud data based on the master coordinate system; The method for performing superposition processing on the point cloud data of each overlapping region comprises the following steps: Step 5.1, for the point cloud data of the multiple laser radars in the overlapping region, selecting the laser radar with the minimum distance to the nearest object in the overlapping region from the multiple laser radars as the master laser radar of the overlapping region through analysis of the point cloud data, and the other laser radars in the overlapping region are supplementary laser radars; Step 5.2, the point cloud data of the master laser radar is the basic data of the overlapping region in the master coordinate system, and the point cloud data of the supplementary laser radar is the supplementary data of the overlapping region in the master coordinate system for optimizing the data of the object in the overlapping region; Step 5.3, performing steps 5.1 and 5.2 on each overlapping region to complete the superposition processing of all overlapping regions.

2. The joint calibration method of multiple laser radars according to claim 1, wherein After obtaining the initial point cloud data of the multiple laser radars, first perform ground removal processing and then convert the initial point cloud data to the master coordinate system.

3. The joint calibration method of multiple laser radars according to claim 2, wherein The ground removal processing of the initial point cloud data comprises the following steps: Step 1, dividing the initial point cloud data into multiple grids; Step 2, calculating the average height of the initial points in each grid; Step 3, calculating the maximum height difference of the points in the preset range from the average height of the initial points in each grid; if the maximum height difference is greater than the preset threshold, all the initial points in the grid are identified as non-ground point clouds; if the maximum height difference is not greater than the preset threshold, all the initial points in the grid are identified as ground point clouds; Step 4, deleting the ground point clouds from all the initial point cloud data.

4. The joint calibration method of multiple laser radars according to claim 1, wherein The joint calibration method of multiple laser radars further comprises the following steps: Step 7, converting the composite point cloud data into an aerial view image; The method for converting the composite point cloud data into an aerial view image comprises the following steps: Step 7.1, converting the composite point cloud data in the master coordinate system into a three-dimensional image of the composite point cloud data; Step 7.2, projecting the three-dimensional image of the composite point cloud data onto a two-dimensional plane as a two-dimensional image of the composite point cloud data; Step 7.3, based on the spatial positions of the multiple laser radars, positioning the position of the vehicle itself in the two-dimensional plane; Step 7.4, merging the vehicle overhead picture and the two-dimensional image of the composite point cloud data to generate a two-dimensional display image.

5. The joint calibration method of multiple laser radars according to claim 4, wherein, Before merging the vehicle overhead picture and the two-dimensional image of the composite point cloud data, the two-dimensional image of the composite point cloud data is first processed to remove noise and outliers.

6. The joint calibration method of multiple laser radars according to claim 4, wherein, The vehicle overhead picture is divided into multiple body regions; When the distance between a body region and an external object is less than a preset distance value, the body region is highlighted.

7. The joint calibration method of multiple laser radars according to claim 6, wherein, When the distance between a body region and an external object is less than a preset distance value, the body region is highlighted by the following steps: Step 1, each body region is defined with multiple contour points; Step 2, the coordinate position of each contour point of each body region on the two-dimensional display image is obtained; Step 3, the coordinate position of each point on the contour of the two-dimensional image of the composite point cloud data on the two-dimensional display image is obtained; Step 4, the distance between the coordinate position of each point on the contour of the two-dimensional image of the composite point cloud data on the two-dimensional display image and the coordinate position of each contour point of each body region on the two-dimensional display image is calculated; Step 5, each distance obtained in step 4 is compared with the preset distance value one by one; if the distance obtained in step 4 is less than the preset distance value, the body region where the contour point corresponding to the distance is located is highlighted.

8. The joint calibration method of multiple laser radars according to claim 7, wherein, The minimum distance from the highlighted body region to the external object is identified on the two-dimensional display image; The minimum distance from the highlighted body region to the external object is displayed within the highlighted body region on the two-dimensional display image.

9. The joint calibration method of multiple laser radars according to claim 8, wherein, The calculation method of the minimum distance from the highlighted body region to the external object includes the following steps: The distances corresponding to the multiple contour points of the highlighted body region are obtained; An arbitrary distance value is selected as the minimum distance; The minimum distance is compared with the values of other distances one by one; if the value of another distance is less than the minimum distance, the minimum distance is updated; After the comparison of all distances, the final minimum distance is taken as the minimum distance from the highlighted body region to the external object.

10. A multi-lidar joint calibration apparatus, characterized by, The joint calibration method of multiple laser radars according to any one of claims 1 to 9 is used for joint calibration.

Images