A multi-laser radar calibration method, device and electronic equipment

Abstract

The application discloses a multi-laser radar calibration method, device and electronic equipment, wherein the multi-laser radar calibration method can be applied to the field of vehicle-road cooperation automatic driving, and the method comprises the following steps: acquiring a point cloud map and a point cloud image of each to-be-calibrated road end laser radar; labeling a feature point set based on the point cloud map; determining a target point set corresponding to the feature point set in each point cloud image, so as to obtain a target point set corresponding to each to-be-calibrated road end laser radar under an initial pose; for each to-be-calibrated road end laser radar, determining an optimized pose of the road end laser radar relative to the point cloud map based on the target point set under the corresponding initial pose; and determining external parameter parameters between the plurality of to-be-calibrated road end laser radars based on the optimized poses of the to-be-calibrated road end laser radars relative to the point cloud map. The application can well control the precision of the external parameters of the multi-laser radars, and can quickly realize the calibration requirements of a large number of road end laser radars in intelligent traffic road end applications.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a multi-laser radar calibration method and device and electronic equipment. BACKGROUND

[0002] Laser radar is a commonly used sensor in the field of V2X (Vehicle to Everything), which can provide semantic information, and its calibration is particularly important for the perception function of the V2X system. The traditional calibration method often uses a calibration board to calibrate the laser radar, which requires multiple laser radars to observe the calibration board at the same time, and has a large common viewing area. However, the road end laser radars are generally placed high and cannot be moved, and their beam distribution is relatively sparse, so they often do not have a large common viewing area, and therefore it is difficult to calibrate their external parameters using the traditional method. SUMMARY

[0003] In order to solve the problems of the prior art, the embodiments of the present application provide a multi-laser radar calibration method, device, electronic equipment and storage medium. The technical solution is as follows:

[0004] In one aspect, a multi-laser radar calibration method is provided, and the method comprises:

[0005] Obtaining a point cloud map and a point cloud image of each of a plurality of road end laser radars to be calibrated, wherein the point cloud map is obtained based on first point cloud data collected by a laser radar on a collection vehicle within a preset range, and the preset range includes the scanning range of the plurality of road end laser radars to be calibrated, and the point cloud image of each road end laser radar to be calibrated is an image generated based on second point cloud data collected by the corresponding road end laser radar to be calibrated;

[0006] Labeling a feature point set based on the point cloud map;

[0007] Determining a target point set corresponding to the feature point set in each point cloud image to obtain a corresponding target point set of each road end laser radar to be calibrated at an initial pose;

[0008] For each road end laser radar to be calibrated, determining an optimized pose of the road end laser radar to be calibrated relative to the point cloud map based on the target point set at the corresponding initial pose;

[0009] Determining external parameter parameters between the plurality of road end laser radars to be calibrated based on the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

[0010] In another aspect, a multi-laser radar calibration device is provided, and the device comprises:

[0011] An information obtaining module is configured to obtain a point cloud map and a point cloud image of each of a plurality of to-be-calibrated roadside laser radars; the point cloud map is obtained based on first point cloud data collected by a laser radar on a collection vehicle within a preset range, and the preset range includes scanning ranges of the plurality of to-be-calibrated roadside laser radars; and the point cloud image of each to-be-calibrated roadside laser radar is an image generated based on second point cloud data collected by the corresponding to-be-calibrated roadside laser radar.

[0012] A feature labeling module is configured to label a feature point set based on the point cloud map.

[0013] A point set matching module is configured to determine, in each of the point cloud images, a target point set corresponding to the feature point set, to obtain a target point set corresponding to each to-be-calibrated roadside laser radar in an initial pose.

[0014] An optimization processing module is configured to, for each to-be-calibrated roadside laser radar, determine an optimized pose of the to-be-calibrated roadside laser radar relative to the point cloud map based on the target point set in the corresponding initial pose.

[0015] An external parameter determining module is configured to determine external parameter(s) between the plurality of to-be-calibrated roadside laser radars based on the optimized pose of each to-be-calibrated roadside laser radar relative to the point cloud map.

[0016] In an exemplary embodiment, the feature labeling module comprises:

[0017] A region determining module is configured to determine, on the point cloud map, a map region coinciding with the scanning ranges of the plurality of to-be-calibrated roadside laser radars.

[0018] A facility labeling module is configured to determine a first preset number of road facilities in the map region, and label the road facilities as the feature point set.

[0019] In an exemplary embodiment, the point set matching module comprises:

[0020] A point cloud moving module is configured to, for each of the point cloud images, move the point cloud image so that the point cloud image coincides with the point cloud map, to obtain the initial pose of the to-be-calibrated roadside laser radar.

[0021] A point set labeling module is configured to determine, on each of the point cloud images, a point set coinciding with the feature point set, and label the point set as the target point set corresponding to the to-be-calibrated roadside laser radar corresponding to each of the point cloud images in the initial pose.

[0022] In an exemplary embodiment, the optimization processing module comprises:

[0023] The neighbor point labeling module is configured to label, for each feature point in the feature point set, a second preset number of nearest neighbor points corresponding to the feature point on the point cloud map; the nearest neighbor points are points on the point cloud map that are within a first preset number in ascending order of distance from the corresponding feature point;

[0024] The distance determination module is configured to determine, for each target point in a target point set corresponding to each to-be-calibrated road end laser radar in an initial pose, a distance between the target point and a second preset number of nearest neighbor points corresponding to a target feature point, to obtain a target distance corresponding to the target point; the target feature point refers to a feature point in the feature point set corresponding to the target point;

[0025] The distance optimization module is configured to perform optimization processing on the target distances of the target points in the target point set corresponding to each to-be-calibrated road end laser radar in the initial pose, to obtain an optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map.

[0026] In an exemplary embodiment, the distance determination module comprises:

[0027] The neighbor point fitting module is configured to fit the second preset number of nearest neighbor points corresponding to the target feature point according to a preset geometric element, to obtain a fitting result corresponding to the target point; the preset geometric element includes a straight line or a plane.

[0028] The first determination module is configured to determine a distance between the target point and the fitting result, to obtain a target distance corresponding to the target point.

[0029] In an exemplary embodiment, the distance optimization module comprises:

[0030] The distance summation module is configured to determine, for each to-be-calibrated road end laser radar, a sum of target distances of the target points in the target point set corresponding to the to-be-calibrated road end laser radar in the corresponding initial pose, to obtain a target optimized distance corresponding to the to-be-calibrated road end laser radar.

[0031] The pose adjustment module is configured to adjust the initial pose of the to-be-calibrated road end laser radar, update the target optimized distance corresponding to the to-be-calibrated road end laser radar based on the target point set corresponding to the to-be-calibrated road end laser radar in the adjusted initial pose, and end the adjustment of the initial pose of the to-be-calibrated road end laser radar when the updated target optimized distance is less than a preset distance threshold.

[0032] The pose determination module is configured to determine the initial pose of the to-be-calibrated road end laser radar after the end of the adjustment as an optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map.

[0033] In an example embodiment, the external parameter determination module comprises:

[0034] a radar determination module configured to determine, among a plurality of road end laser radars to be calibrated, a first road end laser radar and a second road end laser radar to be mutually calibrated;

[0035] a second determination module configured to determine a first optimized pose of the first road end laser radar relative to the point cloud map and a second optimized pose of the second road end laser radar relative to the point cloud map;

[0036] a first coordinate transformation module configured to determine, based on the first optimized pose and the second optimized pose, a coordinate transformation matrix of the first road end laser radar relative to the second road end laser radar as an external parameter of the first road end laser radar;

[0037] a second coordinate transformation module configured to determine, based on the first optimized pose and the second optimized pose, a coordinate transformation matrix of the second road end laser radar relative to the first road end laser radar as an external parameter of the second road end laser radar.

[0038] In an example embodiment, the device further comprises a map making module for making a point cloud map, the map making module comprising:

[0039] a point cloud acquisition module configured to acquire first point cloud data collected by a laser radar on a collection vehicle at a plurality of time instants during movement; the plurality of time instants are time instants separated by a preset time period, and the first point cloud data is point cloud data within the preset range collected by the laser radar on the collection vehicle;

[0040] a map creation module configured to select a first time instant from the plurality of time instants, and determine a current point cloud map based on first point cloud data collected at the first time instant;

[0041] a map update module configured to select a second time instant from the remaining time instants, and update the current point cloud map based on first point cloud data collected at the second time instant; the remaining time instants refer to time instants that are not selected from the plurality of time instants;

[0042] a map optimization module configured to, in a case where a coverage degree of the updated current point cloud map and the preset range does not reach a preset coverage degree, repeat the steps of selecting a second time instant from the remaining time instants, and updating the current point cloud map based on first point cloud data collected at the second time instant, until the coverage degree of the updated current point cloud map and the preset range reaches the preset coverage degree;

[0043] a map determination module configured to take the updated current point cloud map when the preset coverage degree is reached as the point cloud map.

[0044] In another aspect, an electronic device is provided, including a processor and a memory having stored therein at least one instruction or at least one program, which is loaded and executed by the processor to implement the multi-lidar calibration method of any of the above aspects.

[0045] In another aspect, a computer-readable storage medium is provided, having stored therein at least one instruction or at least one program, which is loaded and executed by a processor to implement the multi-lidar calibration method of any of the above aspects.

[0046] In another aspect, a computer program product or computer program is provided, including computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the electronic device to perform the multi-lidar calibration method of any of the above aspects.

[0047] The embodiments of the present application create a point cloud map by collecting the laser radars on the vehicle, label the road facilities on the point cloud map, and then match the point cloud of the road end laser radar to be calibrated to the point cloud map, to realize the external parameter calibration of multiple radars. Even if there is no any common view area between the two radars, the external parameter calibration can be completed; at the same time, the accuracy of the external parameter can be well controlled, and almost only depends on the accuracy of the ranging of the radar itself; the calibration demand of a large number of road end laser radars in intelligent traffic road end application can be quickly realized. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 is a flowchart of a multi-lidar calibration method provided by the embodiments of the present application;

[0050] Figure 2 is a flowchart of labeling a feature point set provided by the embodiments of the present application;

[0051] Figure 3 is a flowchart of labeling a target point set provided by the embodiments of the present application;

[0052] Figure 4 is a flowchart of optimization processing provided by the embodiments of the present application;

[0053] Figure 5 is a flowchart of a multi-laser radar external parameter calibration process provided by an embodiment of the present application;

[0054] Figure 6 is a flowchart of a point cloud map making process provided by an embodiment of the present application;

[0055] Figure 7 is a structural block diagram of a multi-laser radar calibration device provided by an embodiment of the present application;

[0056] Figure 8 is a hardware structural block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily mean a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0059] It can be understood that in the specific embodiments of the present application, data related to user information is involved, and when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards in relevant countries and regions.

[0060] Please refer to Figure 1As shown in the flowchart, the method for calibrating multiple laser radars according to the embodiments of the present application is provided. It should be noted that the method steps provided in the specification as described in the embodiments or flowcharts can include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many execution orders, and does not represent the only execution order. In actual system or product execution, the method order can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) as shown in the embodiments or the drawings. Specifically as Figure 1 As shown in the flowchart, the method can include:

[0061] S101, acquiring a point cloud map and a point cloud image of each of the multiple to-be-calibrated road end laser radars.

[0062] The point cloud map is obtained based on first point cloud data collected by a laser radar on a collection vehicle within a preset range. The first point cloud data is point cloud data of objects within the scanning range of the laser radar on the collection vehicle.

[0063] In a specific implementation, the collection vehicle can move around the multiple to-be-calibrated road end laser radars to collect point cloud data within a preset range of the road end laser radars by the laser radar on the collection vehicle. The point cloud data is used to make a point cloud map, and one point cloud map can be made for the multiple to-be-calibrated road end laser radars, or a point cloud map can be made for each to-be-calibrated road end laser radar.

[0064] The preset range includes the scanning ranges of the multiple to-be-calibrated road end laser radars.

[0065] The point cloud image of each to-be-calibrated road end laser radar is an image generated based on second point cloud data collected by the corresponding to-be-calibrated road end laser radar. The second point cloud data is point cloud data of objects within the scanning range of the to-be-calibrated road end laser radar.

[0066] S103, labeling a feature point set based on the point cloud map.

[0067] The feature point set can include a ground plane point set, a road edge point set, and a lamp pole edge point set. In a specific implementation, if the road edge point set is labeled but the lamp pole edge point set is not labeled, each feature point set should include multiple groups of road edges, and not all of them are parallel to each other, in order to ensure that two independent coordinates in the horizontal plane direction can be uniquely solved.

[0068] In an exemplary embodiment, as shown in the flowchart, Figure 2 The above step S103 can include:

[0069] S201, determining a map region on the point cloud map which coincides with the scanning range of the plurality of road end laser radars to be calibrated.

[0070] The determination of the map region is to ensure that the coinciding region of the point cloud map and the point cloud image of each road end laser radar to be calibrated is selected, i.e., the labeled feature point set should be located in the common view region of the laser radar on the collection vehicle and the road end laser radar to be calibrated.

[0071] S203, determining a first preset number of road facilities in the map region, and labeling as a feature point set.

[0072] The road facilities include ground, road edge, lamp pole, and these road facilities should be located in the common view region of the laser radar on the collection vehicle and the road end laser radar to be calibrated. In a specific implementation, the labeling of the feature point set is completed by manually labeling the road facilities.

[0073] As can be seen from the above technical solutions of the embodiments of the present application, the embodiments of the present application determine the common view region of the laser radar on the collection vehicle and the road end laser radar to be calibrated, label the road facilities in the common view region, and thus ensure that the preliminary extrinsic calibration of the road end laser radar to be calibrated can be completed based on the point cloud map.

[0074] S105, determining a target point set corresponding to the feature point set in each of the point cloud images, to obtain the target point set corresponding to each road end laser radar to be calibrated in the initial pose.

[0075] In a specific implementation, the labeling of the target point set needs to be performed on the basis of the coarse registration of the point cloud map and the point cloud image, and in this process, the coordinate transformation matrix of the coordinates in the point cloud image of each road end laser radar to be calibrated and the coordinates in the point cloud map can be determined, i.e., the initial pose of each road end laser radar to be calibrated relative to the point cloud map.

[0076] In an exemplary embodiment, as shown in Figure 3 the above step S105 can include:

[0077] S301, for each of the point cloud images, moving the point cloud image so that the point cloud image coincides with the point cloud map, to obtain the initial pose of the road end laser radar to be calibrated.

[0078] This step is the process of coarse registration of each point cloud image, and the coordinate transformation matrix determined based on the moving process is determined as the initial pose of the road end laser radar to be calibrated.

[0079] S303, determining a point set coinciding with the feature point set on each of the point cloud images, and labeling as the target point set corresponding to each of the point cloud images to the road end laser radar to be calibrated in the initial pose.

[0080] The target point set is a point set corresponding to the road facility coinciding with the feature point set on the point cloud image.

[0081] In the implementation, after the feature point set is labeled with a number and the target point set coinciding with the feature point set on the point cloud image is determined, the target point set should also be labeled with a corresponding number, and the feature point sets and the target point sets with different numbers cannot be matched with each other.

[0082] As can be seen from the above technical solutions of the embodiments of the present application, the embodiments of the present application determine the target point set corresponding to the feature point set on the point cloud map for each to-be-calibrated road end laser radar in the initial pose thereof through the coarse registration of the point cloud image and the point cloud map of each to-be-calibrated road end laser radar, so that the optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map can be determined by gradually optimizing the position of the target point set relative to the point cloud map subsequently.

[0083] S107, for each to-be-calibrated road end laser radar, determining the optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map based on the target point set in the corresponding initial pose.

[0084] The position of the target point set relative to the point cloud map is optimized by adjusting the initial pose, and the adjusted initial pose is determined as the optimized pose when the expected optimization result is reached.

[0085] In an exemplary embodiment, as shown in Figure 4 The above step S107 can include:

[0086] S401, for each feature point in the feature point set, labeling the second preset number of nearest neighbor points corresponding to the feature point on the point cloud map.

[0087] The nearest neighbor point is a point on the point cloud map whose distance from the corresponding feature point is within the first second preset number in ascending order.

[0088] Each feature point is traversed, and the second preset number of nearest neighbor points corresponding to each feature point is labeled by traversing the point cloud map from the current feature point.

[0089] S403, for each target point in the target point set corresponding to each to-be-calibrated road end laser radar in the initial pose, determining the distance between the second preset number of nearest neighbor points corresponding to the target feature point and the target point, to obtain the target distance corresponding to the target point.

[0090] The target feature point refers to the feature point in the feature point set corresponding to the target point.

[0091] Herein, the nearest neighbor point corresponding to the target feature point is the point on the point cloud map closest to the target point.

[0092] The step S403 can include the following steps:

[0093] The second preset number of nearest neighbor points corresponding to the target feature point are fitted according to a preset geometric element to obtain a fitting result corresponding to the target point; the preset geometric element includes a straight line or a plane.

[0094] The distance between the target point and the fitting result is determined to obtain a target distance corresponding to the target point.

[0095] The preset geometric element is different according to the type of the feature point set to which the target feature point belongs. For a ground plane point set, the preset geometric element is a plane; for a road edge point set and a lamp pole edge point set, the preset geometric element is a straight line.

[0096] The target distance is also different according to the type of the target point set to which the target point belongs. For a ground plane point set, the target distance is the distance between the target point and the plane fitted by the nearest neighbor point corresponding to the target feature point; for a road edge point set and a lamp pole edge point set, the target distance is the distance between the target point and the straight line fitted by the nearest neighbor point corresponding to the target feature point.

[0097] As can be seen from the above technical solutions of the embodiments of the present application, the distance between the target point and the geometric element fitted by the nearest neighbor point corresponding to the target feature point is used to correspond the error of the initial pose, which provides a cut-in for subsequent optimization of the current pose.

[0098] S405, the target distance of each target point in the target point set corresponding to the initial pose of each to-be-calibrated road end laser radar is optimized to obtain the optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map.

[0099] The target distance can reflect the error of the initial pose of the to-be-calibrated road end laser radar relative to the point cloud map compared with the actual pose of the to-be-calibrated road end laser radar relative to the world coordinate system to a certain extent, and the initial pose is optimized based on the error to obtain the optimized pose of the to-be-calibrated road end laser radar.

[0100] The optimization processing refers to obtaining the target distance based on the initial pose, slightly changing the initial pose, reducing the target distance, and determining the initial pose after adjustment that makes the target distance minimum as the optimized pose of the to-be-calibrated road end laser radar until the target distance reaches the minimum.

[0101] It can be seen from the technical solutions of the embodiments of the present application that the distance between the target points in the target point set corresponding to the to-be-calibrated road end laser radar in the initial pose and the second preset number of nearest neighbor points corresponding to the target feature points is continuously reduced by taking the distance as the error of the initial pose, so that the initial pose is continuously optimized.

[0102] The step S405 can include the following steps:

[0103] For each to-be-calibrated road end laser radar, the sum of the target distances of the target points in the corresponding target point set in the corresponding initial pose is determined to obtain the target optimization distance corresponding to the to-be-calibrated road end laser radar;

[0104] The initial pose of the to-be-calibrated road end laser radar is adjusted, and the target optimization distance corresponding to the to-be-calibrated road end laser radar is updated based on the target point set corresponding to the to-be-calibrated road end laser radar in the adjusted initial pose, and the adjustment of the initial pose of the to-be-calibrated road end laser radar is ended when the updated target optimization distance is less than a preset distance threshold.

[0105] The initial pose of the to-be-calibrated road end laser radar after the adjustment is ended is determined as the optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map.

[0106] The target optimization distance is a whole reflection of the error of the position of the target point relative to the point cloud map compared with the actual coordinates of the target point in the world coordinate system, and the error reflects the error of the initial pose of the to-be-calibrated road end laser radar relative to the point cloud map compared with the actual pose of the to-be-calibrated road end laser radar relative to the world coordinate system. Based on the target optimization distance, it is determined whether the error of the position of the target point set of each to-be-calibrated road end laser radar relative to the point cloud map in the initial pose has reached the minimum.

[0107] The preset distance threshold is set according to the acceptable error of the initial pose of the to-be-calibrated road end laser radar relative to the point cloud map compared with the actual pose of the to-be-calibrated road end laser radar relative to the world coordinate system. The smaller the preset distance threshold is set, the more accurate the optimized pose is, and the more accurate the calibration parameters are.

[0108] The initial pose of the to-be-calibrated road end laser radar after the adjustment is ended, that is, in the initial pose, the target optimization distance is the minimum, that is, the error of the error of the position of the target point relative to the point cloud map compared with the actual coordinates of the target point in the world coordinate system is the minimum, and the optimization of the initial pose is completed at this time.

[0109] In a specific implementation, for each to-be-calibrated road end laser radar, the initial pose thereof is adjusted, the point cloud image is moved according to the initial pose, the target point set on the point cloud image is also moved, the distance between the target points in the moved target point set and the second preset number of nearest neighbor points corresponding to the target feature points is determined, that is, the target optimization distance. It is judged whether the target optimization distance has reached the minimum. If the target optimization distance has reached the minimum, the initial pose adjusted in this step is determined as the optimized pose. If the target optimization distance has not reached the minimum, the initial pose is continuously adjusted, the above step is repeated, until the target optimization distance reaches the minimum, the adjustment is ended, and the initial pose after the adjustment is ended is determined as the optimized pose. This is a process of continuous exploration. Through continuous adjustment of the initial pose, the minimum value of the error of the position of the target point relative to the point cloud map compared with the actual coordinates of the target point in the world coordinate system is explored, and the optimization of the initial pose is completed.

[0110] As can be seen from the above technical solutions of the embodiments of the present application, the embodiments of the present application judge whether the distance between the target points and the second preset number of nearest neighbor points corresponding to the target feature points has reached the minimum, judge whether the initial pose has reached the optimum, and decide whether to continue adjusting the initial pose according to this, so as to reduce the target optimization distance, thereby ensuring that the error of the optimized pose is minimized.

[0111] In S109, based on the optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map, the extrinsic parameter between the plurality of to-be-calibrated road end laser radars is determined.

[0112] Based on the coordinate transformation matrix of each to-be-calibrated road end laser radar relative to the point cloud map, the coordinate transformation matrix between the plurality of to-be-calibrated road end laser radars is determined through matrix operation.

[0113] As can be seen from the above technical solutions of the embodiments of the present application, the embodiments of the present application create a point cloud map for a region by collecting laser radars on a vehicle, label a feature point set on the point cloud map, and then match the point cloud of the to-be-calibrated road end laser radar to the point cloud map, to realize extrinsic calibration of multiple radars. Even if there is no any common view area between two radars, the extrinsic calibration can also be completed. At the same time, the precision of the extrinsic parameter can be well controlled, and almost only depends on the precision of the ranging of the radar itself. The calibration demand of a large number of road end laser radars in intelligent traffic road end applications can be quickly realized.

[0114] In one exemplary embodiment, as shown in Figure 5 S109 can include:

[0115] In S501, among the plurality of to-be-calibrated road end laser radars, a first road end laser radar and a second road end laser radar to be mutually calibrated are determined.

[0116] The known coordinate transformation matrix of the plurality of to-be-calibrated road end lidars relative to the point cloud map, and the coordinate transformation matrix between the plurality of to-be-calibrated road end lidars, need to determine the coordinate transformation matrix between each other, so it is necessary to determine the two road end lidars to be calibrated.

[0117] S503, determine the first optimized pose of the first road end lidar relative to the point cloud map and the second optimized pose of the second road end lidar relative to the point cloud map.

[0118] The first optimized pose is the optimized coordinate transformation matrix of the first road end lidar relative to the point cloud map, and the second optimized pose is the optimized coordinate transformation matrix of the second road end lidar relative to the point cloud map.

[0119] S505, determine the coordinate transformation matrix of the first road end lidar relative to the second road end lidar based on the first optimized pose and the second optimized pose, as the external parameter of the first road end lidar.

[0120] Based on the coordinate transformation matrix of the first road end lidar relative to the point cloud map and the coordinate transformation matrix of the second road end lidar relative to the point cloud map, the coordinate transformation matrix of the first road end lidar relative to the second road end lidar is obtained through matrix operation.

[0121] S507, determine the coordinate transformation matrix of the second road end lidar relative to the first road end lidar based on the first optimized pose and the second optimized pose, as the external parameter of the second road end lidar.

[0122] Based on the coordinate transformation matrix of the first road end lidar relative to the point cloud map and the coordinate transformation matrix of the second road end lidar relative to the point cloud map, the coordinate transformation matrix of the second road end lidar relative to the first road end lidar is obtained through matrix operation.

[0123] From the above technical solutions of the embodiments of the present application, the embodiments of the present application determine the external parameter between the plurality of to-be-calibrated road end lidars by determining the optimized pose of the plurality of to-be-calibrated road end lidars relative to the point cloud map, thereby completing the external parameter calibration of the plurality of to-be-calibrated road end lidars, even if there is no any common view area between the two radars. The external parameter calibration can be completed.

[0124] The following describes the making of the point cloud map before obtaining the point cloud map and the point cloud image of each to-be-calibrated road end lidar in the plurality of to-be-calibrated road end lidars, as shown in Figure 6 The specific steps can include the following steps:

[0125] S601, acquire first point cloud data collected by a laser radar on a collection vehicle at multiple time points during movement.

[0126] The multiple time points are time points separated by a preset time period, and in this embodiment, data is collected every 100 milliseconds.

[0127] The first point cloud data is point cloud data collected by the laser radar on the collection vehicle within the preset range.

[0128] S603, select a first time point from the multiple time points, and determine a current point cloud map based on first point cloud data collected at the first time point.

[0129] A first time point is randomly selected from the multiple time points, first point cloud data collected by the laser radar on the collection vehicle at the first time point is obtained, a current pose of the laser radar on the collection vehicle at the first time point is obtained based on a laser radar odometer of the collection vehicle, a current point cloud map is generated based on the first point cloud data and the current pose, and the current point cloud map obtained based on the first point cloud data collected at the first time point is a base map for subsequent updating of the map.

[0130] S605, select a second time point from the remaining time points, and update the current point cloud map based on first point cloud data collected at the second time point.

[0131] The remaining time points refer to time points that have not been selected from the multiple time points.

[0132] First point cloud data collected by the laser radar on the collection vehicle at the second time point is obtained, a current pose of the laser radar on the collection vehicle at the second time point is obtained based on a laser radar odometer of the collection vehicle, a point cloud map at the second time point is determined based on the first point cloud data and the current pose, and the point cloud map is supplemented to the current point cloud map, thereby completing updating of the current point cloud map.

[0133] S607, in a case where a coverage degree of the updated current point cloud map and the preset range does not reach a preset coverage degree, repeat the steps of selecting a second time point from the remaining time points and updating the current point cloud map based on first point cloud data collected at the second time point, until the coverage degree of the updated current point cloud map and the preset range reaches the preset coverage degree.

[0134] The preset coverage degree is a specific value for determining whether the coverage rate of the current point cloud map for the preset range meets a standard.

[0135] This is a process of cyclic updating, by collecting first point cloud data at different time points, the coverage range of the current point cloud map is continuously expanded, until the current point cloud map can cover the scanning ranges of multiple road end laser radars to be calibrated.

[0136] S609, the updated current point cloud map when the preset coverage degree is reached is taken as the point cloud map.

[0137] From the above technical solutions of the embodiments of the present application, it can be seen that the embodiments of the present application make a point cloud map by collecting point cloud data in the scanning range of the to-be-calibrated road end laser radar collected by the laser radar on the collection vehicle, take the point cloud map as a reference benchmark for optimizing the initial pose of the to-be-calibrated road end laser radar relative to the point cloud map, collect point cloud data of different ranges at different times by the movement of the collection vehicle, take the point cloud map at different times corresponding to the point cloud map of different ranges, update the current point cloud map by making the point cloud map at different times, and complete the splicing of the point cloud maps of different ranges, so as to ensure that the finally completed point cloud map covers the scanning ranges of the multiple to-be-calibrated road end laser radars.

[0138] Corresponding to the multi-laser radar calibration method provided by the above several embodiments, the embodiments of the present application also provide a multi-laser radar calibration device. Since the multi-laser radar calibration device provided by the embodiments of the present application corresponds to the multi-laser radar calibration method provided by the above several embodiments, the implementation modes of the foregoing multi-laser radar calibration method are also applicable to the multi-laser radar calibration device provided by the present embodiments, which will not be described in detail in the present embodiments.

[0139] Please refer to Figure 7 , which is a structural schematic diagram of a multi-laser radar calibration device provided by the embodiments of the present application. The device has the function of implementing the multi-laser radar calibration method in the above method embodiments, which can be realized by hardware or corresponding software executed by hardware. As Figure 7 indicated, the device can include:

[0140] An information acquisition module is configured to acquire a point cloud map and a point cloud image of each to-be-calibrated road end laser radar in the multiple to-be-calibrated road end laser radars. The point cloud map is obtained based on first point cloud data collected by a laser radar on a collection vehicle in a preset range, and the preset range includes the scanning ranges of the multiple to-be-calibrated road end laser radars. The point cloud image of each to-be-calibrated road end laser radar is an image generated based on second point cloud data collected by the corresponding to-be-calibrated road end laser radar.

[0141] A feature labeling module is configured to label a feature point set based on the point cloud map.

[0142] A point set matching module is configured to determine a target point set corresponding to the feature point set in each point cloud image, and obtain a target point set corresponding to each to-be-calibrated road end laser radar at an initial pose.

[0143] an optimization processing module, configured to determine, for each to-be-calibrated road end laser radar, an optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map based on the target point set in the corresponding initial pose;

[0144] an external parameter determination module, configured to determine external parameter of the plurality of to-be-calibrated road end laser radars relative to each other based on the optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map.

[0145] In an exemplary embodiment, the feature labeling module comprises:

[0146] a region determination module, configured to determine, on the point cloud map, a map region that coincides with the scanning range of the plurality of to-be-calibrated road end laser radars;

[0147] a facility labeling module, configured to determine, in the map region, a first preset number of road facilities as the feature point set.

[0148] In an exemplary embodiment, the point set matching module comprises:

[0149] a point cloud movement module, configured to, for each of the point cloud images, move the point cloud image so that the point cloud image coincides with the point cloud map, to obtain an initial pose of the to-be-calibrated road end laser radar;

[0150] a point set labeling module, configured to determine, on each of the point cloud images, a point set that coincides with the feature point set as a target point set corresponding to the to-be-calibrated road end laser radar in the initial pose corresponding to the point cloud image.

[0151] In an exemplary embodiment, the optimization processing module comprises:

[0152] a neighboring point labeling module, configured to, for each feature point in the feature point set, label a second preset number of nearest neighbor points corresponding to the feature point on the point cloud map; the nearest neighbor points are points on the point cloud map that are within the first second preset number in ascending order of distance from the corresponding feature point;

[0153] a distance determination module, configured to, for each target point in the target point set corresponding to each to-be-calibrated road end laser radar in the initial pose, determine a target distance corresponding to the target point, by determining a distance between the target point and a second preset number of nearest neighbor points corresponding to a target feature point; the target feature point refers to a feature point in the feature point set corresponding to the target point.

[0154] a distance optimization module, configured to optimize a target distance of each target point in a target point set corresponding to each to-be-calibrated road end laser radar in an initial pose, to obtain an optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map.

[0155] In an example implementation, the distance determination module comprises:

[0156] a neighbor point fitting module, configured to fit a second preset number of nearest neighbor points corresponding to the target feature point according to a preset geometric element to obtain a fitting result corresponding to the target point; the preset geometric element comprises a straight line or a plane;

[0157] a first determination module, configured to determine a distance between the target point and the fitting result to obtain a target distance corresponding to the target point.

[0158] In an example implementation, the distance optimization module comprises:

[0159] a distance summation module, configured to determine, for each to-be-calibrated road end laser radar, a sum of target distances of target points in a target point set corresponding to the to-be-calibrated road end laser radar in a corresponding initial pose to obtain a target optimized distance corresponding to the to-be-calibrated road end laser radar;

[0160] a pose adjustment module, configured to adjust the initial pose of the to-be-calibrated road end laser radar, update the target optimized distance corresponding to the to-be-calibrated road end laser radar based on a target point set corresponding to the to-be-calibrated road end laser radar in the adjusted initial pose, and end the adjustment of the initial pose of the to-be-calibrated road end laser radar when the updated target optimized distance is less than a preset distance threshold.

[0161] a pose determination module, configured to determine the initial pose of the to-be-calibrated road end laser radar after the end of the adjustment as an optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map.

[0162] In an example implementation, the external parameter determination module comprises:

[0163] a radar determination module, configured to determine, from a plurality of to-be-calibrated road end laser radars, a first road end laser radar and a second road end laser radar to be mutually calibrated;

[0164] a second determination module, configured to determine a first optimized pose of the first road end laser radar relative to the point cloud map and a second optimized pose of the second road end laser radar relative to the point cloud map;

[0165] The first coordinate transformation module is configured to determine a coordinate transformation matrix of the first road end laser radar relative to the second road end laser radar based on the first optimized pose and the second optimized pose as an external parameter of the first road end laser radar.

[0166] The second coordinate transformation module is configured to determine a coordinate transformation matrix of the second road end laser radar relative to the first road end laser radar based on the first optimized pose and the second optimized pose as an external parameter of the second road end laser radar.

[0167] In an exemplary embodiment, the device further comprises a map making module for making a point cloud map, the map making module comprising:

[0168] The point cloud acquisition module is configured to acquire first point cloud data collected by a laser radar on a collection vehicle at multiple time points during movement of the laser radar; the multiple time points are time points separated by a preset time period, and the first point cloud data is point cloud data within the preset range collected by the laser radar on the collection vehicle;

[0169] The map creating module is configured to select a first time point from the multiple time points and determine a current point cloud map based on first point cloud data collected at the first time point;

[0170] The map updating module is configured to select a second time point from the remaining time points and update the current point cloud map based on first point cloud data collected at the second time point; the remaining time points refer to time points that are not selected from the multiple time points;

[0171] The map optimization module is configured to, in a case where a coverage degree of the updated current point cloud map and the preset range does not reach a preset coverage degree, repeat the steps of selecting a second time point from the remaining time points and updating the current point cloud map based on first point cloud data collected at the second time point until the coverage degree of the updated current point cloud map and the preset range reaches the preset coverage degree;

[0172] The map determination module is configured to determine the updated current point cloud map when the preset coverage degree is reached as the point cloud map.

[0173] It should be noted that the device provided in the above embodiments, in realizing its functions, only divides the above-mentioned various functional modules by way of example, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0174] The electronic device provided in the embodiments of the present application comprises a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the multi-laser radar calibration methods provided in the above method embodiments.

[0175] The memory can be used to store software programs and modules, and the processor can execute various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access of the processor to the memory.

[0176] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal, a server or a similar computing device, that is, the above electronic device can include a computer terminal, a server or a similar computing device. Figure 8 is a hardware structure block diagram of the electronic device running a multi-laser radar calibration method provided in the embodiments of the present application, as Figure 8 shown, the internal structure of the computer device can include but is not limited to a processor, a network interface and a memory. Among them, the processor, the network interface and the memory in the computer device can be connected through a bus or other means, in the embodiments of the present application Figure 8 are taken as examples of connection through a bus.

[0177] The processor (or CPU (Central Processing Unit)) is the computing core and control core of the computer device. The network interface can optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.). The memory is a memory device in the computer device, used to store programs and data. It can be understood that the memory here can be a high-speed RAM memory device, or a non-volatile memory, for example, at least one disk storage device; optionally, it can also be at least one storage device located away from the aforementioned processor. The memory provides a storage space that stores the operating system of the electronic device, which can include but is not limited to: a Windows system (an operating system), a Linux (an operating system), an Android (a mobile operating system) system, an IOS (a mobile operating system) system, etc., and the present application is not limited thereto; and in the storage space, one or more instructions suitable for being loaded and executed by the processor are also stored, and these instructions can be one or more computer programs (including program codes). In the embodiment of the present application, the processor loads and executes one or more instructions stored in the memory to implement the multi-laser radar calibration method provided by the above method embodiment.

[0178] The embodiment of the present application also provides a computer readable storage medium, which can be arranged in an electronic device to save at least one instruction or at least one program related to a multi-laser radar calibration method, and the at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the multi-laser radar calibration methods provided by the above method embodiments.

[0179] Optionally, in the present embodiment, the storage medium can include but is not limited to: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0180] It should be noted that the above-mentioned order of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes the specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or necessary.

[0181] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0182] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0183] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-lidar calibration method, characterized by, The method comprises: acquiring a point cloud map and a point cloud image of each of a plurality of to-be-calibrated roadside laser radars; the point cloud map is obtained based on first point cloud data collected by a laser radar on a collection vehicle within a preset range, and the preset range includes scanning ranges of the plurality of to-be-calibrated roadside laser radars; the point cloud image of each to-be-calibrated roadside laser radar is an image generated based on second point cloud data collected by the corresponding to-be-calibrated roadside laser radar; annotating a feature point set based on the point cloud map; determining a target point set corresponding to the feature point set in each of the point cloud images to obtain a target point set corresponding to each to-be-calibrated roadside laser radar in an initial pose; for each to-be-calibrated roadside laser radar, determining an optimized pose of the to-be-calibrated roadside laser radar relative to the point cloud map based on the target point set in the corresponding initial pose; determining external parameter parameters between the plurality of to-be-calibrated roadside laser radars based on the optimized poses of each of the plurality of to-be-calibrated roadside laser radars relative to the point cloud map; wherein the determining of the target point set corresponding to the feature point set in each of the point cloud images to obtain the target point set corresponding to each to-be-calibrated roadside laser radar in the initial pose comprises: for each of the point cloud images, moving the point cloud image to make the point cloud image coincide with the point cloud map to obtain the initial pose of the to-be-calibrated roadside laser radar; and determining a point set coinciding with the feature point set on each of the point cloud images to annotate the target point set corresponding to the to-be-calibrated roadside laser radar corresponding to each of the point cloud images in the initial pose; wherein the determining of the optimized pose of each to-be-calibrated roadside laser radar relative to the point cloud map based on the target point set in the corresponding initial pose comprises: for each feature point in the feature point set, annotating a second preset number of nearest neighbor points corresponding to the feature point on the point cloud map; the nearest neighbor points are points on the point cloud map having distances from the corresponding feature point within a front second preset number in ascending order; for each target point in the target point set corresponding to each to-be-calibrated roadside laser radar in the initial pose, determining distances between the target point and a second preset number of nearest neighbor points corresponding to a target feature point to obtain a target distance corresponding to the target point; the target feature point refers to a feature point in the feature point set corresponding to the target point; and performing optimization processing on the target distances of the target points in the target point set corresponding to each to-be-calibrated roadside laser radar in the initial pose to obtain the optimized pose of each to-be-calibrated roadside laser radar relative to the point cloud map.

2. The multi-lidar calibration method of claim 1, wherein, The annotation of the feature point set based on the point cloud map comprises: determining a map region coinciding with scanning ranges of the plurality of to-be-calibrated roadside laser radars on the point cloud map; determining a first preset number of road facilities in the map region to annotate as the feature point set.

3. The method of claim 1, wherein, The determination of the distances between the target point and a second preset number of nearest neighbor points corresponding to a target feature point to obtain a target distance corresponding to the target point comprises: fitting the second preset number of nearest neighbor points corresponding to the target feature points according to a preset geometric element to obtain a fitting result corresponding to the target point; the preset geometric element includes a straight line or a plane; determining a distance between the target point and the fitting result to obtain a target distance corresponding to the target point.

4. The multi-lidar calibration method of claim 1, wherein, The method further comprises: determining a sum of the target distances of each target point in the target point set corresponding to the initial pose of each to-be-calibrated road end laser radar to obtain a target optimization distance corresponding to the to-be-calibrated road end laser radar; adjusting the initial pose of the to-be-calibrated road end laser radar, updating the target optimization distance corresponding to the to-be-calibrated road end laser radar based on the target point set corresponding to the to-be-calibrated road end laser radar in the adjusted initial pose, until the updated target optimization distance is less than a preset distance threshold, the adjustment of the initial pose of the to-be-calibrated road end laser radar is ended; determining the initial pose of the to-be-calibrated road end laser radar after the adjustment as the optimization pose of the to-be-calibrated road end laser radar relative to the point cloud map.

5. The method of claim 1, wherein, The method further comprises: determining a first road end laser radar and a second road end laser radar to be mutually calibrated in the plurality of to-be-calibrated road end laser radars; determining a first optimization pose of the first road end laser radar relative to the point cloud map and a second optimization pose of the second road end laser radar relative to the point cloud map; determining a coordinate transformation matrix of the first road end laser radar relative to the second road end laser radar based on the first optimization pose and the second optimization pose as an extrinsic parameter of the first road end laser radar; determining a coordinate transformation matrix of the second road end laser radar relative to the first road end laser radar based on the first optimization pose and the second optimization pose as an extrinsic parameter of the second road end laser radar.

6. The multi-lidar calibration method of claim 1, wherein, The method further comprises: acquiring first point cloud data collected by a laser radar on a collection vehicle at a plurality of moments in a moving process; the plurality of moments are moments separated by a preset time period, and the first point cloud data is point cloud data collected by the laser radar on the collection vehicle within the preset range; selecting a first moment from the plurality of moments, and determining a current point cloud map based on the first point cloud data collected at the first moment; selecting a second moment from the remaining moments, and updating the current point cloud map based on the first point cloud data collected at the second moment; the remaining moments refer to moments that are not selected from the plurality of moments; In a case where the updated current point cloud map does not reach the preset coverage degree with the preset range, the step of selecting a second time from the remaining time points and updating the current point cloud map based on the first point cloud data collected at the second time is repeated until the updated current point cloud map reaches the preset coverage degree with the preset range. The updated current point cloud map that reaches the preset coverage degree is taken as the point cloud map.

7. A multi-lidar calibration device, characterized in that, The apparatus comprises: an information acquisition module configured to acquire a point cloud map and a point cloud image of each of a plurality of to-be-calibrated road end laser radars; the point cloud map is obtained based on first point cloud data collected by a laser radar on a collection vehicle within a preset range, and the preset range includes scanning ranges of the plurality of to-be-calibrated road end laser radars; and the point cloud image of each to-be-calibrated road end laser radar is an image generated based on second point cloud data collected by the corresponding to-be-calibrated road end laser radar; a feature labeling module configured to label a feature point set based on the point cloud map; a point set matching module configured to determine a target point set corresponding to the feature point set in each point cloud image to obtain a target point set corresponding to each to-be-calibrated road end laser radar in an initial pose; an optimization processing module configured to determine, for each to-be-calibrated road end laser radar, an optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map based on the target point set in the corresponding initial pose; an external parameter determination module configured to determine external parameter parameters between the plurality of to-be-calibrated road end laser radars based on the optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map; The point set matching module comprises: a point cloud moving module configured to move each point cloud image so that the point cloud image coincides with the point cloud map to obtain an initial pose of the to-be-calibrated road end laser radar; and a point set labeling module configured to determine a point set coinciding with the feature point set on each point cloud image and label the point set as a target point set corresponding to the to-be-calibrated road end laser radar in the initial pose corresponding to each point cloud image. The optimization processing module comprises: a neighboring point labeling module, configured to label, for each feature point in the feature point set, a second preset number of nearest neighboring points corresponding to the feature point on the point cloud map; the nearest neighboring points are points on the point cloud map having distances to the corresponding feature point located within a first preset number in ascending order; a distance determining module, configured to determine, for each target point in a target point set corresponding to each to-be-calibrated road end laser radar in an initial pose, a distance between the target point and a second preset number of nearest neighboring points corresponding to a target feature point, to obtain a target distance corresponding to the target point; the target feature point refers to a feature point in the feature point set corresponding to the target point; and a distance optimization module, configured to perform optimization processing on the target distances of the target points in the target point set corresponding to each to-be-calibrated road end laser radar in the initial pose, to obtain an optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map.

8. An electronic device, comprising: The device comprises a processor and a memory, and the memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the multi-laser radar calibration method according to any one of claims 1-6.

Images