Vehicle-mounted multi-laser radar calibration system and method, electronic equipment and medium

Abstract

The invention discloses a vehicle-mounted multi-laser radar calibration system and method, electronic equipment and a medium, and relates to the technical field of radio orientation, and the method comprises the steps: enabling a to-be-calibrated vehicle to be fixed in a calibration scene meeting a preset data collection condition through a scene calibration device, so as to reduce the dependence on a specific environment through the fixed calibration scene, the radar calibration device performs registration among the plurality of laser radars by taking the main laser radar in the plurality of laser radars as a reference so as to determine the pose adjustment parameters of the other laser radars, and further realizes the pose adjustment of the plurality of laser radars through the point cloud pose conversion device so as to complete an actual calibration task, thereby solving the problem that in the related technology, the calibration efficiency is high. The technical problems that the calibration efficiency, the calculation power required by calibration and the calibration effect are difficult to balance, and the calibration cost and the calibration difficulty in a large-batch calibration scene are high due to certain dependence on a static calibration environment in the prior art are solved, and the effects that the influence of the environment on the calibration result is reduced through a fixed scene, so that the calibration effect is guaranteed, and the calibration efficiency is greatly improved are achieved. The calculation amount is reduced; and the calibration efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of radio direction finding, and particularly to a calibration system, method, electronic device and medium for vehicle-mounted multi-lidar. Background Art

[0002] Lidar is one of the most commonly used sensors in vehicles. Lidar calibration is a process of estimating its internal parameters (such as the rotation center of the laser beam, scanning inclination angle and divergence) and external parameters (such as the position and orientation of the lidar relative to the vehicle coordinate system).

[0003] In related technologies, the calibration of multi-lidar can be achieved by various methods. For example, through the known installation positions of the lidars, manually measure the relative poses of multiple lidars, and then fine-tune the calibration parameters through optimization. However, the measurement error is large, the accuracy is low, manual adjustment is required, the efficiency is low, it is not suitable for high-precision applications, and mechanical errors accumulate after long-term use, requiring re-calibration; by finding common features (such as corner points, planes, cylinders, etc.) in the environmental point cloud data collected by multiple lidars, and then calculating the relative poses between the lidars through feature matching. However, it depends on environmental features. If the environment changes greatly or the features are scarce, the calibration effect is poor, the calculation amount is large, and it may take a long time for optimization; by aligning the point clouds collected by two lidars based on the ICP (Iterative Closest Point) algorithm to calculate their relative poses. However, it depends on the initial pose estimation. Incorrect initial values may lead to incorrect alignment, the calculation amount is large, there are certain requirements for computing resources, and it is not suitable for movable or large-angle change situations.

[0004] Therefore, how to reduce the calculation amount and improve the calibration efficiency while ensuring the calibration accuracy of lidar is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The present application provides a calibration system, method, electronic device and medium for vehicle-mounted multi-lidar to at least solve the technical problems in related technologies that it is difficult to balance the calibration efficiency, computing power required for calibration and calibration effect, and there is a certain dependence on the static calibration environment, resulting in relatively high calibration costs and calibration difficulties in large-scale calibration scenarios.

[0006] The present application provides a calibration system for a vehicle-mounted multi-lidar, including: a scene calibration device for fixing a vehicle to be calibrated in a calibration scene that meets preset data acquisition conditions based on a target calibration task; a point cloud pose transformation device for adjusting the pose parameters of multiple lidars; a lidar calibration device for acquiring point cloud data scanned by multiple lidars in the calibration scene, taking the main lidar among the multiple lidars as a reference, performing point cloud registration of the multiple lidars based on the point cloud data to obtain a point cloud registration result, generating a corresponding pose parameter transformation instruction based on the point cloud registration result, and using the pose parameter transformation instruction to control the point cloud pose transformation device until the target calibration task is completed.

[0007] Optionally, in an embodiment of the present application, the scene calibration device includes: a fixed chassis, on which a wheel groove is provided for fixing the vehicle to be calibrated in the wheel groove so that the center point of the rear axle of the vehicle to be calibrated corresponds to the target projection point of the fixed chassis; a plurality of reflection columns fixed on the fixed chassis, wherein the positional relationship between the plurality of reflection columns satisfies a preset distribution condition, and a reflection sticker is provided in the calibration area of each reflection column and / or the calibration area of the fixed bottom plate to perform point cloud registration in combination with the position information of the reflection sticker and the point cloud data.

[0008] Optionally, in an embodiment of the present application, the lidar calibration device includes: a conversion module for converting the point cloud coordinates of at least one secondary lidar other than the main lidar among the multiple lidars to the point cloud coordinate system of the main lidar to obtain the transferred point cloud corresponding to at least one secondary lidar; a rough calibration module for performing rough calibration of the multiple lidars based on the transferred point cloud and the point cloud data of the main lidar to obtain a rough calibration result; a generation module for generating a corresponding initial pose parameter transformation instruction based on the rough calibration result and using the initial pose parameter transformation instruction to control the point cloud pose transformation device.

[0009] Optionally, in an embodiment of the present application, the lidar calibration device further includes: a first acquisition module for acquiring the pose data of the multiple lidars under the initial pose parameter transformation instruction; a fine registration module for performing probability density modeling on the target point cloud of the main lidar, taking the pose data as an initial value, calculating the matching probability between the point cloud to be registered and the target point cloud of at least one secondary lidar to obtain a matching score; an optimization module for performing registration iterative optimization based on the matching score until a preset convergence condition is met to obtain a point cloud registration result.

[0010] Optionally, in an embodiment of the present application, the lidar calibration device further includes: a verification module for calculating the error between new multi-frame point cloud data acquired by the multiple lidars under the point cloud registration result and verifying whether the point cloud registration result meets a preset qualified condition using the error.

[0011] Optionally, in an embodiment of the present application, the point cloud pose transformation device includes: a first adjustment module for adjusting a plurality of lidars based on an initial pose parameter transformation instruction; a second adjustment module for adjusting the plurality of lidars based on the pose parameter transformation instruction.

[0012] Optionally, in an embodiment of the present application, the lidar calibration device includes: a second acquisition module for acquiring the configuration parameters of each lidar among the plurality of lidars; a processing module for performing time synchronization processing on each lidar based on the configuration parameters.

[0013] The present application also provides a calibration method for a vehicle-mounted multi-lidar, including: based on a target calibration task, acquiring point cloud data scanned by a plurality of lidars in a calibration scene; taking the main lidar among the plurality of lidars as a reference, performing point cloud registration of the plurality of lidars based on the point cloud data to obtain a point cloud registration result; generating a corresponding pose parameter transformation instruction based on the point cloud registration result to adjust the pose parameters of the plurality of lidars by using the pose parameter transformation instruction until the target calibration task is completed.

[0014] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any one of the above-mentioned calibration methods for a vehicle-mounted multi-lidar when executing the computer program.

[0015] The present application also provides a computer-readable storage medium, in which a computer program is stored, and wherein the computer program implements the steps of any one of the above-mentioned calibration methods for a vehicle-mounted multi-lidar when executed by a processor.

[0016] The present application also provides a computer program product, including a computer program, and the computer program implements the steps of any one of the above-mentioned calibration methods for a vehicle-mounted multi-lidar when executed by a processor.

[0017] Through the present application, due to the scene calibration device, the vehicle to be calibrated can be fixed in a calibration scene that meets the preset data acquisition conditions, so as to reduce the influence of the environment on the calibration effect by using the fixed calibration scene. By the lidar calibration device, registration is performed among a plurality of lidars with the main lidar among the plurality of lidars as a reference to determine the pose adjustment parameters of the remaining lidars, and then the pose of the plurality of lidars is adjusted through the point cloud pose transformation device to complete the actual calibration task. Therefore, it can solve the technical problems in the related art that it is difficult to balance the calibration efficiency, the computing power required for calibration, and the calibration effect, and there is a certain dependence on the static calibration environment, resulting in relatively high calibration costs and calibration difficulties in a large number of calibration scenarios, and achieves the technical effect of reducing the influence of the environment on the calibration result by fixing the scene, thereby reducing the calculation amount and improving the calibration efficiency while ensuring the calibration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 FIG.

[0020] Figure 2 FIG.

[0021] Figure 3 FIG.

[0022] Figure 4 FIG.

[0023] Figure 5 FIG. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, rather than all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0025] It should be noted that in the description of the present application, the terms "including", "comprising" or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and not to describe a specific order or sequence.

[0026] To enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments.

[0027] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the calibration system for vehicle-mounted multi-lidar depends, the specific application environment architecture or specific hardware architecture is described herein.

[0028] Embodiments of the present application provide a calibration system for vehicle-mounted multi-lidar. In combination with the structure of the calibration system for vehicle-mounted multi-lidar, the system is described in detail.

[0029] As Figure 1 shown, it is a schematic structural diagram of a calibration system 100 for vehicle-mounted multi-lidar according to an embodiment of the present application. Among them, the calibration system 100 for vehicle-mounted multi-lidar includes: a scene calibration device 101, a point cloud pose transformation device 102, and a lidar calibration device 103.

[0030] Specifically, the scene calibration device 101 is used to fix the vehicle to be calibrated in a calibration scene that meets the preset data acquisition conditions based on the target calibration task.

[0031] In the actual execution process, the embodiment of the present application can use the scene calibration device 101 to fix the vehicle to be calibrated in a set fixed scene.

[0032] Among them, in the target calibration task, it may include the vehicle model of the vehicle to be calibrated, the number of lidars of the vehicle to be calibrated, etc. Through the target calibration task, the scene calibration device 101 of the embodiment of the present application can achieve targeted fixation of the vehicle to be calibrated and subsequent lidar calibration corresponding to the number of lidars.

[0033] The data acquisition conditions of the calibration scene can be set accordingly according to the actual situation. For example, it is determined whether there are reference objects such as calibration plates in the calibration scene, whether the distribution of the reference objects in the calibration scene is reasonable, and whether the reference objects can be scanned by the lidar of the vehicle to be calibrated, etc.

[0034] Optionally, in an embodiment of the present application, the scene calibration device 101 includes: a fixed chassis, a reflecting column, and a reflecting sticker.

[0035] Among them, for the fixed chassis, wheel grooves are provided on the fixed chassis for fixing the vehicle to be calibrated in the wheel grooves so that the center point of the rear axle of the vehicle to be calibrated corresponds to the target projection point of the fixed chassis.

[0036] Multiple reflecting columns fixed on the fixed chassis, where the positional relationship between the multiple reflecting columns meets the preset distribution conditions, and the calibration area of each reflecting column and / or the calibration area of the fixed bottom plate are provided with reflecting stickers to perform point cloud registration in combination with the position information of the reflecting stickers and the point cloud data.

[0037] In the related art, by placing a specially designed calibration board (such as a plane with reflectors), multiple lidars can be scanned, and the pose of each lidar relative to the calibration board can be calculated, and then the relative pose between the lidars can be solved. Its advantages are high calibration accuracy, suitability for high-precision applications (such as autonomous driving), and applicability to most lidars, without relying on environmental features. Its disadvantages are that additional calibration equipment and sites are required, and affected by the placement accuracy of the calibration board, multiple acquisitions and optimizations may be needed.

[0038] In the embodiments of the present application, when calibrating lidars, the calibration concept of the calibration board can be referred to, and the relative pose of the reflection column and the reflection sticker can be used to calibrate multiple lidars. In order to reduce the influence on additional calibration equipment, sites, and the placement accuracy of the calibration board, the embodiments of the present application can directly form a fixed scene on the fixed chassis, so that when any vehicle to be calibrated is fixed on the fixed chassis, the pose information of each item on the fixed chassis can be directly used for calibration.

[0039] In some embodiments, the fixed chassis can be a rectangular calibration chassis 9, and the material can be steel or other hard objects. The projection point of the center of the vehicle's rear axle on the bottom plate can be set on the calibration chassis, and then a groove can be cut with the projection point as the center as the fixed slot for the vehicle's rear wheels. The front wheel slot can be set as a movable slot to adapt to the calibration of vehicles of different models.

[0040] The fixed positions of the reflection column and the reflection sticker can be preset, and the aim of the setting is to form a fixed scene with strong versatility, so as to ensure that after the vehicle to be calibrated is fixed, the lidar calibration can be completed according to the scan data of the reflection sticker.

[0041] For example, in the embodiments of the present application, a reflection column can be set at a certain distance, such as 20 m, directly in front of the extension part of the calibration chassis 9. Among them, the diameter of the reflection column can be 10 cm and the height can be 2 m, and a reflection sticker can be pasted on the upper part of the reflection column; and two reflection columns can be respectively set on both sides of the calibration chassis. The heights of the reflection columns set on both sides can be different from that of the reflection column directly in front. For example, the height of the reflection column can be 1.5 m and the diameter can be 10 cm. Similarly, a reflection sticker can be pasted on the upper part of the reflection column, and the three calibration columns are distributed in a triangle.

[0042] In addition, the embodiments of the present application can also set reflection stickers on the fixed chassis to ensure calibration from multiple angles and improve the calibration effect.

[0043] The point cloud pose transformation device 102 is used to adjust the pose parameters of multiple lidars.

[0044] In order to achieve posture adjustment and calibration of multiple laser radars, the embodiment of the present application can use the point cloud posture transformation device 102 to convert posture parameters and adjustment values, thereby achieving posture adjustment of multiple laser radars.

[0045] Among them, the point cloud posture transformation device 102 can complete the adjustment of multiple laser radar postures based on instructions, and can also perform manual adjustments according to the operation of technicians.

[0046] In the manual adjustment part, the working principle of the point cloud pose transformation device 102 can be: input the initial calibration value (x, y, z, roll, pitch, yaw) (x-axis, y-axis, z-axis, roll angle, pitch angle, heading angle), and then adjust the value by pressing the up and down buttons in the corresponding box of each value. The background loads the calibration value, generates a transformation matrix, and publishes the transformed point cloud through a ROS message, and presents the published point cloud using the ROS built-in rviz visual tool.

[0047] In the automatic adjustment part, the point cloud pose transformation device 102 can read the target pose or relative adjustment pose in the pose parameter transformation instruction, and convert it into an (x, y, z, roll, pitch, yaw) matrix based on the obtained data, thereby realizing automatic adjustment of multiple lidar poses.

[0048] Among them, in the vehicle body coordinate system, the origin is located at the center of the rear axle of the vehicle, and the axial directions of the XYZ coordinate axes conform to the right-hand rule, that is, the X axis points to the front of the vehicle body, the Y axis points to the left side of the vehicle body, and the Z axis points to the top, which conforms to the front upper left principle;

[0049] In the laser radar coordinate system, the origin is located inside the laser radar shell. Different laser radars have different positions. The XYZ axes also conform to the right-hand rule, but different radars have different directions.

[0050] Optionally, in one embodiment of the present application, the point cloud posture transformation device 102 includes: a first adjustment module and a second adjustment module.

[0051] Among them, the first adjustment module is used to adjust multiple laser radars based on the initial posture parameter transformation instruction.

[0052] The second adjustment module is used to adjust multiple laser radars based on posture parameter transformation instructions.

[0053] During the actual execution process, the point cloud posture transformation device 102 can exchange data with the radar calibration device 103. According to the alignment process of the radar calibration device 103, the point cloud posture transformation device 102 can adjust the postures of multiple laser radars.

[0054] Among them, the adjustment relationship between the radar calibration device 103 and the point cloud pose transformation device 102 will be elaborated later.

[0055] The radar calibration device 103 is used to obtain the point cloud data scanned by multiple lidars in the calibration scene. Based on the master lidar among the multiple lidars, point cloud registration of the multiple lidars is performed based on the point cloud data to obtain a point cloud registration result. Then, a corresponding pose parameter transformation instruction is generated based on the point cloud registration result, and the point cloud pose transformation device 102 is controlled by using the pose parameter transformation instruction until the target calibration task is completed.

[0056] It can be understood that by using a multi-lidar system and accurately registering and fusing its data, the perception range of the vehicle can be effectively expanded, and these blind spots can be reduced or even eliminated, thereby improving driving safety. In an autonomous driving system, the redundant design of sensors is one of the important means to ensure the reliability of the system. The use of multi-lidar and the effective registration of its data can still maintain sufficient environmental perception ability in the case of partial sensor failures, ensuring driving safety.

[0057] Therefore, the registration between multi-lidars is extremely important. The registration of in-vehicle multi-lidars can build a more complete and accurate surrounding environment model, thereby enhancing the vehicle's understanding ability of the surrounding environment. Especially in complex urban environments, such as identifying pedestrians, other vehicles, road signs, etc., it can also provide richer environmental information, laying a foundation for realizing these high-level autonomous driving functions. The registration of in-vehicle multi-lidars is of great significance for improving the safety, reliability, and intelligent level of autonomous driving vehicles. However, it is not easy to achieve efficient and accurate multi-lidar registration, which involves complex algorithm design and a large amount of experimental verification work.

[0058] In the embodiment of the present application, the radar calibration device 103 may perform point cloud registration based on the point cloud data scanned by multiple lidars in the calibration scenario. To avoid the increased computational complexity or the generation of registration errors caused by the mutual matching between multiple lidars during registration, for example, lidar A is registered with lidar B, lidar C is registered with lidar D, and then the registered lidar A is registered with the registered lidar C, resulting in the situation where the poses of the already registered lidar B or lidar D become invalid. In the embodiment of the present application, a main lidar may be selected, and based on the main lidar, the registration of other slave lidars is performed, that is, each slave lidar is registered with the main lidar, and a pose parameter transformation instruction is generated according to the registration result, that is, the transformation coordinates of the slave lidar are determined according to the registration result, so that the point cloud pose transformation device 102 determines the target pose of the slave lidar according to the transformation coordinates and adjusts the pose of the slave lidar. In the embodiment of the present application, all slave lidars may be registered and their poses adjusted until the target calibration task is completed.

[0059] Optionally, in an embodiment of the present application, the radar calibration device 103 includes: a second acquisition module and a processing module.

[0060] Among them, the second acquisition module is used to acquire the configuration parameters of each lidar among the multiple lidars.

[0061] The processing module is used to perform time synchronization processing on each lidar based on the configuration parameters.

[0062] Before registration, in the embodiment of the present application, the second acquisition module may be used to acquire the configuration parameters of each lidar. On the one hand, it can be used to determine whether the lidars are compatible, and on the other hand, corresponding schemes can be selected according to the configuration parameters to achieve the time synchronization of each lidar, so as to facilitate subsequent registration.

[0063] Generally speaking, mechanical lidars support satellite time synchronization functions and network PTP (Precision Time Protocol) time synchronization functions, while solid-state lidars generally only support network PTP synchronization. In terms of time synchronization, considering compatibility and time synchronization accuracy, the embodiment of the present application may adopt the method of network time synchronization. The specific implementation method is to connect two or more lidars to the controller respectively, and then configure them in the network time synchronization mode. In this mode, the computing unit provides time to the lidars, and at the same time adjusts the time synchronization mode of the lidars to the network synchronization mode.

[0064] When multiple lidar sensors are time-synchronized, the data they collect at the same moment can more accurately reflect the true state of the vehicle's surrounding environment. This helps create a more consistent and precise three-dimensional point cloud model, thus improving the quality of the registration result. And precise time synchronization allows the relative position relationship between the datasets of each lidar scan to be more accurate. This is crucial for subsequent registration algorithms because it reduces the additional variables or uncertainties caused by time asynchronization, enabling the registration process to focus more on spatial alignment and thereby improving the overall registration accuracy.

[0065] Optionally, in an embodiment of the present application, the lidar calibration device 103 includes: a conversion module, a rough calibration module, and a generation module.

[0066] Among them, the conversion module is used to convert the point cloud coordinates of at least one secondary lidar among multiple lidars except the primary lidar to the point cloud coordinate system of the primary lidar, so as to obtain the transferred point cloud corresponding to at least one secondary lidar.

[0067] The rough calibration module is used to perform rough calibration of multiple lidars based on the transferred point cloud and the point cloud data of the primary lidar, so as to obtain a rough calibration result.

[0068] The generation module is used to generate a corresponding initial pose parameter transformation instruction based on the rough calibration result, so as to control the point cloud pose transformation device 102 based on the initial pose parameter transformation instruction.

[0069] As a possible implementation manner, the embodiment of the present application may include two steps during registration, a rough calibration step and a fine registration step. Here, the rough calibration step is described.

[0070] The embodiment of the present application can convert the point cloud coordinates of the secondary lidar to the point cloud coordinate system of the primary lidar, and adjust the pose of the secondary lidar to achieve the coincidence of the two point clouds. At this stage, the adjustment of the secondary lidar can include two methods:

[0071] One is manual adjustment. The embodiment of the present application can display the actual situation in the point cloud coordinate system of the primary lidar, and assign different colors to the point cloud of the primary lidar and the point cloud of the secondary lidar in the point cloud coordinate system of the primary lidar. Technicians can directly control the point cloud pose transformation device 102 through the color, that is, directly generate an initial pose parameter transformation instruction to call the first adjustment module of the point cloud pose transformation device 102 to adjust the pose of the secondary lidar. Since the adjustment at this time is only the visual observation of technicians, this adjustment is only a rough calibration. In order to ensure the calibration effect, fine registration is required later.

[0072] Another is automatic adjustment. In the embodiments of the present application, preliminary alignment can be performed on obvious geometric features or statistical characteristics to generate an initial pose parameter transformation instruction, and the coincidence degree between the point cloud of the main lidar and the point cloud of the secondary lidar in the coordinate system of the main lidar point cloud is roughly calculated. Thus, it is judged whether the rough calibration stage is completed according to the coincidence degree, or an initial pose parameter transformation instruction is generated based on the coincidence degree and the area of the overlapping part, so as to adjust the pose of the secondary lidar through the first adjustment module of the point cloud pose transformation device 102. At this time, the embodiments of the present application can set a coincidence determination range, that is, when the coincidence degree between the adjusted point clouds reaches a certain coincidence determination range, the embodiments of the present application can determine that the rough calibration is completed.

[0073] Taking the main lidar A and the secondary lidar B as an example, in the embodiments of the present application, the point clouds emitted by different lidars can be set to different colors. Here, the point cloud emitted by the main lidar A is set as point cloud A, and the point cloud emitted by the secondary lidar B is set as point cloud B. The pose of the secondary lidar B is adjusted through the point cloud pose transformation device 102 to adjust the parameters of the secondary lidar B, such as x, y, z, roll, pitch, and yaw. The point cloud B after pose transformation is gradually overlapped with point cloud A to obtain the initial pose between point cloud A and point cloud B, and record it as the initial pose input for the fine registration of the lidar.

[0074] Through the rough calibration stage, the embodiments of the present application can quickly roughly align two or more point cloud datasets. Since this process does not require high precision and has a low computational complexity, an initial approximate transformation matrix can be found relatively quickly. This greatly reduces the spatial range and time that the subsequent fine registration algorithm needs to search, and can avoid being easily trapped in a local optimal solution due to too large an initial position difference. By first using the rough calibration to provide a better initial guess value, this risk can be significantly reduced, making the fine registration algorithm easier to converge to the global optimal solution. It can also remove some obvious mismatched points or noise points, thus providing a cleaner data input for the fine registration, thereby enhancing the robustness of the entire registration process.

[0075] Optionally, in an embodiment of the present application, the lidar calibration device 103 further includes: a first acquisition module, a fine registration module, and an optimization module.

[0076] Among them, the first acquisition module is used to acquire the pose data of multiple lidars under the initial pose parameter transformation instruction.

[0077] The fine registration module is used to perform probability density modeling on the target point cloud of the main lidar, and calculate the matching probability between the point cloud to be registered and the target point cloud of at least one secondary lidar with the pose data as the initial value, so as to obtain a matching score.

[0078] An optimization module for iteratively optimizing registration based on a matching score until a preset convergence condition is met to obtain a point cloud registration result.

[0079] Furthermore, embodiments of the present application can perform fine registration based on the result of rough calibration.

[0080] Embodiments of the present application can perform inter-frame registration of lidar, and read the x, y, z, roll, pitch, and yaw values after the above-mentioned rough calibration as the initial values for inter-frame registration. The inter-frame matching algorithm used in this case is a probability density matching algorithm, namely the NDT (Normal Distributions Transform) algorithm. To support offline matching and reduce the amount of calibration data collected, embodiments of the present application can set a special structure in the programs involved in the calculation to achieve that even for a single frame of lidar point cloud, the optimal pose transformation value can be obtained through iterative calculation. Among them, the pose transformation relationship between lidar A and lidar B is P1 = T 12 ·P2, where P1 represents the coordinates of the point cloud in the main lidar coordinate system, P2 represents the coordinates of the point cloud in the secondary lidar coordinate system, and T 12 represents the coordinate transformation from the main lidar to the secondary lidar. Expanding this equation gives the following formula:

[0081]

[0082] Among them, R is the rotation transformation from the main lidar to the secondary lidar, which is obtained through operations on roll, pitch, and yaw; t is the translation transformation from the main lidar to the secondary lidar, which is obtained from the above parameters x, y, z.

[0083] The second adjustment module of the point cloud pose transformation device 102 can then adjust the parameters of the lidar according to the pose parameter transformation instruction generated based on the above calculation results to complete the fine registration process, so as to reduce the calculation amount while ensuring the calibration effect, thereby quickly and accurately realizing the calibration of the lidar and the vehicle body coordinate system.

[0084] Optionally, in an embodiment of the present application, the lidar calibration device 103 further includes: a verification module.

[0085] Among them, the verification module is used to calculate the error between multiple new frames of point cloud data obtained by multiple lidars under the point cloud registration result, and use the error to verify whether the point cloud registration result meets the preset qualified conditions.

[0086] Furthermore, embodiments of the present application can use the pose transformation matrix T 12 between lidar A and lidar B obtained after fine registration for verification. If it is qualified, the calibration is completed.

[0087] During the verification process, the verification module of the embodiments of the present application can acquire new multi-frame point cloud data again in the accurately registered pose, and calculate the error between the multi-frame point cloud data. If the errors are all less than a certain threshold, the current registration is qualified, so as to further ensure the reliability of the calibration result.

[0088] Among them, the error threshold can be set accordingly according to the actual situation, and no specific limitation is made here.

[0089] Combined Figures 2 to 4 As shown, the working principle of the calibration system 100 of the vehicle-mounted multi-lidar of the embodiments of the present application will be elaborated in detail with an embodiment.

[0090] As Figure 2 and Figure 3 As shown, the scene calibration module 101 of the embodiments of the present application may include: reflection column 1, reflection column 2, reflection column 3, rear wheel groove 4, rear wheel groove baffle 5, front wheel groove 6, front wheel groove baffle 7, center line 8, calibration chassis 9, reflective sticker 10, reflective sticker 11, reflective sticker 12, chassis reflective sticker 13 and chassis reflective sticker 14.

[0091] Among them, the installation position of reflection column 1 is on the center line 8 of the calibration chassis 9, about 20 m away from the center point P of the rear wheel groove 4 (the specific data is adjustable). There is a reflective sticker 10 in the center part of reflection column 1;

[0092] The installation position of reflection column 2 is about 1 m to the left of the center line 8 of the calibration chassis 9. There is a reflective sticker 11 at its center position. Since the lidar point cloud carries the intensity information of the scanned object, it is used to mark the point cloud where the lidar hits this area;

[0093] The installation position of reflection column 3 is about 1 m to the right of the center line 8 of the calibration chassis 9, which is symmetrical to reflection column 2. The three reflection columns present a triangle, and there is a reflective sticker 12 at its center position. Since the lidar point cloud carries the intensity information of the scanned object, it is used to mark the point cloud where the lidar hits this area;

[0094] The function of the rear wheel groove 4 is to fix the vehicle rear wheels in the front-rear direction;

[0095] The rear wheel groove baffle 5 is a movable device for fixing the vehicle rear wheels in the left-right direction. The advantage of the movable device is that it can adapt to the calibration of vehicles with different wheelbases;

[0096] The front wheel groove 6 can fix the vehicle front wheels in the front-rear direction. The front wheel groove is movable and is used for calibrating vehicles with different wheelbases;

[0097] The front wheel groove baffle 7 is a movable device for fixing the vehicle front wheels in the left-right direction. The advantage of the movable device is that it can adapt to the calibration of vehicles with different wheelbases;

[0098] The structure of the calibration chassis 9 is a rectangular calibration base plate, and the material can be steel or other hard objects. It is the basis of the scene calibration device 101 and is used to fix the vehicle to be calibrated.

[0099] Both the chassis reflective stickers 13 and 14 are arranged on the calibration chassis 9.

[0100] The center point P of the rear slot of the calibration chassis 9 is the coordinate center point of the entire device 101, and its coordinates are set as P(0, 0, -h). Here, h represents the distance between the center of the rear axle of the vehicle and the center point of the calibration chassis. When calibrating, the vehicle is fixed through the movable wheel slots. After fixing, the projection point of the center position of the vehicle's rear axle on the calibration chassis 9 coincides with the center point of the rear wheel slot, and the center of the rear axle of the calibrated vehicle is the calibration origin P B (0, 0, 0).

[0101] Taking manual adjustment as an example, the point cloud pose transformation device 102 can be as Figure 4 shown. There are input windows for 6 parameters: x, y, z, roll, pitch, and yaw on it. After inputting the initial values, the numerical values can be adjusted by the up and down arrow buttons on the right side of the digital box. The background working principle of the point cloud pose transformation device 102:

[0102] The program of the point cloud pose transformation device 102 reads the 6 parameters in the tool box in real time and converts the 6 parameters in the tool box into a pose transformation matrix T. Suppose the point cloud coordinate in the B lidar coordinate system is P B , and the point cloud coordinate in the A lidar coordinate system is P A . Then, through the pose transformation matrix T, P A = T·P B can be used to convert the coordinate of the point cloud in the B lidar coordinate system to the A lidar coordinate system; the point cloud topic after conversion is published through the ros interface; for point cloud visualization, the ros-built-in rviz tool can be used.

[0103] Among them, P A (x a , y a , z a ), P B (x b , y b , z b ),

[0104] The lidar calibration device 103 can include three stages: rough calibration, fine registration, and verification.

[0105] At the beginning of calibration, the embodiment of the present application can drive the vehicle to be calibrated onto the scene calibration device 101, align the center of the rear axle of the vehicle to be calibrated with the center point of the calibration chassis 9, and then fix the vehicle to be calibrated. During the alignment process, a plumb line device can be placed at the center of the rear axle of the vehicle to be calibrated to align with the center point of the calibration chassis 9. Power on the lidar, check whether the data transmission is correct, open rviz to subscribe to the point cloud, and the point cloud can be displayed in real time. Open the point cloud pose transformation device 102 and its background program, temporarily set the values of its six parameters to 0, click the numerical adjustment button, subscribe to the adjusted point cloud in rviz, and check whether it changes in real time. After everything is normal, start the calibration work.

[0106] In the rough calibration stage, the embodiment of the present application can set the point clouds emitted by different lidars to different colors. Here, the point cloud emitted by lidar A (the main lidar) is set as point cloud A, and the point cloud emitted by lidar B is set as point cloud B. The function of the point cloud pose transformation tool is to transform point cloud B into point cloud B' through pose transformation. The coordinate of a certain point in point cloud A is P A , and the coordinate of a certain point in point cloud B is P B , and the coordinate of a certain point after the pose transformation of point cloud B is P B’ . Adjust the x, y, z, roll, pitch, and yaw buttons of the pose transformation tool so that point cloud B' gradually coincides with point cloud A or the poses are relatively close, that is, P A = T · P B . Obtain the initial pose T between point cloud A and point cloud B and record it, which is used as the initial pose input for the fine registration of the lidar.

[0107] In the fine registration stage, start the lidar inter-frame registration program, read the parameter values of x, y, z, roll, pitch, and yaw in the above rough calibration step as the initial values of the inter-frame registration program and give them to the point cloud registration program. The inter-frame matching algorithm used this time is the probability density matching algorithm. Among them, the pose transformation relationship between lidar A and lidar B is P A = T AB · P B , where P A represents the coordinate of the point cloud in the main lidar coordinate system, P B represents the coordinate of the point cloud in the secondary lidar coordinate system, and T AB represents the coordinate transformation from the main lidar to the secondary lidar. Expanding this formula gives the following formula:

[0108]

[0109] Among them, R is the rotation transformation from the main lidar to the secondary lidar, which is obtained through operations on roll, pitch, and yaw; t is the translation transformation from the main lidar to the secondary lidar, which is obtained from the above parameters x, y, and z.

[0110] After the calibration is completed, record the calibration parameters x, y, z, roll, pitch, and yaw in the calibration program.

[0111] In the verification stage, the embodiments of the present application can observe whether the point cloud A and the point cloud B' coincide. The specific observation method is as follows: First, check whether the two point clouds projected onto the calibration floor are flush. Second, check whether the three calibration rods in the point cloud A and the calibration rods in the point cloud B' coincide completely from different dimensions.

[0112] Reflective stickers are pasted on each of the three reflective columns. Assume that the coordinates of the center point of the reflective sticker 10 pasted on the reflective column 1 in the main lidar coordinate system are P A10 (x A10 , y A10 , z A10 ), the coordinates of the center point of the reflective sticker 11 pasted on the reflective column 2 in the main lidar coordinate system are P A11 (x A11 , y A11 , z A11 ), the coordinates of the center point of the reflective sticker 12 pasted on the reflective column 3 in the main lidar coordinate system are P A12 (x A12 , y A12 , z A12 ), the coordinates of the center point of the chassis reflective sticker 13 in the main lidar coordinate system are P A13 (x A13 , y A13 , z A13 ), the coordinates of the center point of the chassis reflective sticker 14 in the main lidar coordinate system are P A14 (x A14 , y A14 , z A14 ).

[0113] The coordinates of the point cloud projected onto the reflective sticker 10 pasted on the reflective column 1 in the B lidar coordinate system are P B10 (x B10 , y B10 , z B10 ), after the point cloud pose transformation, its coordinates in the A lidar coordinate system are P B’10 (x B’10 , y B’10 , z B’10 ); the coordinates of the point cloud projected onto the reflective sticker 11 pasted on the reflective column 2 in the B lidar coordinate system are P B11 (xB11 , y B11 , z B11 ), after the point cloud pose transformation, its coordinates in the A lidar coordinate system are P B’11 (x B’11 , y B’11 , z B’11 ); the coordinates of the point cloud on the retroreflective sticker 12 on the retroreflective column 3 in the B lidar coordinate system are P B12 (x B12 , y B12 , z B12 ), after the point cloud pose transformation, its coordinates in the A lidar coordinate system are P B’12 (x B’12 , y B’12 , z B’12 ); the coordinates of the point cloud on the chassis retroreflective sticker 13 in the B lidar coordinate system are P B13 (x B13 , y B13 , z B13 ), after the point cloud pose transformation, its coordinates in the A lidar coordinate system are P B’13 (x B’13 , y B’13 , z B’13 ); the coordinates of the point cloud on the chassis retroreflective sticker 14 in the B lidar coordinate system are P B14 (x B14 , y B14 , z B14 ), after the point cloud pose transformation, its coordinates in the A lidar coordinate system are P B’14 (x B’14 , y B’14 , z B’14 ).

[0114] Extract multiple frames of point cloud data to calculate the difference between P B’10 (x B’10 , y B’10 , z B’10 ) and P A10 (x A10 , y A10 , z A10 ). If the difference of multiple frame results is within 1 cm, the calibration result is qualified. By analogy, calculate other points. After all points are qualified, the calibration verification is completed.

[0115] In summary, in the embodiment of the present application, due to the scenario calibration device 101, the vehicle to be calibrated can be fixed in a calibration scenario that meets the preset data acquisition conditions, so as to reduce the influence of the environment on the calibration effect by using the fixed calibration scenario. The lidar calibration device 103 performs registration between multiple lidars with the main lidar among the multiple lidars as a reference to determine the pose adjustment parameters of the remaining lidars. Then, the point cloud pose transformation device 102 is used to adjust the poses of the multiple lidars to complete the actual calibration task. Therefore, it is possible to solve the technical problems in the related art that it is difficult to balance the calibration efficiency, the computing power required for calibration, and the calibration effect, and there is a certain dependence on the static calibration environment, resulting in relatively high calibration costs and calibration difficulties in a large number of calibration scenarios. The technical effect of reducing the influence of the environment on the calibration result by fixing the scenario is achieved, so as to reduce the amount of calculation and improve the calibration efficiency while ensuring the calibration effect.

[0116] Through the description of the above embodiments, those skilled in the art can clearly understand that the system according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0117] The embodiment of the present application also provides a calibration method for a vehicle-mounted multi-lidar.

[0118] As Figure 5 shown, the calibration method for a vehicle-mounted multi-lidar includes the following steps:

[0119] In step S501, based on the target calibration task, the point cloud data scanned by multiple lidars in the calibration scenario is acquired.

[0120] In step S502, with the main lidar among the multiple lidars as a reference, point cloud registration of the multiple lidars is performed based on the point cloud data to obtain a point cloud registration result.

[0121] In step S503, a corresponding pose parameter transformation instruction is generated based on the point cloud registration result to adjust the pose parameters of the multiple lidars by using the pose parameter transformation instruction until the target calibration task is completed.

[0122] For the description of the features in the corresponding embodiment of the calibration method for a vehicle-mounted multi-lidar, reference can be made to the relevant description of the corresponding embodiment of the calibration system for a vehicle-mounted multi-lidar, which will not be elaborated here one by one.

[0123] The embodiment of the present application also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above embodiments of the calibration method for a vehicle-mounted multi-lidar.

[0124] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any of the above-described embodiments of the calibration method for vehicle-mounted multi-lidar when running.

[0125] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0126] An embodiment of the present application further provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the calibration method for vehicle-mounted multi-lidar.

[0127] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the calibration method for vehicle-mounted multi-lidar.

[0128] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0129] The above has introduced in detail a calibration system, method, electronic device, and medium for vehicle-mounted multi-lidar provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A calibration system for a vehicle-mounted multi-lidar, characterized in that, Including: A scene calibration device for fixing a vehicle to be calibrated in a calibration scene that meets preset data acquisition conditions based on a target calibration task; A point cloud pose transformation device for adjusting the pose parameters of multiple lidars; A lidar calibration device for obtaining point cloud data scanned by the multiple lidars in the calibration scene, taking the master lidar among the multiple lidars as a reference, performing point cloud registration of the multiple lidars based on the point cloud data to obtain a point cloud registration result, generating a corresponding pose parameter transformation instruction based on the point cloud registration result, and using the pose parameter transformation instruction to control the point cloud pose transformation device until the target calibration task is completed.

2. The system according to claim 1, wherein The scene calibration device includes: A fixed chassis provided with wheel grooves on which the vehicle to be calibrated is fixed in the wheel grooves so that the center point of the rear axle of the vehicle to be calibrated corresponds to the target projection point of the fixed chassis; Multiple reflection columns fixed on the fixed chassis, wherein the positional relationship between the multiple reflection columns satisfies a preset distribution condition, and a reflection sticker is provided in the calibration area of each reflection column and / or the calibration area of the fixed bottom plate to perform point cloud registration in combination with the position information of the reflection sticker and the point cloud data.

3. The system according to claim 2, characterized in that, The lidar calibration device includes: A conversion module for converting the point cloud coordinates of at least one slave lidar other than the master lidar among the multiple lidars to the point cloud coordinate system of the master lidar to obtain the transferred point cloud corresponding to the at least one slave lidar; A rough calibration module for performing rough calibration of the multiple lidars based on the transferred point cloud and the point cloud data of the master lidar to obtain a rough calibration result; A generation module for generating a corresponding initial pose parameter transformation instruction based on the rough calibration result and controlling the point cloud pose transformation device based on the initial pose parameter transformation instruction.

4. The system according to claim 3, wherein The lidar calibration device further includes: A first acquisition module for acquiring the pose data of the multiple lidars under the initial pose parameter transformation instruction; A fine registration module for performing probability density modeling on the target point cloud of the master lidar, using the pose data as an initial value, calculating the matching probability between the point cloud to be registered of the at least one slave lidar and the target point cloud to obtain a matching score; An optimization module for performing registration iterative optimization based on the matching score until a preset convergence condition is met to obtain the point cloud registration result.

5. The system according to claim 3, wherein The lidar calibration device further includes: A verification module for calculating the error between new multi-frame point cloud data acquired by the multiple lidars under the point cloud registration result and verifying whether the point cloud registration result meets a preset qualified condition using the error.

6. The system according to claim 4, wherein The point cloud pose transformation device includes: A first adjustment module for adjusting the multiple lidars based on the initial pose parameter transformation instruction; A second adjustment module for adjusting the multiple lidars based on the pose parameter transformation instruction.

7. The system according to claim 1, wherein The lidar calibration device includes: A second acquisition module, configured to acquire configuration parameters of each of the multiple lidars; A processing module, configured to perform time synchronization processing on each of the lidars based on the configuration parameters.

8. A calibration method for a vehicle-mounted multi-lidar, characterized in that, Using the calibration system for vehicle-mounted multiple lidars according to any one of claims 1-7, wherein the method comprises the following steps: Based on a target calibration task, acquiring point cloud data scanned by the multiple lidars in the calibration scenario; Taking the master lidar among the multiple lidars as a reference, performing point cloud registration of the multiple lidars based on the point cloud data to obtain a point cloud registration result; Generating a corresponding pose parameter transformation instruction based on the point cloud registration result, and using the pose parameter transformation instruction to adjust the pose parameters of the multiple lidars until the target calibration task is completed.

9. An electronic device, characterized in that, Comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the calibration method for vehicle-mounted multiple lidars according to claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to be used for implementing the calibration method for vehicle-mounted multiple lidars according to claim 8.

Images