A multi-lidar calibration method and system based on dynamic target point cloud

Abstract

The present invention relates to the technical field of laser radar calibration, and particularly to a multi-laser radar calibration method and system based on a dynamic target point cloud. The method comprises acquiring first information and second information, the first information comprising point cloud data collected by a first laser radar, and the second information comprising point cloud data collected by a second laser radar; identifying the first information and the second information using a vehicle perception algorithm to obtain an identification result, the identification result comprising trajectory points of the vehicle; determining the trajectories of all vehicles included in the two laser radars based on the identification result to obtain trajectory information; performing trajectory matching based on the trajectory information to obtain a vehicle trajectory matching result between the two laser radars; and calibrating the first laser radar and the second laser radar based on the vehicle trajectory matching result between the two laser radars. The present invention can achieve a high-precision multi-laser radar calibration effect even when the laser radars are widely distributed and have a low degree of overlap.

Description

Technical Field

[0001] The present invention relates to the field of laser radar calibration technology, and in particular to a multi-laser radar calibration method and system based on dynamic target point cloud. Background Art

[0002] With the increasing deployment of roadside sensors such as lidar to monitor traffic flow and provide beyond-visual-range awareness for autonomous vehicles, it has become feasible to achieve continuous vehicle detection over a wider range by fusing data from multiple roadside lidars. However, accurate multi-lidar calibration is crucial for effective multi-lidar fusion. Most existing methods rely on static background point clouds, requiring high overlap and distinct features in the detection area. Therefore, in large-scale roadside inspection scenarios with low overlap, such as highways and overpasses, it is difficult to utilize background features for calibration and achieve high-precision multi-lidar calibration. Summary of the Invention

[0003] The purpose of the present invention is to provide a multi-lidar calibration method and system based on dynamic target point cloud to improve the above problems.

[0004] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0005] On the one hand, an embodiment of the present application provides a multi-lidar calibration method based on a dynamic target point cloud, the method comprising:

[0006] Acquiring first information and second information, wherein the first information includes point cloud data collected by a first laser radar, and the second information includes point cloud data collected by a second laser radar, wherein the first laser radar and the second laser radar are arranged on a highway at intervals;

[0007] Identify the first information and the second information using a vehicle perception algorithm to obtain an identification result, the identification result including a trajectory point of the vehicle;

[0008] Determine the trajectories of all vehicles included in the two laser radars according to the recognition results to obtain trajectory information;

[0009] Performing trajectory matching based on the trajectory information to obtain a vehicle trajectory matching result between the two laser radars;

[0010] Calibrate the first laser radar and the second laser radar according to the vehicle trajectory matching result between the two laser radars;

[0011] The process of performing trajectory matching based on the trajectory information to obtain a vehicle trajectory matching result between the two laser radars includes:

[0012] Projecting the first information and the second information into two dimensions to obtain projected point cloud data, wherein the projected point cloud data includes trajectory information corresponding to the first laser radar and trajectory information corresponding to the second laser radar;

[0013] Rasterizing the projected point cloud data to obtain a first grid unit and a second grid unit;

[0014] Matching the trajectory information included in the first grid unit and the second grid unit, and determining mutually matching grids to obtain a matching result;

[0015] The first laser radar and the second laser radar are calibrated according to the vehicle trajectory matching result between the two laser radars, including:

[0016] Determine mutually matching grids in the first grid unit and the second grid unit according to the matching result to obtain overlapping grids;

[0017] Searching for two trajectory points corresponding to the same vehicle at the same time in the overlapping grids to obtain a first vehicle point cloud and a second vehicle point cloud;

[0018] The first vehicle point cloud and the second vehicle point cloud are stacked into the overlapping grid, and the first laser radar and the second laser radar are calibrated according to the stacked point cloud information.

[0019] In a second aspect, an embodiment of the present application provides a multi-lidar calibration system based on a dynamic target point cloud, the system comprising:

[0020] an acquisition module, configured to acquire first information and second information, wherein the first information includes point cloud data collected by a first laser radar, and the second information includes point cloud data collected by a second laser radar, wherein the first laser radar and the second laser radar are arranged on a highway at intervals;

[0021] a first processing module, configured to identify the first information and the second information using a vehicle perception algorithm to obtain an identification result, wherein the identification result includes a trajectory point of the vehicle;

[0022] A second processing module is used to determine the tracks of all vehicles included in the two laser radars according to the recognition results to obtain track information;

[0023] A third processing module is used to perform trajectory matching according to the trajectory information to obtain a matching result;

[0024] a fourth processing module, configured to calibrate the first laser radar and the second laser radar according to a vehicle trajectory matching result between the two laser radars;

[0025] Wherein, the third processing module includes:

[0026] A first processing unit is configured to project the first information and the second information into two dimensions to obtain projected point cloud data, wherein the projected point cloud data includes trajectory information corresponding to the first laser radar and trajectory information corresponding to the second laser radar;

[0027] A second processing unit is configured to rasterize the projected point cloud data to obtain a first raster unit and a second raster unit;

[0028] a third processing unit, configured to match the trajectory information included in the first grid unit and the second grid unit, and determine mutually matching grids to obtain a matching result;

[0029] Wherein, the fourth processing module includes:

[0030] a twelfth processing unit, configured to determine mutually matching grids in the first grid unit and the second grid unit according to the matching result, to obtain overlapping grids;

[0031] a thirteenth processing unit, configured to search the overlapping grids for two trajectory points corresponding to the same vehicle at the same time, to obtain a first vehicle point cloud and a second vehicle point cloud;

[0032] The fourteenth processing unit stacks the first vehicle point cloud and the second vehicle point cloud into the overlapping grid, and calibrates the first laser radar and the second laser radar according to the stacked point cloud information.

[0033] In a third aspect, embodiments of the present application provide a multi-lidar calibration device based on a dynamic target point cloud, the device comprising a memory and a processor. The memory is configured to store a computer program; the processor is configured to implement the steps of the multi-lidar calibration method based on a dynamic target point cloud when executing the computer program.

[0034] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned multi-lidar calibration method based on dynamic target point cloud are implemented.

[0035] The beneficial effects of the present invention are:

[0036] The present invention collects point cloud data from two lidars, identifies the vehicle's trajectory points based on the point cloud data of the two lidars, obtains an identification result, and determines the vehicle's trajectory based on the identification result. The trajectory information is then matched to identify the overlapping area between the two lidars, thereby obtaining a matching result. Finally, based on the matching result, high-precision calibration of the two widely distributed lidars is achieved.

[0037] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the embodiments of the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 Schematic diagram of the process of the multi-lidar calibration method based on dynamic target point cloud described in an embodiment of the present invention.

[0040] Figure 2 Schematic diagram of the structure of a multi-lidar calibration system based on dynamic target point cloud described in an embodiment of the present invention.

[0041] Figure 3 Schematic diagram of the structure of a multi-lidar calibration device based on dynamic target point cloud described in an embodiment of the present invention.

[0042] Figure 4 Schematic diagram of point cloud stack.

[0043] Labels in the figure: 800, multi-lidar calibration equipment based on dynamic target point cloud; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component; 901 acquisition module; 902, first processing module; 903, second processing module; 904, third processing module; 905, fourth processing module. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance.

[0046] Example 1:

[0047] This embodiment provides a multi-lidar calibration method based on dynamic target point cloud. It can be understood that in this embodiment, a scene can be laid out, for example: on a highway, when the lidars are widely distributed and the overlap is low, it is necessary to calibrate two adjacent lidars.

[0048] See also Figure 1 , the figure shows that the method includes steps S1, S2, S3, S4 and S5, which specifically include:

[0049] Step S1: Acquire first information and second information, wherein the first information includes point cloud data collected by a first laser radar, and the second information includes point cloud data collected by a second laser radar, wherein the first laser radar and the second laser radar are arranged on a highway at intervals;

[0050] In this step, the first laser radar and the second laser radar are both laser radars to be calibrated.

[0051] Step S2: using a vehicle perception algorithm to identify the first information and the second information to obtain an identification result, wherein the identification result includes a trajectory point of the vehicle;

[0052] In this step, a specific implementation is to use a DBSCAN-based vehicle perception algorithm to identify the vehicle from the first and second information. The identification result includes the vehicle's track points, vehicle head direction, and bounding box. The point cloud within the bounding box is the vehicle surface point cloud. Each track point corresponds to a set of vehicle surface point clouds. In this way, a set of track points in each of the two lidar coordinate systems can be extracted from the point cloud data. It is understood that using a DBSCAN-based vehicle perception algorithm to identify a vehicle is a technical solution well known to those skilled in the art and will not be detailed here.

[0053] Step S3: determining the trajectories of all vehicles included in the two laser radars according to the recognition results to obtain trajectory information;

[0054] In this step, the Kalman filter is used to track the respective trajectory point sets in the two lidar coordinate systems to obtain trajectory information. Kalman filtering can combine the historical state of the target with the current observation results to realize dynamic estimation and prediction of the target trajectory, thereby improving the robustness and continuity of tracking. The specific process is as follows: 1) In each radar coordinate system, the Kalman filter is used to predict and update the state of the trajectory points, and estimate the current position and speed of each vehicle and other information; 2) In the new frame, the Euclidean distance between the currently detected trajectory point and the predicted position is calculated, and it is judged whether it belongs to an existing trajectory based on the spatial distance and time interval; 3) The Hungarian algorithm is used to optimally associate the new trajectory point with the existing trajectory: if the association is successful, the point is added to the corresponding trajectory and the filter state is updated; if the association cannot be made, a new trajectory path is initialized. Through this method, the continuous tracking of the trajectory points of the same vehicle can be achieved, and the complete trajectory path can be gradually constructed. After all the trajectory points are processed and associated, a complete trajectory set for each vehicle is finally formed. It can be understood that the trajectory set of all vehicles in the first lidar coordinate system is recorded as , the trajectory set of all vehicles in the second laser radar coordinate system is recorded as .

[0055] Step S4: performing trajectory matching according to the trajectory information to obtain a matching result;

[0056] The step S4 also includes steps S41, S42 and S43, which specifically include:

[0057] Step S41: Projecting the first information and the second information into two dimensions to obtain projected point cloud data, wherein the projected point cloud data includes trajectory information corresponding to the first laser radar and trajectory information corresponding to the second laser radar;

[0058] Step S42: rasterizing the projected point cloud data to obtain a first grid unit and a second grid unit;

[0059] In this step, the projected point cloud data is rasterized according to the size of the side length l. The side length l of the square grid is limited to less than 2m. Since the width of a vehicle is generally about two meters, and there is a certain distance between vehicles during driving, this ensures that at most one vehicle passes through each grid at any time. The first grid unit set is recorded as , the second grid cell set is recorded as , each grid has its corresponding number (i and j), the coordinates of the center point and the side length l.

[0060] Step S43: Match the trajectory information included in the first grid unit and the second grid unit, and determine mutually matching grids to obtain a matching result.

[0061] Step S43 further includes steps S431, S432, S433, S434, and S435, which specifically include:

[0062] Step S431: Determine the trajectory point of each grid in the first grid unit according to the trajectory information corresponding to the first laser radar, and obtain first trajectory point information;

[0063] In this step, the trajectory set of all vehicles in the first laser radar coordinate system is recorded as , it is possible to determine which tracks are contained in each grid in each first grid unit.

[0064] Step S432: Determine the trajectory point of each grid in the second grid unit according to the trajectory information corresponding to the second laser radar, and obtain second trajectory point information;

[0065] In this step, the trajectory set of all vehicles in the second laser radar coordinate system is recorded as , it is possible to determine which tracks are contained in each grid in each second grid unit.

[0066] Step S433: determining a first trajectory spatiotemporal distribution graph corresponding to each grid according to the first trajectory point information, wherein the first trajectory spatiotemporal distribution graph is used to indicate whether a trajectory point exists in the grid at each moment;

[0067] The step S433 further includes steps S4331, S4332, and S4333, which specifically include:

[0068] Step S4331: determining all trajectory points existing in each grid according to the first trajectory point information;

[0069] Step S4332: Determine the time information corresponding to each trajectory point;

[0070] Step S4333: Set the time information corresponding to the existing trajectory point to 1, and the time information corresponding to the non-existing trajectory point to 0, and draw the first trajectory spatiotemporal distribution diagram.

[0071] In this step, a vehicle will pass through each grid surface. At each moment, if a vehicle passes through the grid surface, the value of the grid at that moment is 1, otherwise it is 0. In this way, based on the trajectory points of vehicles passing through a grid at different moments, the spatiotemporal distribution of the vehicle distribution of each grid can be obtained, thereby drawing the first spatiotemporal distribution map of the trajectory. The first spatiotemporal distribution map of the trajectory can reflect the time series of the trajectory appearance in the grid.

[0072] Step S434: determining a second trajectory spatiotemporal distribution graph corresponding to each grid according to the second trajectory point information;

[0073] In this step, the principle of drawing the second trajectory spatiotemporal distribution map is the same as that of the first trajectory spatiotemporal distribution map, so it will not be described here in detail.

[0074] Step S435 : determining whether grids match based on the first trajectory spatiotemporal distribution graph and the second trajectory spatiotemporal distribution graph, and obtaining a matching result.

[0075] In this step, the Pearson correlation coefficient is used to calculate the correlation coefficient of the spatiotemporal distribution graph of the trajectories between all grids of the two lidars. When the correlation coefficient is greater than the preset correlation coefficient threshold, the grids are matched as associated grids. It should be noted that the specific process of calculating the Pearson correlation coefficient is as follows:

[0076] ;

[0077] in, is the Pearson correlation coefficient, which is used to measure the similarity of the time series of trajectory occurrence on grid i and grid j in the first grid cell and the second grid cell. and Represent the spatiotemporal distribution graph of the first trajectory and the spatiotemporal distribution graph of the second trajectory respectively. and is the spatiotemporal distribution of the trajectories of grid i and grid j of the first grid unit and the second grid unit over the entire time period and The average value of . n is the total number of time frames.

[0078] Step S5: calibrate the first laser radar and the second laser radar according to the vehicle trajectory matching result between the two laser radars.

[0079] In this step, a group of grids with the highest Pearson correlation coefficient is selected for subsequent registration, which is recorded as and .

[0080] Step S5 further includes steps S51, S52, and S53, which specifically include:

[0081] Step S51: determining mutually matching grids in the first grid unit and the second grid unit according to the matching result to obtain overlapping grids;

[0082] Step S52: searching for two trajectory points corresponding to the same vehicle at the same time in the overlapping grid to obtain a first vehicle point cloud and a second vehicle point cloud;

[0083] In this step, and The grid represents the same area of the overlapped part of the two lidars. Therefore, at the same moment, the trajectory points passed by the two associated grids are the trajectory points of the same vehicle. The corresponding point clouds can be matched based on the trajectory points.

[0084] Step S53: stacking the first vehicle point cloud and the second vehicle point cloud into the overlapping grid, and calibrating the first laser radar and the second laser radar according to the stacked point cloud information;

[0085] The step S53 further includes steps S531, S532, and S533, which specifically include:

[0086] Step S531: stacking the point clouds of the first vehicle at different times into overlapping grids to obtain stacked first point cloud information;

[0087] In this step, the point cloud stack is as follows Figure 4 As shown in the figure, the point clouds of the same vehicle at different times (such as t-1, t-2...) are rotated and translated to the position of the point cloud at time t. This method can increase the point cloud density on the vehicle surface and thus improve the accuracy of point cloud registration.

[0088] Step S532: stacking the point clouds of the second vehicle at different times into overlapping grids to obtain stacked second point cloud information;

[0089] Step S533: calibrate the first laser radar and the second laser radar according to the stacked first point cloud information and the stacked second point cloud information.

[0090] In this embodiment, since the distance between the two lidars is far and the overlap is low, the point cloud in the overlapping area is very sparse, and it is impossible to use the sparse point cloud at the edge of the lidar detection to achieve accurate multi-lidar calibration. Therefore, the present invention superimposes the point cloud data in multiple time frames to increase the density of the target surface point cloud, thereby achieving higher-precision point cloud alignment.

[0091] Step S533 further includes steps S5331, S5332, S5333, and S5334, which specifically include:

[0092] Step S5331: Rotate and translate the stacked second point cloud information to the position of the stacked first point cloud information to obtain a rotation and translation matrix;

[0093] In this step, the two coordinate points are completely aligned and the vehicle head direction is consistent. The specific rotation and translation matrix calculation process is:

[0094] ;

[0095] in, represents the rotation and translation matrix, is the angular difference between the vehicle’s head directions in the two coordinate systems, , , The translation of the relative position change of the vehicle in the two coordinate systems is used to transform the coordinates of the second point cloud information after stacking using the calculated rotation and translation matrix to obtain the point cloud after coarse registration.

[0096] Step S5332: Obtain a precise registration transformation matrix using the ICP point cloud registration algorithm;

[0097] In this step, the coarsely registered point cloud and the first point cloud information after stacking have been roughly aligned in space. The ICP point cloud registration algorithm is then used to achieve further fine registration. Specifically, the ICP algorithm will find the closest point pairs between the two point cloud datasets through a continuous iterative process and calculate the transformation (translation and rotation) between each pair of points to minimize the error between the point pairs. As the iterations proceed, the accuracy of the registration gradually improves until it converges to a more accurate fine registration transformation matrix. , that is, the optimal alignment state of the two point cloud data in space.

[0098] Step S5333: Calculate based on the rotation and translation matrix and the fine registration transformation matrix to obtain a global transformation matrix;

[0099] In this step, the global transformation matrix is:

[0100] ;

[0101] In the above formula, is the global transformation matrix, is the fine registration transformation matrix, is the rotation and translation matrix.

[0102] Step S5334: calibrate the first lidar and the second lidar according to the global transformation matrix.

[0103] The global transformation matrix calculated from the dynamic vehicle target point cloud can be extended to the static background point cloud to map all point clouds collected by the second lidar to the coordinate system of the first lidar, thereby achieving accurate calibration between the two lidars.

[0104] The present invention analyzes the spatiotemporal similarity of vehicle trajectory distribution in different lidar detection areas, identifies the overlapping areas between the lidars, and associates the same overlapping areas in the coordinate systems of the two lidars. After the overlapping areas of the two lidars are associated, the trajectories of the same vehicle are associated, and then the association of the point clouds corresponding to the trajectories can be achieved. By superimposing the point cloud data in multiple time frames, the density of the target surface point cloud is increased, thereby achieving more accurate point cloud alignment. The present invention can achieve high-precision multi-lidar calibration effects even when the lidars are widely distributed and have low overlap.

[0105] Example 2:

[0106] like Figure 2 As shown, this embodiment provides a multi-lidar calibration system based on a dynamic target point cloud, the system including an acquisition module 901, a first processing module 902, a second processing module 903, a third processing module 904 and a fourth processing module, which specifically include:

[0107] An acquisition module 901 is configured to acquire first information and second information, wherein the first information includes point cloud data collected by a first laser radar, and the second information includes point cloud data collected by a second laser radar, wherein the first laser radar and the second laser radar are spaced apart on a highway;

[0108] A first processing module 902 is configured to identify the first information and the second information using a vehicle perception algorithm to obtain an identification result, wherein the identification result includes a vehicle trajectory point;

[0109] A second processing module 903 is configured to determine the trajectories of all vehicles included in the two laser radars according to the recognition results to obtain trajectory information;

[0110] The third processing module 904 is used to perform trajectory matching according to the trajectory information to obtain a matching result;

[0111] The fourth processing module 905 is used to calibrate the first laser radar and the second laser radar according to the vehicle trajectory matching result between the two laser radars.

[0112] In a specific embodiment of the present disclosure, the third processing module further includes a first processing unit, a second processing unit and a third processing unit, which specifically include:

[0113] A first processing unit is configured to project the first information and the second information into two dimensions to obtain projected point cloud data, wherein the projected point cloud data includes trajectory information corresponding to the first laser radar and trajectory information corresponding to the second laser radar;

[0114] A second processing unit is configured to rasterize the projected point cloud data to obtain a first raster unit and a second raster unit;

[0115] The third processing unit is configured to match the trajectory information included in the first grid unit and the second grid unit, and determine grids that match each other to obtain a matching result.

[0116] In a specific embodiment of the present disclosure, the third processing unit further includes a fourth processing unit, a fifth processing unit, a sixth processing unit, a seventh processing unit, and an eighth processing unit, which specifically include:

[0117] A fourth processing unit is configured to determine a trajectory point existing in each grid in the first grid unit according to the trajectory information corresponding to the first laser radar, and obtain first trajectory point information;

[0118] A fifth processing unit is configured to determine a trajectory point existing in each grid in the second grid unit according to the trajectory information corresponding to the second laser radar, and obtain second trajectory point information;

[0119] a sixth processing unit, configured to determine a first trajectory spatiotemporal distribution graph corresponding to each grid according to the first trajectory point information, wherein the first trajectory spatiotemporal distribution graph is used to indicate whether a trajectory point exists in the grid at each moment;

[0120] a seventh processing unit, configured to determine a second trajectory spatiotemporal distribution map corresponding to each grid according to the second trajectory point information;

[0121] An eighth processing unit is configured to determine whether grids match according to the first trajectory spatiotemporal distribution graph and the second trajectory spatiotemporal distribution graph, and obtain a matching result.

[0122] In a specific embodiment of the present disclosure, the sixth processing unit further includes a ninth processing unit, a tenth processing unit, and an eleventh processing unit, which specifically include:

[0123] a ninth processing unit, configured to determine all trajectory points existing in each grid according to the first trajectory point information;

[0124] a tenth processing unit, configured to determine the time information corresponding to each trajectory point;

[0125] The eleventh processing unit is configured to set the time information corresponding to the existing trajectory point to 1 and the time information corresponding to the non-existing trajectory point to 0, thereby drawing a first trajectory spatiotemporal distribution diagram.

[0126] In a specific embodiment of the present disclosure, the fourth processing module further includes a twelfth processing unit, a thirteenth processing unit, and a fourteenth processing unit, which specifically include:

[0127] a twelfth processing unit, configured to determine mutually matching grids in the first grid unit and the second grid unit according to the matching result, to obtain overlapping grids;

[0128] a thirteenth processing unit, configured to search the overlapping grids for two trajectory points corresponding to the same vehicle at the same time, to obtain a first vehicle point cloud and a second vehicle point cloud;

[0129] The fourteenth processing unit stacks the first vehicle point cloud and the second vehicle point cloud into the overlapping grid, and calibrates the first laser radar and the second laser radar according to the stacked point cloud information.

[0130] In a specific embodiment of the present disclosure, the fourteenth processing unit further includes a fifteenth processing unit, a sixteenth processing unit, and a seventeenth processing unit, which specifically include:

[0131] a fifteenth processing unit, configured to stack the point clouds of the first vehicle point cloud at different times into overlapping grids to obtain stacked first point cloud information;

[0132] a sixteenth processing unit, configured to stack the point clouds of the second vehicle point cloud at different times into overlapping grids to obtain stacked second point cloud information;

[0133] The seventeenth processing unit is used to calibrate the first laser radar and the second laser radar according to the stacked first point cloud information and the stacked second point cloud information.

[0134] In a specific embodiment of the present disclosure, the seventeenth processing unit further includes an eighteenth processing unit, a nineteenth processing unit, a twentieth processing unit, and a twenty-first processing unit, which specifically include:

[0135] An eighteenth processing unit, configured to rotate and translate the second point cloud information after the stacking to a position of the first point cloud information after the stacking, to obtain a rotation and translation matrix;

[0136] A nineteenth processing unit is used to obtain a precise registration transformation matrix using an ICP point cloud registration algorithm;

[0137] a twentieth processing unit, configured to perform calculations based on the rotation and translation matrix and the fine registration transformation matrix to obtain a global transformation matrix;

[0138] The twenty-first processing unit is used to calibrate the first laser radar and the second laser radar according to the global transformation matrix.

[0139] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0140] Example 3:

[0141] Corresponding to the above method embodiment, this embodiment also provides a multi-lidar calibration device based on a dynamic target point cloud. The multi-lidar calibration device based on a dynamic target point cloud described below and the multi-lidar calibration method based on a dynamic target point cloud described above can be referenced to each other.

[0142] Figure 3 FIG is a block diagram of a multi-lidar calibration device 800 based on a dynamic target point cloud according to an exemplary embodiment. Figure 3 As shown, the multi-lidar calibration device 800 based on dynamic target point cloud may include: a processor 801, a memory 802. The multi-lidar calibration device 800 based on dynamic target point cloud may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0143] The processor 801 is used to control the overall operation of the multi-lidar calibration device 800 based on a dynamic target point cloud to complete all or part of the steps in the multi-lidar calibration method based on a dynamic target point cloud. The memory 802 is used to store various types of data to support the operation of the multi-lidar calibration device 800 based on a dynamic target point cloud. Such data may include, for example, instructions for any application or method operating on the multi-lidar calibration device 800 based on a dynamic target point cloud, as well as application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or transmitted via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be a keyboard, a mouse, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the multi-lidar calibration device 800 based on the dynamic target point cloud and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 may include: Wi-Fi module, Bluetooth module, NFC module.

[0144] In an exemplary embodiment, the multi-lidar calibration device 800 based on dynamic target point cloud can be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned multi-lidar calibration method based on dynamic target point cloud.

[0145] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When executed by a processor, the program instructions implement the steps of the aforementioned multi-lidar calibration method based on a dynamic target point cloud. For example, the computer-readable storage medium may be the aforementioned memory 802 including the program instructions. The program instructions may be executed by the processor 801 of the multi-lidar calibration device 800 based on a dynamic target point cloud to implement the aforementioned multi-lidar calibration method based on a dynamic target point cloud.

[0146] Example 4:

[0147] Corresponding to the above method embodiment, a readable storage medium is also provided in this embodiment. The readable storage medium described below and the multi-lidar calibration method based on dynamic target point cloud described above can be referenced to each other.

[0148] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-lidar calibration method based on dynamic target point cloud of the above method embodiment.

[0149] The readable storage medium may specifically be any readable storage medium that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0150] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0151] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A multi-lidar calibration method based on dynamic target point cloud, characterized in that: include: Acquiring first information and second information, wherein the first information includes point cloud data collected by a first laser radar, and the second information includes point cloud data collected by a second laser radar, wherein the first laser radar and the second laser radar are arranged on a highway at intervals; Identify the first information and the second information using a vehicle perception algorithm to obtain an identification result, the identification result including a trajectory point of the vehicle; Determine the trajectories of all vehicles included in the two laser radars according to the recognition results to obtain trajectory information; Performing trajectory matching based on the trajectory information to obtain a vehicle trajectory matching result between the two laser radars; Calibrate the first laser radar and the second laser radar according to the vehicle trajectory matching result between the two laser radars; The process of performing trajectory matching based on the trajectory information to obtain a vehicle trajectory matching result between the two laser radars includes: Projecting the first information and the second information into two dimensions to obtain projected point cloud data, wherein the projected point cloud data includes trajectory information corresponding to the first laser radar and trajectory information corresponding to the second laser radar; Rasterizing the projected point cloud data to obtain a first grid unit and a second grid unit; Matching the trajectory information included in the first grid unit and the second grid unit, and determining mutually matching grids to obtain a matching result; The first laser radar and the second laser radar are calibrated according to the vehicle trajectory matching result between the two laser radars, including: Determine mutually matching grids in the first grid unit and the second grid unit according to the matching result to obtain overlapping grids; Searching for two trajectory points corresponding to the same vehicle at the same time in the overlapping grids to obtain a first vehicle point cloud and a second vehicle point cloud; The first vehicle point cloud and the second vehicle point cloud are stacked into the overlapping grid, and the first laser radar and the second laser radar are calibrated according to the stacked point cloud information.

2. The multi-lidar calibration method based on dynamic target point cloud according to claim 1, characterized in that: Determining mutually matching grids according to the first grid unit and the second grid unit includes: Determine the trajectory point of each grid in the first grid unit according to the trajectory information corresponding to the first laser radar, and obtain first trajectory point information; Determine the trajectory point of each grid in the second grid unit according to the trajectory information corresponding to the second laser radar, and obtain second trajectory point information; Determine a first trajectory spatiotemporal distribution graph corresponding to each grid according to the first trajectory point information, wherein the first trajectory spatiotemporal distribution graph is used to indicate whether a trajectory point exists in the grid at each moment; Determine a second trajectory spatiotemporal distribution map corresponding to each grid according to the second trajectory point information; Determine whether grids match based on the first trajectory spatiotemporal distribution graph and the second trajectory spatiotemporal distribution graph to obtain a matching result.

3. The multi-lidar calibration method based on dynamic target point cloud according to claim 2, characterized in that: Determining a first trajectory spatiotemporal distribution graph corresponding to each grid according to the first trajectory point information includes: Determine all trajectory points existing in each grid according to the first trajectory point information; Determine the time information corresponding to each trajectory point; The time information corresponding to the existing trajectory point is set to 1, and the time information corresponding to the non-existing trajectory point is set to 0, and the first trajectory spatiotemporal distribution map is drawn.

4. The multi-lidar calibration method based on dynamic target point cloud according to claim 1, characterized in that: Stacking the first vehicle point cloud and the second vehicle point cloud into the overlapping grid, and calibrating the first laser radar and the second laser radar according to the stacked point cloud information, including: Stacking the point clouds of the first vehicle at different times into overlapping grids to obtain stacked first point cloud information; Stacking the point clouds of the second vehicle at different times into overlapping grids to obtain stacked second point cloud information; The first laser radar and the second laser radar are calibrated according to the stacked first point cloud information and the stacked second point cloud information.

5. A multi-lidar calibration system based on dynamic target point cloud, characterized in that: include: an acquisition module, configured to acquire first information and second information, wherein the first information includes point cloud data collected by a first laser radar, and the second information includes point cloud data collected by a second laser radar, wherein the first laser radar and the second laser radar are arranged on a highway at intervals; a first processing module, configured to identify the first information and the second information using a vehicle perception algorithm to obtain an identification result, wherein the identification result includes a trajectory point of the vehicle; A second processing module is used to determine the tracks of all vehicles included in the two laser radars according to the recognition results to obtain track information; A third processing module is used to perform trajectory matching based on the trajectory information to obtain a vehicle trajectory matching result between the two laser radars; a fourth processing module, configured to calibrate the first laser radar and the second laser radar according to a vehicle trajectory matching result between the two laser radars; Wherein, the third processing module includes: A first processing unit is configured to project the first information and the second information into two dimensions to obtain projected point cloud data, wherein the projected point cloud data includes trajectory information corresponding to the first laser radar and trajectory information corresponding to the second laser radar; A second processing unit is configured to rasterize the projected point cloud data to obtain a first raster unit and a second raster unit; a third processing unit, configured to match the trajectory information included in the first grid unit and the second grid unit, and determine mutually matching grids to obtain a matching result; Wherein, the fourth processing module includes: a twelfth processing unit, configured to determine mutually matching grids in the first grid unit and the second grid unit according to the matching result, to obtain overlapping grids; a thirteenth processing unit, configured to search the overlapping grids for two trajectory points corresponding to the same vehicle at the same time, to obtain a first vehicle point cloud and a second vehicle point cloud; The fourteenth processing unit stacks the first vehicle point cloud and the second vehicle point cloud into the overlapping grid, and calibrates the first laser radar and the second laser radar according to the stacked point cloud information.

6. The multi-lidar calibration system based on dynamic target point cloud according to claim 5, characterized in that: The third processing unit includes: A fourth processing unit is configured to determine a trajectory point existing in each grid in the first grid unit according to the trajectory information corresponding to the first laser radar, and obtain first trajectory point information; A fifth processing unit is configured to determine a trajectory point existing in each grid in the second grid unit according to the trajectory information corresponding to the second laser radar, and obtain second trajectory point information; a sixth processing unit, configured to determine a first trajectory spatiotemporal distribution graph corresponding to each grid according to the first trajectory point information, wherein the first trajectory spatiotemporal distribution graph is used to indicate whether a trajectory point exists in the grid at each moment; a seventh processing unit, configured to determine a second trajectory spatiotemporal distribution map corresponding to each grid according to the second trajectory point information; An eighth processing unit is configured to determine whether grids match according to the first trajectory spatiotemporal distribution graph and the second trajectory spatiotemporal distribution graph, and obtain a matching result.

7. The multi-lidar calibration system based on dynamic target point cloud according to claim 6, characterized in that: The sixth processing unit includes: a ninth processing unit, configured to determine all trajectory points existing in each grid according to the first trajectory point information; a tenth processing unit, configured to determine the time information corresponding to each trajectory point; The eleventh processing unit is configured to set the time information corresponding to the existing trajectory point to 1 and the time information corresponding to the non-existing trajectory point to 0, thereby drawing a first trajectory spatiotemporal distribution diagram.

8. The multi-lidar calibration system based on dynamic target point cloud according to claim 5, characterized in that: The fourteenth processing unit includes: a fifteenth processing unit, configured to stack the point clouds of the first vehicle point cloud at different times into overlapping grids to obtain stacked first point cloud information; a sixteenth processing unit, configured to stack the point clouds of the second vehicle point cloud at different times into overlapping grids to obtain stacked second point cloud information; The seventeenth processing unit is used to calibrate the first laser radar and the second laser radar according to the stacked first point cloud information and the stacked second point cloud information.

Images