Method, device and equipment for compensating distortion of laser radar point cloud and storage medium

Abstract

The application relates to a laser radar point cloud distortion compensation method and device, equipment and a storage medium, wherein the method comprises the following steps: based on laser radar data and inertial measurement unit (IMU) measurement data, obtaining a point cloud set of a target point cloud and a point cloud set obtained by removing the target point cloud from an original laser point cloud; based on the point cloud set, converting the laser radar moving target point cloud to a target moment through coordinate conversion, and calculating the time relationship and the speed relationship of each point with the target point cloud to obtain point cloud data after static scene distortion compensation; meanwhile, by calculating the relative speed relationship and the relative time relationship of the laser radar moving target point cloud and the target point cloud, point cloud data after moving target distortion compensation is obtained; and the two types of point cloud data are spliced to obtain compensated moving target point cloud, so that the actual surrounding environment of an automatic driving vehicle can be determined, the position information of the target can be accurately perceived, the collision point of the target can be reflected, and the safety and reliability of the vehicle are improved.

Description

Technical Field

[0001] This application relates to the field of lidar technology, and in particular to a method, apparatus, device and storage medium for distortion compensation of lidar point clouds. Background Technology

[0002] In autonomous driving technology, the perception module plays a crucial role as the front-end module. The accuracy of the perception input determines the effectiveness of the back-end autonomous driving. LiDAR sensors, with their accurate ranging capabilities and ability to scan 3D environment models, hold a dominant position among perception sensors. However, when LiDAR is deployed on autonomous vehicles, the slow-exposure principle of TOF (Time-of-Flight) LiDAR, coupled with the vehicle's own motion and the target's motion, leads to distortion in the 3D environment model it generates. This distortion fails to accurately reflect the surrounding environment of the autonomous vehicle at any given moment, and the target's position also introduces errors, resulting in inaccurate collision point predictions.

[0003] Currently, related technologies can acquire a frame of raw laser point cloud data, select the acquisition time of the starting laser point from the frame as the target time, interpolate the coordinate transformation relationship corresponding to the selected starting and ending laser points to obtain the coordinate transformation relationship corresponding to other laser points, and transform the coordinates of other laser points to the target time. Furthermore, these technologies can sort the lidar 3D point cloud data and IMU (Inertial Measurement Unit) data according to timestamp order, divide each frame of lidar 3D point cloud data into data blocks according to the time sequence of the IMU output lidar 3D point cloud data, and perform three-axis rotation compensation on the lidar 3D point cloud data; finally, estimate the inter-frame motion of the point cloud data based on the rotation-compensated point cloud data frames, and perform translation compensation on the point cloud data.

[0004] However, the relevant technologies all compensate for static scenes in 3D point cloud scenarios and cannot accurately reflect the motion state and actual position information of moving targets at a certain moment. Furthermore, TOF lidar in these technologies cannot measure the velocity information of moving targets, making it difficult to effectively solve the point cloud distortion problem. Summary of the Invention

[0005] This application provides a method, device, equipment, and storage medium for distortion compensation of lidar point clouds, in order to solve the problems that related technologies cannot truly reflect the motion state and actual position information of a moving target at a certain moment, and that TOF lidar cannot measure the velocity information of a moving target.

[0006] The first aspect of this application provides a method for distortion compensation of lidar point clouds, comprising the following steps: acquiring lidar data from a lidar and simultaneously acquiring measurement data from an inertial measurement unit (IMU); acquiring a point cloud set of a target point cloud and a point cloud set of the original lidar point cloud after removing the target point cloud based on the lidar data and the measurement data; based on the point cloud set, selecting the timestamp of the starting laser point of the lidar in the current frame as the target time, unifying the moving target point cloud of the lidar to the target time through coordinate transformation, and calculating the time and velocity relationships with the target point cloud point by point to obtain the transformation relationship, thereby obtaining a static point cloud. The system generates two sets of data: first point cloud data after scene distortion compensation; second point cloud data after scene distortion compensation; and third point cloud data after scene distortion compensation. The system first generates the first point cloud data after scene distortion compensation. Then, it generates the second point cloud data after scene distortion compensation. Based on the point cloud data, it selects the timestamp of the starting laser point of the current frame LiDAR as the target time, transforms the coordinates of the moving target point cloud to the target time, calculates the relative velocity and relative time relationships between the moving target point cloud and the target point cloud, and obtains the coordinate transformation relationship of the moving target point cloud to obtain the second point cloud data after scene distortion compensation. Finally, it stitches the first point cloud data and the second point cloud data together to obtain the compensated moving target point cloud, thus determining the actual surrounding environment of the autonomous vehicle.

[0007] Based on the above technical means, the embodiments of this application synchronize the lidar and IMU in time and space, acquire lidar data and IMU data, perform data preprocessing, and then perform distortion compensation on static scenes and moving targets. The compensated point cloud data is stitched together and the moving target point cloud is output, thereby determining the actual surrounding environment of the autonomous vehicle, accurately perceiving the target's position information and the target's collision point, and improving the vehicle's safety and reliability.

[0008] Optionally, in one embodiment of this application, before obtaining the second point cloud data after distortion compensation of the moving target, the method further includes: performing correlation matching on the detection results of the current frame laser point cloud based on the detection results of the previous frame laser point cloud, estimating the state information of the target, so as to perform distortion compensation on the point cloud set of the target point cloud.

[0009] Based on the above technical means, the embodiments of this application perform correlation matching on the detection results of laser point clouds in consecutive frames to obtain target speed information for distortion compensation of the target point cloud set. Thus, while considering the distortion caused by the vehicle's motion, the distortion caused by the target's motion is also taken into account, accurately reflecting the target's motion state, position information, and collision point, effectively improving the reliability of target detection and making the vehicle more intelligent and technological.

[0010] Optionally, in one embodiment of this application, before acquiring the lidar data and the measurement data, the method further includes: time synchronization of the lidar and the IMU.

[0011] Based on the above technical means, the embodiments of this application effectively ensure data reliability by synchronizing the lidar and IMU in time.

[0012] Optionally, in one embodiment of this application, obtaining the target point cloud set and the original laser point cloud with the target point cloud removed based on the lidar data and the measurement data includes: obtaining the target point cloud information, bounding boxes, and heading information based on the lidar data and the measurement data using a preset clustering algorithm, so as to separate the target point cloud set and the original laser point cloud with the target point cloud removed based on the clustering results.

[0013] Based on the above technical means, the embodiments of this application perform preprocessing operations on lidar data and measurement data to separate the target point cloud set and the original lidar point cloud to remove the target point cloud set, thereby further improving the data quality and ensuring the performance of subsequent distortion compensation.

[0014] Optionally, in one embodiment of this application, the lidar data includes triaxial coordinate information, intensity information, and timestamp information for each point cloud, and the measurement data includes triaxial angular velocity, triaxial axial velocity, and timestamp information.

[0015] Based on the above technical means, the embodiments of this application provide reliable data support for subsequent distortion compensation of lidar point clouds by collecting point cloud data information such as the three-axis coordinates of lidar and the three-axis angular velocity of inertial navigation IMU.

[0016] A second aspect of this application provides a distortion compensation device for lidar point clouds, comprising: a first acquisition module, configured to acquire lidar data from a lidar and measurement data from an inertial measurement unit (IMU); a second acquisition module, configured to acquire a point cloud set of a target point cloud and a point cloud set of the original lidar point cloud minus the target point cloud based on the lidar data and the measurement data; and a third acquisition module, configured to, based on the point cloud set, select the timestamp of the start laser point of the lidar in the current frame as the target time, unify the moving target point cloud of the lidar to the target time through coordinate transformation, and calculate the time and velocity relationships with the target point cloud point by point to obtain the transformation relationship, so as to... The system acquires first point cloud data after static scene distortion compensation; a fourth acquisition module is used to select the timestamp of the start laser point of the current frame LiDAR as the target time based on the point cloud set, unify the LiDAR moving target point cloud to the target time through coordinate transformation, calculate the relative velocity relationship and relative time relationship between the LiDAR moving target point cloud and the target point cloud, obtain the coordinate transformation relationship of the LiDAR moving target point cloud, and obtain second point cloud data after moving target distortion compensation; and a stitching module is used to stitch the first point cloud data and the second point cloud data to obtain the compensated moving target point cloud and determine the actual surrounding environment of the autonomous vehicle.

[0017] Optionally, in one embodiment of this application, it further includes: a matching module, used to perform association matching on the detection results of the current frame laser point cloud based on the detection results of the previous frame laser point cloud before obtaining the second point cloud data after distortion compensation of the moving target, to estimate the state information of the target, so as to perform distortion compensation on the point cloud set of the target point cloud.

[0018] Optionally, in one embodiment of this application, it further includes: a synchronization module, used to synchronize the lidar and the IMU in time before acquiring the lidar data and the measurement data.

[0019] Optionally, in one embodiment of this application, the second acquisition module includes: a separation unit, used to acquire point cloud information, bounding boxes, and heading information of the target based on the lidar data and the measurement data through a preset clustering algorithm, so as to separate the point cloud set of the target point cloud and the point cloud set of the original lidar point cloud after removing the target point cloud based on the clustering results.

[0020] Optionally, in one embodiment of this application, the lidar data includes triaxial coordinate information, intensity information, and timestamp information for each point cloud, and the measurement data includes triaxial angular velocity, triaxial axial velocity, and timestamp information.

[0021] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the distortion compensation method for lidar point clouds as described in the above embodiments.

[0022] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for distortion compensation of lidar point clouds.

[0023] Therefore, the embodiments of this application have the following beneficial effects:

[0024] (1) In this embodiment of the application, the lidar and IMU are synchronized in time and space to obtain lidar data and IMU data, and the data is preprocessed. Then, distortion compensation is performed on static scenes and moving targets, the point cloud data after compensation is stitched together, and the point cloud of moving targets is output. In this way, the actual surrounding environment of the autonomous vehicle can be determined, the position information of the target and the collision point of the target can be accurately perceived, and the safety and reliability of the vehicle can be improved.

[0025] (2) In this embodiment, the detection results of laser point clouds in the preceding and following frames are correlated and matched to obtain target speed information to compensate for the distortion of the target point cloud set. Thus, while considering the distortion of the vehicle's motion, the distortion caused by the target's motion is also taken into account, accurately reflecting the target's motion state, position information, and collision point, etc., effectively improving the reliability of target detection and making the vehicle more intelligent and technological.

[0026] (3) The embodiments of this application effectively ensure the reliability of the data by synchronizing the lidar and IMU in time.

[0027] (4) In this embodiment of the application, the point cloud set of the target and the point cloud set of the original laser point cloud are separated by preprocessing the lidar data and measurement data, thereby further improving the quality of the data and ensuring the performance of subsequent distortion compensation.

[0028] (5) The embodiments of this application collect point cloud data information such as the three-axis coordinates of the lidar and the three-axis angular velocity of the inertial navigation IMU, providing reliable data support for subsequent distortion compensation of lidar point cloud.

[0029] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0030] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0031] Figure 1 This is a flowchart of a method for distortion compensation of lidar point clouds according to an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the execution logic for point cloud compensation in a static scene using a TOF lidar according to an embodiment of this application;

[0033] Figure 3 This is a flowchart illustrating a method for obtaining velocity information of a moving target according to an embodiment of this application;

[0034] Figure 4 This is a schematic diagram of the execution logic for point cloud compensation of a moving target in a TOF lidar according to an embodiment of this application;

[0035] Figure 5 This is a schematic diagram of a TOF lidar point cloud imaging before compensation, according to an embodiment of this application.

[0036] Figure 6 This is a schematic diagram of a TOF lidar image after point cloud compensation for a moving target, provided according to an embodiment of this application.

[0037] Figure 7 This is a schematic diagram illustrating the execution logic of a distortion compensation method for lidar point clouds according to an embodiment of this application;

[0038] Figure 8 This is an example diagram of a distortion compensation device for lidar point clouds according to an embodiment of this application;

[0039] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0040] Among them, 10-distortion compensation device for lidar point cloud, 100-first acquisition module, 200-second acquisition module, 300-third acquisition module, 400-fourth acquisition module, 500-stitching module, 901-memory, 902-processor, and 903-communication interface. Detailed Implementation

[0041] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0042] The following describes a method, apparatus, device, and storage medium for compensating the distortion of LiDAR point clouds according to embodiments of this application, with reference to the accompanying drawings. Addressing the problems mentioned in the background art, this application provides a method for compensating the distortion of LiDAR point clouds. In this method, based on LiDAR data and IMU measurement data, a point cloud set of the target point cloud and a point cloud set of the original LiDAR point cloud minus the target point cloud are obtained. Based on the point cloud set, the moving target point cloud of the LiDAR is unified to the target time through coordinate transformation, and the time and velocity relationships with the target point cloud are calculated point by point to obtain static scene distortion-compensated point cloud data. Simultaneously, the relative velocity and relative time relationships between the moving target point cloud of the LiDAR and the target point cloud are calculated to obtain moving target distortion-compensated point cloud data. The two types of point cloud data are then stitched together to obtain the compensated moving target point cloud, thereby determining the actual surrounding environment of the autonomous vehicle, accurately perceiving the target's position information, reflecting the target's collision point, and improving the vehicle's safety and reliability. This solves the problems of related technologies failing to accurately reflect the motion state and actual position information of a moving target at a certain moment, and the inability of TOF LiDAR to measure the velocity information of a moving target.

[0043] Specifically, Figure 1 This is a flowchart illustrating a method for distortion compensation of lidar point clouds provided in an embodiment of this application.

[0044] like Figure 1 As shown, the distortion compensation method for the lidar point cloud includes the following steps:

[0045] In step S101, while acquiring the lidar data of the lidar, the measurement data of the inertial measurement unit (IMU) is also acquired.

[0046] The embodiments of this application can utilize a LiDAR system mounted on the front of an autonomous vehicle to complete horizontal and vertical scanning within one cycle to obtain a frame of laser point cloud data. This frame of laser point cloud data contains all point cloud data captured during the scanning cycle, thereby enabling the creation of a three-dimensional environment model through scanning. Furthermore, the embodiments of this application can also acquire measurement data from an inertial measurement unit (IMU).

[0047] Therefore, the embodiments of this application can provide the vehicle's speed information through TOF lidar combined with inertial navigation IMU, which effectively ensures the subsequent realization of motion compensation for static point clouds.

[0048] Optionally, in one embodiment of this application, before acquiring lidar data and measurement data, the method further includes: time synchronization of the lidar and IMU.

[0049] It should be noted that, before acquiring lidar data and measurement data, embodiments of this application can also utilize PTP (Precision Time Protocol) to synchronize the lidar and IMU sensors via the host, so that the sensors acquire timestamps under the same clock, thereby synchronizing the lidar and IMU in time and effectively ensuring the reliability of the data.

[0050] Optionally, in one embodiment of this application, the lidar data includes triaxial coordinate information, intensity information, and timestamp information for each point cloud, and the measurement data includes triaxial angular velocity, triaxial axial velocity, and timestamp information.

[0051] After synchronizing the lidar and IMU in time, embodiments of this application can collect data from both. The lidar data includes triaxial coordinate information, intensity information, and timestamp information for each point cloud, while the IMU data includes triaxial angular velocity, triaxial axial velocity, and timestamp information.

[0052] Therefore, the embodiments of this application provide reliable data support for subsequent distortion compensation of lidar point clouds by collecting point cloud data information such as the three-axis coordinates of lidar and data information such as the three-axis angular velocity of inertial navigation IMU.

[0053] In step S102, the point cloud set of the target point cloud and the point cloud set of the original laser point cloud after removing the target point cloud are obtained based on the lidar data and measurement data.

[0054] After acquiring lidar point cloud data and inertial navigation IMU data, embodiments of this application can further acquire a target point cloud set and a point cloud set of the original lidar point cloud after removing the target point cloud, thereby providing reliable technical support for acquiring point cloud data after distortion compensation for static scenes and moving targets.

[0055] Optionally, in one embodiment of this application, obtaining the target point cloud set and the original laser point cloud with the target point cloud removed based on lidar data and measurement data includes: obtaining the target point cloud information, bounding boxes, and heading information through a preset clustering algorithm based on lidar data and measurement data, so as to separate the target point cloud set and the original laser point cloud with the target point cloud removed based on the clustering results.

[0056] Before performing distortion compensation on static scenes and moving targets, embodiments of this application can perform preprocessing operations on the point cloud data obtained above, that is, obtain the point cloud information, bounding box and heading information of the detected target through clustering algorithms, and separate the point cloud set of the target and the point cloud set of the original laser point cloud to remove the target point cloud based on the clustering results.

[0057] Therefore, the embodiments of this application perform preprocessing operations on lidar data and measurement data to separate the target point cloud set and the original lidar point cloud to remove the target point cloud set, thereby further improving the data quality and ensuring the performance of subsequent distortion compensation.

[0058] In step S103, based on the point cloud set, the timestamp of the starting laser point of the current frame lidar is selected as the target time. The moving target point cloud of the lidar is unified to the target time through coordinate transformation, and the time relationship and velocity relationship with the target point cloud are calculated point by point to obtain the transformation relationship, so as to obtain the first point cloud data after static scene distortion compensation.

[0059] After obtaining the point cloud set of the target point cloud and the point cloud set of the original laser point cloud after removing the target point cloud, the embodiments of this application can perform distortion compensation on the point cloud set of the original laser point cloud after removing the target point cloud, that is, compensate for the static scene.

[0060] Specifically, embodiments of this application can obtain target information based on the original laser point cloud data of the current frame through a clustering algorithm, then separate the point cloud set of the original laser point cloud from the target point cloud, and finally transform the original laser point cloud set with the target removed to the starting laser point time of the current frame data.

[0061] It should be noted that the velocity of a static scene point is zero. In this scenario, only distortion compensation caused by the vehicle's motion needs to be considered. The following is the transformation process of a static laser point:

[0062] 1. Calculate the translation:

[0063] Δx = -v x0 *(t k -t0)

[0064] Δy=-v y0 *(t k -t0)

[0065] Δz=-v z0 *(t k -t0)

[0066] 2. Translation transformation:

[0067]

[0068]

[0069]

[0070] 3. By traversing the point cloud set after removing the target point cloud, the result of laser point cloud compensation for the moving target can be obtained.

[0071] The following example further illustrates the process of distortion compensation for the original laser point cloud in a static scene.

[0072] Figure 2 This is a schematic diagram illustrating the point cloud compensation execution logic for a static scene using a TOF lidar system. Figure 2 As shown, the specific process of distortion compensation for the original laser point cloud of a static scene in the embodiments of this application is as follows:

[0073] S21: Raw laser point cloud data for the current frame;

[0074] S22: Clustering algorithm to obtain target information;

[0075] S23: Remove the original laser point cloud from the target;

[0076] S24: Translation compensation of moving target point cloud, the coordinate system is moved to the initial laser point of this frame;

[0077] S25: Output the compensated moving target point cloud.

[0078] Therefore, the embodiments of this application perform distortion compensation on the original laser point cloud point by point, making the distortion compensation result more accurate, which can realistically construct three-dimensional scene information and effectively ensure the distortion compensation performance of the lidar point cloud.

[0079] In step S104, based on the point cloud set, the timestamp of the starting laser point of the current frame lidar is selected as the target time. The lidar moving target point cloud is unified to the target time through coordinate transformation. The relative velocity relationship and relative time relationship between the lidar moving target point cloud and the target point cloud are calculated to obtain the coordinate transformation relationship of the lidar moving target point cloud, so as to obtain the second point cloud data after moving target distortion compensation.

[0080] After performing distortion compensation on the original laser point cloud of a static scene, the embodiments of this application can further perform distortion compensation on moving targets, that is, perform motion compensation on the original laser points of the target point cloud.

[0081] It should be noted that since the target contains motion data information, the embodiments of this application must consider not only the distortion caused by the vehicle's motion, but also the distortion caused by the target's motion, in order to achieve motion target distortion compensation and accurately reflect the target's motion state and position information.

[0082] Optionally, in one embodiment of this application, before obtaining the second point cloud data after distortion compensation for the moving target, the method further includes: performing correlation matching on the detection results of the current frame laser point cloud based on the detection results of the previous frame laser point cloud, estimating the state information of the target, so as to perform distortion compensation on the point cloud set of the target point cloud.

[0083] It should be noted that, before performing moving target distortion compensation, the embodiments of this application also need to obtain the velocity information of the moving target. For example... Figure 3 As shown, embodiments of this application can obtain the current target detection result through clustering, obtain the target's parameter information through tracking gate calculation, and then use the detection result of the previous frame's laser point cloud as the observation value to perform association matching on the detection result of the current frame to estimate the target's state information. The state information includes the target's three-axis velocities, i.e., the velocities of the target's laser point cloud, with the three-axis velocities being v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v21, v12, v11 x v y and v z .

[0084] Subsequently, embodiments of this application can transform the moving target point cloud to the starting laser point time of the current frame data through target point cloud translation transformation. The transformation process of the laser point cloud is as follows:

[0085] 1. Obtain the translation amount:

[0086] Δx=(v x -v x0 )*(t k -t0)

[0087] Δy=(v y -v y0 )*(t k -t0)

[0088] Δz=(v z -v z0 )*(t k -t0)

[0089] Where Δx: index is the displacement of laser point k relative to the initial laser point of this frame in the x-axis direction;

[0090] v x The index represents the velocity of laser point k along the x-axis.

[0091] v x0 : The speed of the vehicle in the x-axis direction;

[0092] t k : The timestamp of the k laser point;

[0093] t0: The timestamp of the initial laser point in this frame;

[0094] Δy: The displacement of laser point k relative to the initial laser point in this frame along the y-axis;

[0095] v y The index represents the velocity of laser point k along the y-axis.

[0096] v y0 : The speed of the vehicle in the y-axis direction;

[0097] Δz: The displacement of laser point k relative to the initial laser point in this frame along the z-axis;

[0098] v z The index represents the velocity of laser point k along the z-axis.

[0099] v z0 : The speed of the vehicle in the z-axis direction.

[0100] 2. Translation transformation:

[0101]

[0102]

[0103]

[0104] in, The index is the value of laser point k after translation along the x-axis relative to the initial laser point of the frame.

[0105] x k : The value of laser point k in the x-direction;

[0106] The index is the value of laser point k after translation and transformation relative to the initial laser point of this frame along the y-axis.

[0107] y k : The value of laser point k in the y direction;

[0108] The index is the value of laser point k after translation and transformation relative to the initial laser point of this frame along the z-axis.

[0109] x k : The value of the laser point k in the z direction.

[0110] 3. By iterating through all the laser points of the target, the compensated result of the moving target laser point cloud can be obtained. Performing the same operation on all targets in this frame of data will yield the compensated result of the moving target laser point cloud in that frame.

[0111] The following example further illustrates the process of distortion compensation for a moving target.

[0112] Figure 4 This is a schematic diagram illustrating the logic for point cloud compensation of moving targets using a TOF lidar system. Figure 4As shown, the specific process of distortion compensation for moving target point clouds in the embodiments of this application is as follows:

[0113] S41: Raw laser point cloud data for the current frame;

[0114] S42: Clustering algorithm to obtain target information;

[0115] S43: Original laser point clusters of separated targets;

[0116] S44: Obtain target velocity information by matching targets in consecutive frames;

[0117] S45: Translation compensation of moving target point cloud, the coordinate system is moved to the initial laser point of this frame;

[0118] S46: Output the compensated moving target point cloud.

[0119] It should be noted that, Figure 5 This is a schematic diagram of the moving target before distortion compensation, such as... Figure 5 As shown, the target is stretched in the mirror direction of the lidar scan, which cannot accurately reflect the target's motion state and position information. Figure 6 The image shows a moving target after distortion compensation. It can be seen that the above problems are effectively solved after the distortion of the moving target is compensated.

[0120] It is understood that the embodiments of this application perform correlation matching on the detection results of laser point clouds in consecutive frames to obtain target speed information in order to compensate for the distortion of the target point cloud set. Thus, while considering the distortion caused by the vehicle's motion, it also considers the distortion caused by the target's motion, accurately reflecting the target's motion state, position information, and collision point, etc., effectively improving the reliability of target detection and making the vehicle more intelligent and technological.

[0121] In step S105, the first point cloud data and the second point cloud data are stitched together to obtain the compensated moving target point cloud, thereby determining the actual surrounding environment of the autonomous vehicle.

[0122] After acquiring the distortion-compensated point cloud data of the static scene and the moving target, the embodiments of this application can stitch together the original laser point cloud after distortion compensation of the static scene and the original laser point cloud after distortion compensation of the moving target to obtain a complete frame of distortion-free original laser point cloud data. This enables the TOF lidar to compensate for the laser point cloud, determine the actual surrounding environment of the autonomous vehicle, effectively solve the problem of laser point cloud distortion of the TOF lidar, and make the distortion-compensated laser point cloud reflect the real environmental information and the motion state and actual position information of the moving target at a certain moment, which greatly improves the safety performance of the vehicle.

[0123] The following detailed explanation of the distortion compensation process for a moving target through a specific embodiment further illustrates this process.

[0124] Figure 7 This is a schematic diagram illustrating the execution logic of distortion compensation for lidar point clouds in an embodiment of this application. Figure 7 As shown, the specific process of distortion compensation for lidar point clouds in the embodiments of this application is as follows:

[0125] S71: Time and space synchronization between lidar and IMU;

[0126] S72: Acquires lidar data and IMU data;

[0127] S73: Data preprocessing;

[0128] S74: Static scene distortion compensation and moving target distortion compensation;

[0129] S75: Point cloud stitching;

[0130] S76: Outputs the compensated moving target point cloud.

[0131] According to the distortion compensation method for LiDAR point clouds proposed in this application, a point cloud set of the target point cloud and a point cloud set of the original LiDAR point cloud after removing the target point cloud are obtained based on LiDAR data and measurement data from the inertial measurement unit (IMU). Based on the point cloud set, the moving target point cloud of the LiDAR is unified to the target time through coordinate transformation, and the time and velocity relationships with the target point cloud are calculated point by point to obtain the static scene distortion-compensated point cloud data. At the same time, the relative velocity relationship and relative time relationship between the moving target point cloud of the LiDAR and the target point cloud are calculated to obtain the moving target distortion-compensated point cloud data. The above two types of point cloud data are stitched together to obtain the compensated moving target point cloud, thereby determining the actual surrounding environment of the autonomous vehicle, accurately perceiving the target's position information, reflecting the target's collision point, and improving the vehicle's safety and reliability.

[0132] Next, with reference to the accompanying drawings, a distortion compensation device for lidar point clouds according to an embodiment of this application is described.

[0133] Figure 8 This is a block diagram of a LiDAR point cloud distortion compensation device according to an embodiment of this application.

[0134] like Figure 8 As shown, the distortion compensation device 10 for the lidar point cloud includes: a first acquisition module 100, a second acquisition module 200, a third acquisition module 300, a fourth acquisition module 400, and a stitching module 500.

[0135] The first acquisition module 100 is used to acquire lidar data from the lidar and measurement data from the inertial measurement unit (IMU) at the same time.

[0136] The second acquisition module 200 is used to acquire the point cloud set of the target point cloud and the point cloud set of the original laser point cloud after removing the target point cloud based on the lidar data and measurement data.

[0137] The third acquisition module 300 is used to select the timestamp of the starting laser point of the current frame lidar as the target time based on the point cloud set, unify the moving target point cloud of the lidar to the target time through coordinate transformation, and calculate the time relationship and velocity relationship with the target point cloud point by point to obtain the transformation relationship, so as to obtain the first point cloud data after static scene distortion compensation.

[0138] The fourth acquisition module 400 is used to select the timestamp of the starting laser point of the current frame lidar as the target time based on the point cloud set, unify the lidar moving target point cloud to the target time through coordinate transformation, calculate the relative velocity relationship and relative time relationship between the lidar moving target point cloud and the target point cloud, obtain the coordinate transformation relationship of the lidar moving target point cloud, and obtain the second point cloud data after moving target distortion compensation.

[0139] The stitching module 500 is used to stitch together the first point cloud data and the second point cloud data to obtain the compensated moving target point cloud and determine the actual surrounding environment of the autonomous vehicle.

[0140] Optionally, in one embodiment of this application, the laser radar point cloud distortion compensation device 10 of this application embodiment further includes: a matching module, used to perform association matching on the detection result of the current frame laser point cloud based on the detection result of the previous frame laser point cloud before obtaining the second point cloud data after distortion compensation of the moving target, to estimate the state information of the target, so as to perform distortion compensation on the point cloud set of the target point cloud.

[0141] Optionally, in one embodiment of this application, the distortion compensation device 10 for lidar point clouds in this application embodiment further includes: a synchronization module, used to synchronize the lidar and IMU in time before acquiring lidar data and measurement data.

[0142] Optionally, in one embodiment of this application, the second acquisition module 200 includes: a separation unit, used to acquire point cloud information, bounding boxes and heading information of the target based on lidar data and measurement data through a preset clustering algorithm, so as to separate the point cloud set of the target point cloud and the point cloud set of the original lidar point cloud after removing the target point cloud based on the clustering results.

[0143] Optionally, in one embodiment of this application, the lidar data includes triaxial coordinate information, intensity information, and timestamp information for each point cloud, and the measurement data includes triaxial angular velocity, triaxial axial velocity, and timestamp information.

[0144] It should be noted that the explanation of the aforementioned embodiment of the distortion compensation method for lidar point clouds also applies to the distortion compensation device for lidar point clouds in this embodiment, and will not be repeated here.

[0145] The distortion compensation device for lidar point clouds proposed in this application obtains a point cloud set of the target point cloud and a point cloud set of the original lidar point cloud after removing the target point cloud from the lidar point cloud based on lidar data and inertial measurement unit (IMU) measurement data. Based on the point cloud set, the lidar moving target point cloud is unified to the target time through coordinate transformation, and the time and velocity relationships with the target point cloud are calculated point by point to obtain static scene distortion-compensated point cloud data. At the same time, the relative velocity and relative time relationships between the lidar moving target point cloud and the target point cloud are calculated to obtain moving target distortion-compensated point cloud data. The above two types of point cloud data are stitched together to obtain the compensated moving target point cloud, thereby determining the actual surrounding environment of the autonomous vehicle, accurately perceiving the target's position information, reflecting the target's collision point, and improving the vehicle's safety and reliability.

[0146] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0147] The memory 901, the processor 902, and the computer program stored on the memory 901 and capable of running on the processor 902.

[0148] When the processor 902 executes the program, it implements the distortion compensation method for lidar point clouds provided in the above embodiments.

[0149] Furthermore, electronic devices also include:

[0150] Communication interface 903 is used for communication between memory 901 and processor 902.

[0151] The memory 901 is used to store computer programs that can run on the processor 902.

[0152] The memory 901 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0153] If the memory 901, processor 902, and communication interface 903 are implemented independently, then the communication interface 903, memory 901, and processor 902 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0154] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0155] The processor 902 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0156] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for distortion compensation of lidar point clouds.

[0157] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0158] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0159] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0160] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0161] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0162] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0164] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for distortion compensation of lidar point clouds, characterized in that, Includes the following steps: While acquiring lidar data from the lidar, measurement data from the inertial measurement unit (IMU) is also acquired. Based on the lidar data and the measurement data, obtain the point cloud set of the moving target and the original lidar point cloud with the point cloud set of the moving target removed; Based on the original laser point cloud, the point cloud set of the moving target is removed. The timestamp of the start laser point of the current frame of the lidar is selected as the target time. The point cloud of the moving target removed from the original laser point cloud of the lidar is unified to the target time through coordinate transformation. The time relationship and velocity relationship with the original laser point cloud of the moving target are calculated point by point to obtain the transformation relationship, so as to obtain the first point cloud data after static scene distortion compensation. Based on the point cloud set of the moving target, the timestamp of the starting laser point of the current frame lidar is selected as the target time. The point cloud of the moving target of the lidar is unified to the target time through coordinate transformation. The relative velocity relationship and relative time relationship between the point cloud set of the moving target and the target time are calculated point by point to obtain the coordinate transformation relationship of the point cloud of the moving target of the lidar, so as to obtain the second point cloud data after distortion compensation of the moving target. as well as By stitching together the first point cloud data and the second point cloud data, a compensated moving target point cloud is obtained, and the actual surrounding environment of the autonomous vehicle is determined.

2. The method according to claim 1, characterized in that, Before obtaining the second point cloud data after distortion compensation for the moving target, the process also includes: Based on the detection results of the laser point cloud in the previous frame, the detection results of the laser point cloud in the current frame are correlated and matched to estimate the state information of the target, so as to perform distortion compensation on the point cloud set of the moving target.

3. The method according to claim 1, characterized in that, Before acquiring the lidar data and the measurement data, the process also includes: The lidar and the IMU are synchronized in time.

4. The method according to claim 1, characterized in that, The step of obtaining a point cloud set of the moving target and a point cloud set of the original laser point cloud after removing the moving target based on the lidar data and the measurement data includes: Based on the lidar data and the measurement data, the point cloud information, bounding box and heading information of the moving target are obtained through a preset clustering algorithm, so as to separate the point cloud set of the moving target and the point cloud set of the original lidar point cloud after removing the moving target based on the clustering results.

5. The method according to any one of claims 1-4, characterized in that, The lidar data includes the three-axis coordinate information, intensity information, and timestamp information of each point cloud, and the measurement data includes the three-axis angular velocity, the three-axis linear velocity, and timestamp information.

6. A distortion compensation device for lidar point clouds, characterized in that, include: The first acquisition module is used to acquire lidar data from the lidar and measurement data from the inertial measurement unit (IMU) at the same time. The second acquisition module is used to acquire a point cloud set of a moving target and a point cloud set of the original laser point cloud with the moving target removed, based on the lidar data and the measurement data. The third acquisition module is used to remove the point cloud set of the moving target from the original laser point cloud, select the timestamp of the start laser point of the current frame of the lidar as the target time, unify the point cloud of the moving target from the original laser point cloud of the lidar to the target time through coordinate transformation, and calculate the time relationship and velocity relationship with the point cloud of the moving target from the original laser point cloud point cloud point by point to obtain the transformation relationship, so as to obtain the first point cloud data after static scene distortion compensation. The fourth acquisition module is used to select the timestamp of the start laser point of the current frame lidar as the target time based on the point cloud set of the moving target, unify the point cloud of the moving target of the lidar to the target time through coordinate transformation, calculate the relative velocity relationship and relative time relationship between the point cloud of the moving target and the target time point by point, and obtain the coordinate transformation relationship of the point cloud of the moving target of the lidar to obtain the second point cloud data after distortion compensation of the moving target; as well as The stitching module is used to stitch together the first point cloud data and the second point cloud data to obtain a compensated moving target point cloud and determine the actual surrounding environment of the autonomous vehicle.

7. The apparatus according to claim 6, characterized in that, Also includes: The matching module is used to perform correlation matching on the detection results of the current frame laser point cloud based on the detection results of the previous frame laser point cloud before obtaining the second point cloud data after distortion compensation of the moving target, and to estimate the state information of the target in order to perform distortion compensation on the point cloud set of the moving target.

8. The apparatus according to claim 6, characterized in that, Also includes: A synchronization module is used to synchronize the lidar and the IMU in time before acquiring the lidar data and the measurement data.

9. The apparatus according to claim 6, characterized in that, The second acquisition module includes: The separation unit is used to obtain the point cloud information, bounding box and heading information of the moving target based on the lidar data and the measurement data through a preset clustering algorithm, so as to separate the point cloud set of the moving target and the point cloud set of the original lidar point cloud after removing the moving target based on the clustering results.

10. The apparatus according to any one of claims 6-9, characterized in that, The lidar data includes the three-axis coordinate information, intensity information, and timestamp information of each point cloud, and the measurement data includes the three-axis angular velocity, the three-axis linear velocity, and timestamp information.

11. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the distortion compensation method for lidar point clouds as described in any one of claims 1-5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the distortion compensation method for lidar point clouds as described in any one of claims 1-5.

Images