Point cloud data processing method and device, electronic equipment and storage medium

Abstract

The application provides a point cloud data processing method and device, electronic equipment and storage medium, comprising: acquiring point cloud data blocks in a current acquisition frame of a laser radar and measurement data of an inertial measurement unit, the point cloud data blocks being obtained by pre-dividing laser points collected by the laser radar, adjusting each point cloud data block to a frame head position of the current acquisition frame, generating a rotation transformation amount and a translation transformation amount of each point cloud data block using the measurement data, determining a point cloud data block corresponding to a laser point, performing the same motion compensation on the laser point using the rotation transformation amount and the translation transformation amount of the point cloud data block, and obtaining target point cloud data after compensation. The application divides one frame of laser radar data into multiple data blocks, calculates the rotation transformation amount and the translation transformation amount of the point cloud data blocks, and directly performs the same rotation transformation and translation transformation on the laser points on each point cloud data block, thereby realizing efficient motion distortion compensation of the laser radar point cloud.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of autonomous driving technology, and in particular to a point cloud data processing method, apparatus, electronic device and storage medium. Background Technology

[0002] LiDAR is a high-precision active sensor that is increasingly being used in advanced autonomous driving solutions. When LiDAR is installed on autonomous vehicles, the scanning and imaging principle of the LiDAR sensor is a slow-exposure process. Since the vehicle itself is moving relative to its surroundings, the coordinate system of the LiDAR sensor changes with the movement of the vehicle during the scanning of the same frame of data. This results in motion distortion in the point cloud data, which cannot accurately reflect the three-dimensional environment around the autonomous vehicle at any given moment.

[0003] To obtain accurate point cloud data, motion distortion compensation is necessary. Currently, common TOF (Time-of-Flight) lidar point cloud distortion compensation methods estimate the lidar's motion during scanning and transform each point cloud data accordingly to eliminate motion-induced distortion. This mainly involves point-by-point interpolation coordinate transformation of the lidar point cloud based on the pose state at the beginning and end of each frame. However, due to the large amount of lidar point cloud data, point-by-point linear interpolation suffers from high computational cost and low efficiency, further affecting the motion distortion compensation effect of lidar point cloud data. Summary of the Invention

[0004] One objective of this invention is to provide a point cloud data processing method to solve the problems of high computational load and low efficiency in processing large amounts of point cloud data using point-by-point linear interpolation, thereby achieving efficient point cloud data processing; a second objective is to provide a point cloud data processing device; a third objective is to provide an electronic device; and a fourth objective is to provide a storage medium.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The first aspect of this invention provides a point cloud data processing method, the method comprising:

[0007] The point cloud data block in the current acquisition frame of the lidar and the measurement data of the inertial measurement unit are acquired; wherein, the point cloud data block is obtained by pre-dividing the laser points acquired by the lidar;

[0008] Adjust each point cloud data block to the beginning of the current acquisition frame, and use the measurement data to generate the rotation transformation and translation transformation of each point cloud data block;

[0009] The point cloud data block corresponding to the laser point is determined, and the same motion compensation is applied to the laser point using the rotation transformation and translation transformation of the point cloud data block to obtain the compensated target point cloud data.

[0010] Optionally, the step involves acquiring point cloud data blocks from the current acquisition frame of the lidar and measurement data from the inertial measurement unit; wherein the point cloud data blocks are pre-divided from the laser points acquired by the lidar, including:

[0011] The lidar and inertial measurement unit are synchronized in advance.

[0012] The laser points collected by the lidar in the current acquisition frame are obtained, and the laser points are divided into data blocks using a preset number of packets to obtain point cloud data blocks for the current acquisition frame; wherein, the number of point cloud data blocks is the same as the preset number of packets.

[0013] Acquire point cloud data blocks from the lidar after time synchronization, as well as measurement data from the inertial measurement unit.

[0014] Optionally, adjusting each point cloud data block to the beginning of the current acquisition frame and generating rotation and translation transformations for each point cloud data block using the measurement data includes:

[0015] Determine the target measurement data at the beginning of the current acquisition frame, and use the target measurement data to adjust the point cloud data block to the beginning of the frame;

[0016] Obtain the ordered timestamps of the point cloud data blocks, use the ordered timestamps to determine the measurement data at the first and second time points, and obtain the rotation quaternion change and translation change from the first to the second time point; wherein, the ordered timestamps are between the first and second time points;

[0017] The rotation and translation transformation amounts of the point cloud data block are generated by using the rotation quaternion changes and translation changes from the first time step to the second time step.

[0018] Optionally, determining the target measurement data at the beginning of the current acquisition frame, and using the target measurement data to adjust the point cloud data block to the beginning of the frame, includes:

[0019] Acquire the first measurement data before the beginning of the current acquisition frame and the second measurement data after the end of the current acquisition frame;

[0020] Using the first and second measurement data, the target measurement data at the frame start time is determined; wherein, the target measurement data includes rotation quaternions and three-axis coordinates;

[0021] Using the rotation quaternion and the three-axis coordinates, the point cloud data block is adjusted to the frame start time.

[0022] Optionally, generating the rotation and translation transformation amounts of the point cloud data block using the rotation quaternion changes and translation changes from the first time step to the second time step includes:

[0023] Interpolation parameters for the ordered timestamps of the point cloud data blocks from the first moment to the second moment are obtained by using the rotation quaternion change and translation change from the first moment to the second moment.

[0024] The rotation and translation transformations of the point cloud data blocks are generated using rotation quaternion changes, translation changes, and interpolation parameters.

[0025] Optionally, determining the point cloud data block corresponding to the laser point and performing motion compensation on the laser point using the rotation and translation transformations of the point cloud data block to obtain compensated target point cloud data includes:

[0026] Determine the sequence number of the laser points and the number of laser points in the point cloud data block;

[0027] The point cloud data block to which the laser point belongs is determined based on the laser point's serial number and the number of laser points in the point cloud data block.

[0028] The laser point is rotated using the rotation transformation amount of the point cloud data block, and then translated using the translation transformation amount of the point cloud data block to obtain the compensated target point cloud data.

[0029] A second aspect of the present invention provides a point cloud data processing apparatus, the apparatus comprising:

[0030] The data acquisition module is used to acquire point cloud data blocks in the current acquisition frame of the lidar and measurement data from the inertial measurement unit; wherein, the point cloud data blocks are obtained by pre-dividing the laser points acquired by the lidar;

[0031] The compensation module is used to adjust each point cloud data block to the beginning of the current acquisition frame and generate the rotation transformation and translation transformation of each point cloud data block using the measurement data.

[0032] The data compensation module is used to determine the point cloud data block corresponding to the laser point, and to perform the same motion compensation on the laser point using the rotation transformation and translation transformation of the point cloud data block to obtain the compensated target point cloud data.

[0033] Optionally, the data acquisition module includes:

[0034] The synchronization submodule is used to pre-synchronize the lidar and inertial measurement unit in time;

[0035] The partitioning submodule is used to acquire the laser points collected by the lidar in the current acquisition frame, and to divide the laser points into data blocks using a preset number of message packets to obtain the point cloud data blocks of the current acquisition frame; wherein, the number of point cloud data blocks is the same as the preset number of message packets;

[0036] The acquisition submodule is used to acquire point cloud data blocks from the LiDAR after time synchronization, as well as measurement data from the inertial measurement unit.

[0037] Optionally, the compensation generation module includes:

[0038] The adjustment submodule is used to determine the target measurement data at the beginning of the current acquisition frame, and to adjust the point cloud data block to the beginning of the frame using the target measurement data;

[0039] The first determining submodule is used to obtain the ordered timestamps of the point cloud data blocks, use the ordered timestamps to determine the measurement data at the first and second moments, and obtain the rotation quaternion change and translation change from the first to the second moment; wherein, the ordered timestamps are between the first and second moments;

[0040] The generation submodule is used to generate the rotation transformation amount and translation transformation amount of the point cloud data block by using the rotation quaternion change amount and translation change amount from the first time moment to the second time moment.

[0041] Optionally, the adjustment submodule includes:

[0042] The acquisition unit is used to acquire the first measurement data before the beginning of the current acquisition frame and the second measurement data after the end of the current acquisition frame.

[0043] A determining unit is used to determine the target measurement data at the beginning of the frame using first measurement data and second measurement data; wherein the target measurement data includes rotational quaternions and three-axis coordinates;

[0044] An adjustment unit is used to adjust the point cloud data block to the frame start time using the rotation quaternion and the three-axis coordinates.

[0045] Optionally, the generation submodule includes:

[0046] The calculation unit is used to obtain the interpolation parameters of the ordered timestamps of the point cloud data blocks from the first time step to the second time step by using the rotation quaternion change and translation change.

[0047] The generation unit is used to generate the rotation and translation transformations of the point cloud data block using rotation quaternion changes, translation changes, and interpolation parameters.

[0048] Optionally, the data compensation module includes:

[0049] The second determination submodule is used to determine the sequence number of the laser point and the number of laser points in the point cloud data block;

[0050] The third determination submodule is used to determine the point cloud data block to which the laser point belongs based on the laser point's serial number and the number of laser points in the point cloud data block.

[0051] The transformation submodule is used to perform rotation transformation on the laser point using the rotation transformation amount of the point cloud data block, and to perform translation transformation on the rotated laser point using the translation transformation amount of the point cloud data block, so as to obtain the compensated target point cloud data.

[0052] A third aspect of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the point cloud data processing method as described in the first aspect of the present invention.

[0053] A fourth aspect of the present invention provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the point cloud data processing method described in the first aspect of the present invention.

[0054] The point cloud data processing method provided in this embodiment of the invention acquires point cloud data blocks in the current acquisition frame of the lidar and measurement data from the inertial measurement unit. The point cloud data blocks are obtained by pre-dividing the laser points acquired by the lidar. Each point cloud data block is adjusted to the beginning of the current acquisition frame. The rotation transformation and translation transformation of each point cloud data block are generated using the measurement data. The point cloud data block corresponding to the laser point is determined. The rotation transformation and translation transformation of the point cloud data blocks are used to perform the same motion compensation on the laser point to obtain the compensated target point cloud data. This invention divides a frame of LiDAR data into multiple data blocks, calculates the rotation and translation transformations of the point cloud data blocks, and directly applies the same rotation and translation transformations to the laser points in each data block. This achieves motion distortion compensation for the LiDAR point cloud, eliminating the need for linear interpolation of every point in the current frame. Instead, the current frame's point cloud data is divided into data blocks, and the laser points in each block use the same rotation and translation transformations, significantly reducing the amount of interpolation calculations. By transforming the point cloud data to the beginning of the current frame, the changes in pose and three-axis displacement caused by the vehicle's motion are removed, thereby restoring the true position of the laser point cloud, improving the efficiency of motion distortion compensation for LiDAR point cloud data, achieving accurate reconstruction of the three-dimensional environment of the point cloud, and further enhancing the safety of autonomous driving. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of the steps of a point cloud data processing method provided in an embodiment of the present invention;

[0057] Figure 2 yes Figure 1 A flowchart of step 101 of a point cloud data processing method provided in this embodiment of the invention;

[0058] Figure 3 yes Figure 1 A flowchart of step 102 of a point cloud data processing method provided in this embodiment of the invention;

[0059] Figure 4 yes Figure 1 A flowchart of step 103 of a point cloud data processing method provided in this embodiment of the invention;

[0060] Figure 5This is the flow chart of the point cloud data processing method provided in the embodiments of the present invention. Figure 1 ;

[0061] Figure 6 This is the flow chart of the point cloud data processing method provided in the embodiments of the present invention. Figure 2 ;

[0062] Figure 7 This is a schematic diagram of a point cloud data processing method provided in an embodiment of the present invention;

[0063] Figure 8 This is a data schematic diagram of a point cloud data processing method provided in an embodiment of the present invention;

[0064] Figure 9 This is a structural block diagram of a point cloud data processing device provided in an embodiment of the present invention;

[0065] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0066] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0067] Reference Figure 1 The diagram illustrates a flowchart of the point cloud data processing method provided in an embodiment of the present invention. The method may include:

[0068] Step 101: Obtain the point cloud data block in the current acquisition frame of the lidar and the measurement data of the inertial measurement unit; wherein, the point cloud data block is obtained by pre-dividing the laser points acquired by the lidar.

[0069] In this embodiment of the invention, to address the problem that the large volume of LiDAR point cloud data to be processed in motion distortion compensation, coupled with the high computational cost and low efficiency of point-by-point linear interpolation, which affects the motion distortion compensation effect of LiDAR point cloud data, this embodiment divides the LiDAR data into data blocks. Each point cloud data block uses the same rotation and translation transformation, thus achieving motion distortion compensation for the LiDAR point cloud. Instead of interpolating the quaternion and triaxial position of each point in the current frame of point cloud data, this embodiment uses the number of UDP packets received in one frame to divide the current frame of point cloud data into data blocks. The point clouds in each point cloud data block use the same rotation and translation transformation, thereby greatly reducing the amount of interpolation computation and ensuring the accuracy of compensation.

[0070] It should be noted that, referring to Figure 7 The illustration shows a scenario diagram of the point cloud data processing method provided by an embodiment of the present invention. The embodiment of the present invention can use a LiDAR set directly above the front of an autonomous vehicle to complete the scanning of the horizontal and vertical viewpoints within one cycle to obtain a frame of LiDAR point cloud data. This frame of LiDAR point cloud data contains all the point clouds scanned and imaged. An Inertial Measurement Unit (IMU) is a sensor device used to measure and report the velocity, direction, and gravitational effects of an object, and is used to determine the position information of the vehicle.

[0071] In this embodiment of the invention, in order to provide reliable data support for distortion compensation of subsequent lidar point clouds, after time synchronization between the lidar and the inertial measurement unit (IMU), the processor acquires the point cloud data blocks in the current acquisition frame of the lidar and the measurement data of the inertial measurement unit. The point cloud data blocks are obtained by pre-dividing the laser points acquired by the lidar. The lidar data includes the triaxial coordinate information, intensity information, sequence number of the laser point in the UDP packet of the current frame point cloud, and ordered timestamp for each point cloud in the current acquisition frame. Typically, one frame of data contains several hundred UDP packets. For example, the data acquired by the AT128 lidar in one frame contains 625 UDP packets. The laser points acquired in the current acquisition frame of the lidar are divided according to the number of UDP packets to obtain 625 point cloud data blocks. The measurement data of the inertial measurement unit includes the vehicle's rotation quaternion, triaxial coordinates, and timestamp information, which will not be described in detail here.

[0072] Step 102: Adjust each point cloud data block to the beginning of the current acquisition frame, and use the measurement data to generate the rotation transformation amount and translation transformation amount of each point cloud data block.

[0073] In this embodiment of the invention, since the lidar data is first divided into data blocks, and each data block's point cloud uses the same rotation and translation transformations for motion distortion compensation, it is necessary to first adjust each point cloud data block to the beginning of the current acquisition frame. Then, the rotation and translation transformation amounts of each point cloud data block are generated using measurement data. The measurement data includes the rotation quaternion change and translation change between two moments. The ordered timestamp of the point cloud data block is located between these two moments and includes the timestamps of each laser point. Based on the rotation quaternion change and translation change between two moments, the interpolation parameters from the previous moment to the ordered timestamp can be calculated. Therefore, the rotation and translation transformation amounts of the point cloud data block are generated using the rotation quaternion change, translation change, and interpolation parameters. The rotation transformation amount is the rotation quaternion of the ordered timestamp of the point cloud data block relative to the IMU measurement data, and the translation transformation amount is the three-axis coordinate of the ordered timestamp of the point cloud data block relative to the IMU measurement data.

[0074] Specifically, the rotation and translation transformation amounts for each point cloud data block are calculated. Since the number of point cloud data blocks (n) is large, and the time interval between the beginning and end of each block is on the order of microseconds, motion distortion compensation can be achieved using the same rotation and translation transformation amounts for each point cloud data block. In this embodiment, the calculation of the rotation and translation transformation amounts for the point cloud data blocks is performed by finding the ordered timestamp t of the point cloud data blocks. i And the ordered timestamp is located in the IMU data t s and t j Between, first calculate t j time relative to t s Calculate t using the rotational quaternion change and translational change. i The rate of change of the interpolation at time t is calculated based on the changes in the rotation quaternion and the changes in the translation. i time relative to t s Interpolation parameters are used to calculate the ordered timestamp t of the point cloud data block based on the changes in rotation quaternions and translation, as well as the interpolation parameters. i time relative to t s rotational transformation quantity q i Translational transformation quantity v i q i The corresponding rotation quaternions w, x, y, z are respectively v i The corresponding three-axis coordinates x, y, z are respectively

[0075] Step 103: Determine the point cloud data block corresponding to the laser point, and use the rotation transformation and translation transformation of the point cloud data block to perform the same motion compensation on the laser point to obtain the compensated target point cloud data.

[0076] In this embodiment of the invention, the point cloud data block is obtained by pre-dividing the laser points collected by the lidar. After determining the rotation and translation transformation amounts of the point cloud data block, the laser points on the point cloud data block are subjected to the same motion compensation according to the rotation and translation transformation amounts of the point cloud data block to obtain the compensated target point cloud data. Specifically, performing the same motion compensation on the laser points using the rotation and translation transformation amounts of the point cloud data block includes rotating the laser points using the rotation transformation amounts of the point cloud data block and translating the rotated laser points using the translation transformation amounts of the point cloud data block, thereby compensating the lidar point cloud data to obtain a complete frame of distortion-free lidar point cloud data.

[0077] The point cloud data processing method provided in this embodiment of the invention acquires point cloud data blocks in the current acquisition frame of the lidar and measurement data from the inertial measurement unit. The point cloud data blocks are obtained by pre-dividing the laser points acquired by the lidar. Each point cloud data block is adjusted to the beginning of the current acquisition frame. The rotation transformation and translation transformation of each point cloud data block are generated using the measurement data. The point cloud data block corresponding to the laser point is determined. The rotation transformation and translation transformation of the point cloud data blocks are used to perform the same motion compensation on the laser point to obtain the compensated target point cloud data. This invention divides a frame of LiDAR data into multiple data blocks, calculates the rotation and translation transformations of the point cloud data blocks, and directly applies the same rotation and translation transformations to the laser points in each data block. This achieves motion distortion compensation for the LiDAR point cloud, eliminating the need for linear interpolation of every point in the current frame. Instead, the current frame's point cloud data is divided into data blocks, and the laser points in each block use the same rotation and translation transformations, significantly reducing the amount of interpolation calculations. By transforming the point cloud data to the beginning of the current frame, the changes in pose and three-axis displacement caused by the vehicle's motion are removed, thereby restoring the true position of the laser point cloud, improving the efficiency of motion distortion compensation for LiDAR point cloud data, achieving accurate reconstruction of the three-dimensional environment of the point cloud, and further enhancing the safety of autonomous driving.

[0078] Furthermore, refer to Figure 2 , Figure 2 yes Figure 1 The flowchart of step 101 of the point cloud data processing method provided in this embodiment of the invention is basically the same as the server testing method provided in the first embodiment of the invention. Step 101 may specifically include the following steps:

[0079] Step 201: Time synchronization is performed on the lidar and inertial measurement unit in advance.

[0080] It should be noted that, in this embodiment, in order to improve the accuracy of point cloud data processing, it is necessary to ensure that the acquired LiDAR and IMU data are synchronized in time. Therefore, the LiDAR and IMU are synchronized in time in advance. Specifically, time synchronization can be achieved through the following methods: hardware synchronization, using hardware synchronization signals to synchronize the clocks of the LiDAR and IMU; software synchronization, by recording the initial timestamp of the LiDAR's IMU and calculating the time offset for synchronization. This embodiment does not make specific limitations on this.

[0081] Step 202: Obtain the laser points collected by the lidar in the current acquisition frame, and divide the laser points into data blocks using a preset number of message packets to obtain the point cloud data blocks of the current acquisition frame; wherein, the number of point cloud data blocks is the same as the preset number of message packets.

[0082] In this embodiment of the invention, the processor acquires the laser points collected by the lidar in the current acquisition frame, divides the laser points into data blocks using a preset number of packets, and obtains the point cloud data blocks of the current acquisition frame. It should be noted that when using the inertial measurement unit and point cloud data for data processing, the received data is divided and processed. In this embodiment, the UDP protocol is used to receive data, and each frame of data contains multiple UDP packets. The point cloud data of the current frame is divided into data blocks according to the number of received UDP packets. The number of received UDP packets in the current frame is counted, and the point cloud data is divided into multiple point cloud data blocks according to the number of received packets. The number of point cloud data blocks is the same as the preset number of packets.

[0083] Step 203: Obtain the point cloud data block of the lidar after time synchronization, as well as the measurement data of the inertial measurement unit.

[0084] This invention divides the laser points of the current acquisition frame into multiple point cloud data blocks. This allows for motion compensation of the same pose for the points in the same point cloud data block. Instead of performing linear interpolation on every point in the current frame's point cloud data, the current frame's point cloud data is divided into data blocks, and the laser points in each data block use the same rotation and translation transformation. This greatly reduces the amount of interpolation calculation and improves the efficiency of motion distortion compensation for lidar point cloud data.

[0085] Furthermore, refer to Figure 3 , Figure 3 yes Figure 1 The flowchart of step 102 of the point cloud data processing method provided in this embodiment of the invention is basically the same as the server testing method provided in the first embodiment of the invention. Step 102 may specifically include the following steps:

[0086] Step 301: Determine the target measurement data at the beginning of the current acquisition frame, and use the target measurement data to adjust the point cloud data block to the beginning of the frame.

[0087] In this embodiment, the target measurement data at the beginning of the current acquisition frame is determined, and the point cloud data block is adjusted to the beginning of the frame using the target measurement data. Specifically, the target measurement data at the beginning of the frame is determined using the measurement data before the beginning of the current acquisition frame and the measurement data after the end of the current acquisition frame. The target measurement data includes rotation quaternions and three-axis coordinates. The point cloud data block is adjusted to the beginning of the frame using the rotation quaternions and three-axis coordinates at the beginning of the frame.

[0088] Specifically, identify the IMU data within the ordered timestamp range of the point cloud data block in this frame, using the IMUs before the beginning of the frame (t0) corresponding to the pose q0 and position v0, and the IMUs after the end of the frame (t). e position q e and position v e Interpolation is used to calculate the first frame time t. s position q s and position v s Using the frame start time t s position q s and position v s Compensate all point cloud data blocks to the beginning of the frame. s time.

[0089] Step 302: Obtain the ordered timestamp of the point cloud data block, use the ordered timestamp to determine the measurement data at the first and second times, and obtain the rotation quaternion change and translation change from the first to the second time; wherein the ordered timestamp is between the first and second times.

[0090] In this embodiment, the ordered timestamps of the point cloud data blocks are obtained, and the ordered timestamps are used to determine the measurement data at the first and second moments, that is, to find the ordered timestamp t of the UDP. i Located in IMU data t s and t j Between the first and second time points, the ordered timestamps are interpolated to calculate the rotation quaternion change and translation change from the first to the second time point.

[0091] Step 303: Using the rotation quaternion change and translation change from the first time to the second time, generate the rotation transformation and translation transformation of the point cloud data block.

[0092] In this embodiment, the rotation quaternion change and translation change from the first time point to the second time point are used to obtain the interpolation parameters of the ordered timestamp of the point cloud data block from the first time point. The rotation transformation and translation transformation of the point cloud data block are generated by using the rotation quaternion change, translation change and interpolation parameters.

[0093] This invention calculates the rotation and translation transformation amounts of point cloud data blocks. The laser points on each point cloud data block directly use the transformation amounts of the data blocks to perform the same rotation and translation transformations, thereby achieving motion distortion compensation for lidar point clouds and greatly reducing the amount of interpolation calculations.

[0094] Specifically, step 301 determines the target measurement data at the beginning of the current acquisition frame, and uses the target measurement data to adjust the point cloud data block to the beginning of the frame. This may include:

[0095] Acquire the first measurement data before the beginning of the current acquisition frame and the second measurement data after the end of the current acquisition frame;

[0096] Using the first and second measurement data, the target measurement data at the frame start time is determined; wherein, the target measurement data includes rotation quaternions and three-axis coordinates;

[0097] Using the rotation quaternion and the three-axis coordinates, the point cloud data block is adjusted to the frame start time.

[0098] It should be noted that in the above steps, the first measurement data before the beginning of the current acquisition frame and the second measurement data after the end of the current acquisition frame are obtained. The first measurement data is the IMU pose q0 and position v0 at time t0 before the beginning of the frame, and the second measurement data is the IMU pose q0 and position v0 at time t0 after the end of the frame. e position q e and position v e Before the frame start time, the IMU corresponds to the pose q0 and position v0 at time t0, and after the frame end time, the IMU corresponds to t... e position q e and position v e Interpolation calculation of frame start time t s position q s and position v s Compensate the point cloud data blocks to t s time.

[0099] Specifically, step 303 generates the rotation and translation transformation amounts of the point cloud data block at the ordered timestamp based on the rotation quaternion changes and translation changes. This may include:

[0100] Interpolation parameters for the ordered timestamps of the point cloud data blocks from the first moment to the second moment are obtained by using the rotation quaternion change and translation change from the first moment to the second moment.

[0101] The rotation and translation transformations of the point cloud data blocks are generated using rotation quaternion changes, translation changes, and interpolation parameters.

[0102] It should be noted that in the above steps, refer to Figure 8 This invention provides a data schematic diagram of a point cloud data processing method. It uses the rotation quaternion change and translation change from a first time point to a second time point to interpolate and calculate the interpolation parameters for the ordered timestamps of the point cloud data blocks from the first time point to the second time point. The ordered timestamps are located between the first and second time points. The method then calculates t. i The rate of change of the interpolation at time t is used to calculate t. i time relative to t s The interpolation parameters are used to generate the rotation and translation transformations of point cloud data blocks by employing rotation quaternion changes, translation changes, and interpolation parameters.

[0103] For example, first calculate t. j time relative to t s The rotational quaternion change q trans Translational change v trans , where t s and t j The rotation quaternion and triaxial coordinates are q s q j v s v j According to t s and t j Given the rotation quaternion and spatial position, the rotation-translation matrix m is calculated. s and m j Calculate t j Time first for t s The change m of the rotation and translation matrix trans Therefore, based on m trans q was calculated trans and v trans Quaternion interpolation uses spherical linear interpolation, while translation transformation uses linear transformation interpolation, based on q. trans and v trans Interpolation calculation t i time relative to t s The time-interpolation parameters param1 and param2, q trans The corresponding quaternions w, x, y, z are respectively v transThe corresponding three-axis coordinates x, y, z are respectively

[0104] In this embodiment, the interpolation parameters can be determined by the following formula:

[0105]

[0106] Where param1 is q trans t i time relative to t s The interpolation parameters at time t, param2 is v trans t i time relative to t s Interpolation parameters at time points, For q trans The corresponding quaternions w, r i It is t i The rate of change of the interpolation at time step.

[0107] In this embodiment, according to q trans and v trans Interpolation calculation t i time relative to t s quaternion q i and three-axis coordinates v i q i The corresponding quaternions w, x, y, z are respectively v i The corresponding three-axis coordinates x, y, z are respectively

[0108]

[0109] v i =r i *v trans

[0110] Furthermore, refer to Figure 4 , Figure 4 yes Figure 1 The flowchart of step 103 of the point cloud data processing method provided in this embodiment of the invention is basically the same as the server testing method provided in the first embodiment of the invention. Step 103 may specifically include the following steps:

[0111] Step 401: Determine the sequence number of the laser points and the number of laser points in the point cloud data block;

[0112] In this embodiment, the point cloud data blocks are divided according to a preset number of packets. The laser points are then finely arranged according to the division order. To determine which point cloud data block a laser point belongs to, the sequence number of the laser point and the number of laser points in the point cloud data block are first determined. The sequence number of the laser point is calculated from the starting point of the point cloud data. Since the point cloud data blocks are divided according to a preset number of packets, the number of laser points in each point cloud data block is the same.

[0113] Step 402: Determine the point cloud data block to which the laser point belongs based on the laser point's serial number and the number of laser points in the point cloud data block.

[0114] It should be noted that the point cloud data block to which a laser point belongs can be determined based on the laser point's index and the number of laser points in the point cloud data block. For example, if there is a laser point index (point_index), which is calculated from the starting point of the point cloud data, and the point cloud data is divided into multiple point cloud data blocks, and each point cloud data block contains block_size laser points, the point cloud data block number to which the laser point belongs can be calculated using the laser point index (point_index) and the size of each point cloud data block (block_size). Specifically, dividing the laser point index (point_index) by the size of each point cloud data block (block_size) and returning the integer part of the quotient can be used to calculate the data block number (block_index) to which the laser point belongs, thus determining the point cloud data block to which the laser point belongs.

[0115] Step 403: Rotate the laser point using the rotation transformation amount of the point cloud data block, and translate the rotated laser point using the translation transformation amount of the point cloud data block to obtain the compensated target point cloud data.

[0116] In this embodiment, the laser points on the point cloud data block are subjected to the same motion distortion compensation as the rotation and translation transformations of the point cloud data block. Specifically, the laser points are rotated using the rotation transformation of the point cloud data block, and then translated using the translation transformation of the point cloud data block. This yields the compensated target point cloud data. Each point cloud data block in the current frame includes a preset number of laser points. Since the point cloud data blocks are sequentially divided, the UDP packet (i.e., the point cloud data block) to which a laser point belongs can be determined based on the laser point's index and the number of laser points in each point cloud data block. This allows for motion distortion compensation of the lidar point cloud data, performing a rotation transformation followed by a translation transformation. For example, point p has original triaxial coordinates v. p Point p is in the corresponding nth UDP packet, and the quaternion q corresponding to the nth packet is... n and three-axis coordinates vn The three-axis coordinates after motion distortion compensation are v′ p , v′ p =q n *v p +v n Calculate the point cloud data after motion distortion compensation of the current frame lidar point cloud data.

[0117] In this embodiment of the invention, the current frame point cloud data is divided into data blocks. The laser points on each data block use the same rotation and translation transformation, which greatly reduces the amount of interpolation calculation, improves the motion distortion compensation efficiency of the lidar point cloud data, and realizes accurate reconstruction of the three-dimensional environment of the point cloud.

[0118] To enable those skilled in the art to more clearly understand the overall flow of the point cloud data processing method disclosed in the above embodiments of the present invention, please refer to... Figure 5 The flow of the point cloud data processing method provided in the embodiments of the present invention. Figure 1 Let's take an example to illustrate.

[0119] S501, the lidar and IMU are synchronized in time and space;

[0120] S502 receives lidar data;

[0121] S503 preprocesses the data to obtain point cloud data blocks;

[0122] S504, acquire IMU data;

[0123] S505, obtained from vehicle position;

[0124] S506 performs motion distortion compensation on the point cloud data of the lidar;

[0125] S507 outputs compensated lidar point cloud data.

[0126] It should be noted that in the above steps, the LiDAR and IMU are synchronized in time and space to ensure that the data of the LiDAR and IMU are aligned in time and space. The LiDAR data is received via UDP protocol, and the data is preprocessed to obtain point cloud data blocks. Specifically, the LiDAR data is preprocessed according to the number of UDP protocol packets, dividing it into multiple point cloud data blocks. While acquiring LiDAR data, IMU data is received and parsed, and the vehicle's pose is calculated based on the IMU data. The pose includes rotational quaternions and three-axis coordinates. Motion distortion compensation is performed on the LiDAR point cloud data based on the vehicle's pose, and the LiDAR point cloud data after motion distortion compensation is output.

[0127] To enable those skilled in the art to more clearly understand the overall flow of the point cloud data processing method disclosed in the above embodiments of the present invention, please refer to... Figure 6 The flow of the point cloud data processing method provided in the embodiments of the present invention. Figure 2 Let's take an example to illustrate.

[0128] S601, the lidar point cloud is divided into data blocks according to UDP packets;

[0129] S602, calculate the rotation quaternion and three-axis coordinates of the first moment of the lidar point cloud frame;

[0130] S603, calculate qtrans and vtans at time tj relative to time ts;

[0131] S604, calculate the interpolation rate of change ri;

[0132] S605, calculate qi and vi at time ti relative to time ts;

[0133] S606, perform rotation and translation transformations based on qi and vi.

[0134] It should be noted that in the above steps, the LiDAR point cloud is divided into data blocks according to UDP packets, resulting in point cloud data blocks with the same number of UDP packets. The rotation quaternion and three-axis coordinates of the LiDAR point cloud at the first moment of the frame are calculated using the measurement data. The IMU before the first moment of the frame corresponds to the pose q0 and position v0 at time t0, and the IMU after the last moment of the frame corresponds to t... e position q e and position v e Interpolation calculation of frame start time t s position q s and position v s Compensate all point cloud data blocks to t s At any given time, after compensation, find the ordered timestamp t of the UDP. i Located in IMU data t s and t j Between, calculate t j time relative to t s The rotational quaternion change q trans Translational change v trann Calculate t i The rate of change of the interpolation at time r i ,in, According to q trans and v trans Interpolation calculation t i time relative to t s quaternion q i and three-axis coordinates v i qi The corresponding quaternions w, x, y, z are respectively v i The corresponding three-axis coordinates x, y, z are respectively

[0135] Specifically, the rotation and translation transformations corresponding to all UDP packets in the current frame point cloud are performed. Based on the UDP packet number to which each point belongs, motion distortion compensation is applied to the LiDAR point cloud data. First, a rotation transformation is performed, followed by a translation transformation. For example, point p has the original three-axis coordinates v. p Point p is in the corresponding nth UDP packet, and the quaternion q corresponding to the nth packet is... n and three-axis coordinates v n The three-axis coordinates after motion distortion compensation are v′ p , v′ p =q n *v p +v n Calculate the point cloud data after motion distortion compensation of the current frame lidar point cloud data.

[0136] Reference Figure 9 The diagram illustrates a structural block diagram of a point cloud data processing device according to an embodiment of the present invention. The device may include:

[0137] The data acquisition module 701 is used to acquire point cloud data blocks in the current acquisition frame of the lidar and measurement data from the inertial measurement unit; wherein, the point cloud data blocks are obtained by pre-dividing the laser points acquired by the lidar;

[0138] The compensation module 702 is used to adjust each point cloud data block to the beginning of the current acquisition frame and generate the rotation transformation and translation transformation of each point cloud data block using the measurement data.

[0139] The data compensation module 703 is used to determine the point cloud data block corresponding to the laser point, and to perform the same motion compensation on the laser point using the rotation transformation amount and translation transformation amount of the point cloud data block to obtain the compensated target point cloud data.

[0140] Furthermore, the data acquisition module 701 includes:

[0141] The synchronization submodule is used to pre-synchronize the lidar and inertial measurement unit in time;

[0142] The partitioning submodule is used to acquire the laser points collected by the lidar in the current acquisition frame, and to divide the laser points into data blocks using a preset number of message packets to obtain the point cloud data blocks of the current acquisition frame; wherein, the number of point cloud data blocks is the same as the preset number of message packets;

[0143] The acquisition submodule is used to acquire point cloud data blocks from the LiDAR after time synchronization, as well as measurement data from the inertial measurement unit.

[0144] Furthermore, the compensation generation module 702 includes:

[0145] The adjustment submodule is used to determine the target measurement data at the beginning of the current acquisition frame, and to adjust the point cloud data block to the beginning of the frame using the target measurement data;

[0146] The first determining submodule is used to obtain the ordered timestamps of the point cloud data blocks, use the ordered timestamps to determine the measurement data at the first and second moments, and obtain the rotation quaternion change and translation change from the first to the second moment; wherein, the ordered timestamps are between the first and second moments;

[0147] The generation submodule is used to generate the rotation transformation amount and translation transformation amount of the point cloud data block by using the rotation quaternion change amount and translation change amount from the first time moment to the second time moment.

[0148] Furthermore, the adjustment submodule includes:

[0149] The acquisition unit is used to acquire the first measurement data before the beginning of the current acquisition frame and the second measurement data after the end of the current acquisition frame.

[0150] A determining unit is used to determine the target measurement data at the beginning of the frame using first measurement data and second measurement data; wherein the target measurement data includes rotational quaternions and three-axis coordinates;

[0151] An adjustment unit is used to adjust the point cloud data block to the frame start time using the rotation quaternion and the three-axis coordinates.

[0152] Furthermore, the generation submodule includes:

[0153] The calculation unit is used to obtain the interpolation parameters of the ordered timestamps of the point cloud data blocks from the first time step to the second time step by using the rotation quaternion change and translation change.

[0154] The generation unit is used to generate the rotation and translation transformations of the point cloud data block using rotation quaternion changes, translation changes, and interpolation parameters.

[0155] Furthermore, the data compensation module 703 includes:

[0156] The second determination submodule is used to determine the sequence number of the laser point and the number of laser points in the point cloud data block;

[0157] The third determination submodule is used to determine the point cloud data block to which the laser point belongs based on the laser point's serial number and the number of laser points in the point cloud data block.

[0158] The transformation submodule is used to perform rotation transformation on the laser point using the rotation transformation amount of the point cloud data block, and to perform translation transformation on the rotated laser point using the translation transformation amount of the point cloud data block, so as to obtain the compensated target point cloud data.

[0159] The point cloud data processing device provided in this embodiment of the invention acquires point cloud data blocks in the current acquisition frame of the lidar and measurement data from the inertial measurement unit. The point cloud data blocks are obtained by pre-dividing the laser points acquired by the lidar. Each point cloud data block is adjusted to the beginning of the current acquisition frame. The rotation transformation amount and translation transformation amount of each point cloud data block are generated using the measurement data. The point cloud data block corresponding to the laser point is determined. The rotation transformation amount and translation transformation amount of the point cloud data block are used to perform the same motion compensation on the laser point to obtain the compensated target point cloud data. This invention divides a frame of LiDAR data into multiple data blocks, calculates the rotation and translation transformations of the point cloud data blocks, and directly applies the same rotation and translation transformations to the laser points in each data block. This achieves motion distortion compensation for the LiDAR point cloud, eliminating the need for linear interpolation of every point in the current frame. Instead, the current frame's point cloud data is divided into data blocks, and the laser points in each block use the same rotation and translation transformations, significantly reducing the amount of interpolation calculations. By transforming the point cloud data to the beginning of the current frame, the changes in pose and three-axis displacement caused by the vehicle's motion are removed, thereby restoring the true position of the laser point cloud, improving the efficiency of motion distortion compensation for LiDAR point cloud data, achieving accurate reconstruction of the three-dimensional environment of the point cloud, and further enhancing the safety of autonomous driving.

[0160] Reference Figure 10 The present invention also provides an electronic device, such as... Figure 10 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804. The processor 801, communication interface 802, and memory 803 communicate with each other via the communication bus 804.

[0161] Memory 803 is used to store computer programs;

[0162] When processor 801 executes a program stored in memory 803, it performs the following steps:

[0163] The point cloud data block in the current acquisition frame of the lidar and the measurement data of the inertial measurement unit are acquired; wherein, the point cloud data block is obtained by pre-dividing the laser points acquired by the lidar;

[0164] Adjust each point cloud data block to the beginning of the current acquisition frame, and use the measurement data to generate the rotation transformation and translation transformation of each point cloud data block;

[0165] The point cloud data block corresponding to the laser point is determined, and the same motion compensation is applied to the laser point using the rotation transformation and translation transformation of the point cloud data block to obtain the compensated target point cloud data.

[0166] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0167] The communication interface is used for communication between the aforementioned terminal and other devices.

[0168] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0169] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0170] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the point cloud data processing methods described in the above embodiments.

[0171] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0172] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0174] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A point cloud data processing method, characterized in that, The method includes: The point cloud data block in the current acquisition frame of the lidar and the measurement data of the inertial measurement unit are acquired; wherein, the point cloud data block is obtained by pre-dividing the laser points acquired in the current acquisition frame of the lidar according to the number of UDP packets; Adjusting each point cloud data block to the beginning of the current acquisition frame, and generating rotation and translation transformations for each point cloud data block using the measurement data, includes: determining the target measurement data at the beginning of the current acquisition frame; adjusting the point cloud data block to the beginning of the frame using the target measurement data; obtaining the ordered timestamp of the point cloud data block; determining the measurement data at the first and second moments using the ordered timestamp; obtaining the rotation quaternion change and translation change from the first moment to the second moment, wherein the ordered timestamp is between the first and second moments; and generating the rotation and translation transformations for the point cloud data block using the rotation quaternion change and translation change from the first moment to the second moment. The point cloud data block corresponding to the laser point is determined, and the same motion compensation is applied to the laser point using the rotation transformation and translation transformation of the point cloud data block to obtain the compensated target point cloud data.

2. The method according to claim 1, characterized in that, The process involves acquiring point cloud data blocks from the current capture frame of the lidar and measurement data from the inertial measurement unit; wherein the point cloud data blocks are pre-divided from the laser points captured by the lidar, including: The lidar and inertial measurement unit are synchronized in advance. The laser points collected by the lidar in the current acquisition frame are obtained, and the laser points are divided into data blocks using a preset number of packets to obtain point cloud data blocks for the current acquisition frame; wherein, the number of point cloud data blocks is the same as the preset number of packets. Acquire point cloud data blocks from the lidar after time synchronization, as well as measurement data from the inertial measurement unit.

3. The method according to claim 1, characterized in that, The step of determining the target measurement data at the beginning of the current acquisition frame, and using the target measurement data to adjust the point cloud data block to the beginning of the frame, includes: Acquire the first measurement data before the beginning of the current acquisition frame and the second measurement data after the end of the current acquisition frame; Using the first and second measurement data, the target measurement data at the frame start time is determined; wherein, the target measurement data includes rotation quaternions and three-axis coordinates; Using the rotation quaternion and the three-axis coordinates, the point cloud data block is adjusted to the frame start time.

4. The method according to claim 1, characterized in that, The step of generating the rotation and translation transformation amounts of the point cloud data blocks using the rotation quaternion changes and translation changes from the first time step to the second time step includes: Interpolation parameters for the ordered timestamps of the point cloud data blocks from the first moment to the second moment are obtained by using the rotation quaternion change and translation change from the first moment to the second moment. The rotation and translation transformations of the point cloud data blocks are generated using rotation quaternion changes, translation changes, and interpolation parameters.

5. The method according to claim 1, characterized in that, The process of determining the point cloud data block corresponding to the laser point, and using the rotation and translation transformations of the point cloud data block to perform motion compensation on the laser point to obtain compensated target point cloud data includes: Determine the sequence number of the laser points and the number of laser points in the point cloud data block; The point cloud data block to which the laser point belongs is determined based on the laser point's serial number and the number of laser points in the point cloud data block. The laser point is rotated using the rotation transformation amount of the point cloud data block, and then translated using the translation transformation amount of the point cloud data block to obtain the compensated target point cloud data.

6. A point cloud data processing device, characterized in that, The device includes: The data acquisition module is used to acquire point cloud data blocks in the current acquisition frame of the lidar and measurement data from the inertial measurement unit; wherein, the point cloud data blocks are pre-divided by dividing the laser points acquired in the current acquisition frame of the lidar according to the number of UDP packets; A compensation quantity generation module is used to adjust each point cloud data block to the beginning position of the current acquisition frame, and to generate rotation transformation and translation transformation quantities for each point cloud data block using the measurement data. The compensation quantity generation module includes: an adjustment submodule, used to determine the target measurement data at the beginning of the current acquisition frame, and to adjust the point cloud data block to the beginning of the frame using the target measurement data; a first determination submodule, used to obtain the ordered timestamps of the point cloud data blocks, and to determine the measurement data at the first and second times using the ordered timestamps to obtain the rotation quaternion change and translation transformation quantities from the first to the second time, wherein the ordered timestamps are between the first and second times; and a generation submodule, used to generate the rotation transformation and translation transformation quantities of the point cloud data blocks using the rotation quaternion change and translation transformation quantities from the first to the second time. The data compensation module is used to determine the point cloud data block corresponding to the laser point, and to perform the same motion compensation on the laser point using the rotation transformation and translation transformation of the point cloud data block to obtain the compensated target point cloud data.

7. The apparatus according to claim 6, characterized in that, The data acquisition module includes: The synchronization submodule is used to pre-synchronize the lidar and inertial measurement unit in time; The partitioning submodule is used to acquire the laser points collected by the lidar in the current acquisition frame, and to divide the laser points into data blocks using a preset number of message packets to obtain the point cloud data blocks of the current acquisition frame; wherein, the number of point cloud data blocks is the same as the preset number of message packets; The acquisition submodule is used to acquire point cloud data blocks from the LiDAR after time synchronization, as well as measurement data from the inertial measurement unit.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the point cloud data processing method as described in any one of claims 1 to 5.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the point cloud data processing method as described in any one of claims 1 to 5.

Images