Point cloud motion compensation method and system

Abstract

The point cloud motion compensation method and system provided in the specification obtain the speed of the target carrier through the speed sensor arranged on the target carrier, calculate the pose transformation amount of the target carrier at different time stamps relative to a target time stamp, determine the target pose compensation data of the laser radar coordinate system relative to the reference coordinate system in which the original point cloud data at different time stamps in the target time window is located, and perform motion compensation on the original point cloud data according to the target pose compensation data. The point cloud motion compensation method and system only use local poses, and by calculating the target pose compensation data at different time stamps, the pose compensation data of all original point cloud data in the target time window can be obtained, so that each frame of original point cloud data can be calculated in parallel, which is conducive to improving the calculation efficiency and the motion compensation processing speed, and meets the high real-time requirement of scenarios such as autonomous driving.

Description

Technical Field

[0001] This specification relates to the field of lidar technology, and in particular to a point cloud motion compensation method and system. Background Technology

[0002] LiDAR, an active sensor that perceives its surroundings by emitting laser beams, is primarily used in fields such as environmental perception for autonomous vehicles, robotic environmental perception, and drone mapping. Point cloud distortion arises from the motion of the LiDAR carrier relative to surrounding objects during data acquisition. As the LiDAR carrier moves, its coordinate system constantly changes, causing points in the same frame of the point cloud to reside in different coordinate systems, resulting in motion distortion and failing to accurately reflect the environment around the LiDAR at any given moment. Therefore, when the object on which the LiDAR is mounted, such as an autonomous vehicle, is in motion, motion compensation is needed for the point cloud to eliminate the distortion caused by motion. Summary of the Invention

[0003] This specification provides a point cloud motion compensation method and system that uses only local pose and allows parallel computation of each frame of point cloud data, which helps improve computational efficiency and motion compensation processing speed, and can meet the real-time requirements of scenarios such as autonomous driving.

[0004] Firstly, this specification provides a point cloud motion compensation method, comprising performing the following steps on each frame of raw point cloud data collected by a lidar mounted on a movable target carrier: acquiring the raw point cloud data; acquiring the carrier velocity of the target carrier within a target time window corresponding to the raw point cloud data; determining target pose compensation data of the lidar coordinate system relative to a reference coordinate system at different timestamps based on the carrier velocity, wherein the reference coordinate system includes the lidar coordinate system at the target timestamp, and the target timestamp is located within the target time window; and transforming the raw point cloud data to the reference coordinate system based on the target pose compensation data to obtain target point cloud data.

[0005] In some embodiments, determining the target pose compensation data of the lidar coordinate system relative to the reference coordinate system at different timestamps based on the carrier velocity includes: determining the second pose data of the carrier coordinate system of the target carrier relative to the first coordinate system at different timestamps based on the carrier velocity, wherein the first coordinate system includes the carrier coordinate system at the target timestamp; and determining the target pose compensation data based on the second pose data and the first pose data of the reference coordinate system relative to the first coordinate system.

[0006] In some embodiments, the original point cloud data includes N sets of point cloud data within the target time window and their corresponding N first timestamps, wherein N is an integer greater than 1.

[0007] In some embodiments, the N first timestamps include at least the frame header first timestamp at the start of the target time window and the frame tail first timestamp at the end of the target time window, and the target timestamp includes one of the frame header first timestamp and the frame tail first timestamp.

[0008] In some embodiments, obtaining the carrier velocity of the target carrier within the target time window corresponding to the original point cloud data includes: based on the velocity sensor of the target carrier, obtaining M carrier velocities of the target carrier within the target time window and their corresponding M second timestamps, where M is a positive integer.

[0009] In some embodiments, the velocity sensor includes a linear velocity sensor and an angular velocity sensor, and the carrier velocity includes the carrier linear velocity and the carrier angular velocity.

[0010] In some embodiments, determining the second pose data of the target carrier's coordinate system relative to the first coordinate system at different timestamps based on the carrier velocity includes: determining M second pose data of the carrier coordinate system relative to the first coordinate system at the M second timestamps based on the M carrier velocities.

[0011] In some embodiments, determining M second pose data of the carrier coordinate system relative to the first coordinate system at the M second timestamps based on the M carrier velocities includes, for each of the M second timestamps: determining second pose change data of the carrier coordinate system relative to the carrier coordinate system at the next second timestamp based on the carrier velocity corresponding to the current second timestamp; and superimposing the second pose change data with the second pose data of the carrier coordinate system relative to the first coordinate system at the current second timestamp to determine the second pose data of the carrier coordinate system relative to the first coordinate system at the next second timestamp.

[0012] In some embodiments, determining the target pose compensation data based on the second pose data and the first pose data of the reference coordinate system relative to the first coordinate system includes: determining N second pose data of the carrier coordinate system relative to the first coordinate system at N first timestamps based on the M second pose data; determining N third pose data of the carrier coordinate system relative to the reference coordinate system at the N first timestamps based on the N second pose data and the first pose data; and determining N target pose compensation data of the lidar coordinate system relative to the reference coordinate system at the N first timestamps based on the N third pose data and the first pose data.

[0013] In some embodiments, determining N second pose data of the carrier coordinate system relative to the first coordinate system at N first timestamps based on the M second pose data includes, for each of the N first timestamps: determining second pose change data of the carrier coordinate system at the current first timestamp relative to the carrier coordinate system at the adjacent previous second timestamp based on the carrier velocity corresponding to the previous second timestamp adjacent to the current first timestamp; and superimposing the second pose change data with the second pose data of the carrier coordinate system relative to the first coordinate system at the adjacent previous second timestamp to determine the second pose data of the carrier coordinate system relative to the first coordinate system at the current first timestamp.

[0014] In some embodiments, determining N third pose data of the carrier coordinate system relative to the reference coordinate system at the N first timestamps based on the N second pose data and the first pose data includes, for each first timestamp within the target time window: superimposing the second pose data at the current first timestamp with the fourth pose data of the first coordinate system relative to the reference coordinate system to determine the third pose data of the carrier coordinate system relative to the reference coordinate system at the current first timestamp.

[0015] In some embodiments, the target pose compensation data includes rotation compensation data and translation compensation data, wherein the rotation compensation data is represented by quaternions.

[0016] Secondly, this specification also provides a point cloud motion compensation system for performing motion compensation on raw point cloud data collected by a lidar mounted on a movable target carrier. The point cloud motion compensation system includes a velocity sensor and a computing device. The velocity sensor is mounted on the target carrier and detects the carrier's velocity during operation. The computing device is communicatively connected to the velocity sensor and the lidar, and during operation, obtains the carrier's velocity from the velocity sensor and the raw point cloud data from the lidar. The system includes at least one storage medium and at least one processor. The at least one storage medium stores at least one instruction set for performing point cloud motion compensation on the raw point cloud data. The at least one processor is communicatively connected to the at least one storage medium. When the point cloud motion compensation system is running, the at least one processor reads the at least one instruction set and implements the point cloud motion compensation method described in the first aspect of this specification.

[0017] As can be seen from the above technical solutions, the point cloud motion compensation method and system provided in this specification acquire the velocity of the target carrier through a velocity sensor mounted on the target carrier, calculates the pose transformation of the target carrier at different timestamps relative to the target timestamp, thereby determining the target pose compensation data of the LiDAR coordinate system relative to the reference coordinate system at different timestamps, and then performs motion compensation on the original point cloud data based on the target pose compensation data. The LiDAR point cloud motion compensation method and system provided in this specification can use motion compensation algorithms to solve the point cloud distortion problem, and motion compensate the point cloud data at different timestamps in the original point cloud data to the LiDAR coordinate system at the target timestamp. Therefore, the point cloud motion compensation method and system only need to be based on local pose data to calculate the target pose compensation data at different timestamps, thereby obtaining the pose compensation data of all point cloud data within the same target time window, enabling parallel calculation of each frame of point cloud data, which is beneficial to improving computational efficiency and motion compensation processing speed, and meeting the high real-time requirements of scenarios such as autonomous driving.

[0018] Other functions of the point cloud motion compensation methods and systems provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the point cloud motion compensation methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1A This specification illustrates point cloud diagrams of static scenes according to some embodiments provided herein;

[0021] Figure 1B This document illustrates point cloud distortion in a motion scene according to some embodiments provided in this specification;

[0022] Figure 2 The following diagram illustrates application scenarios of a point cloud motion compensation system provided according to some embodiments of this specification;

[0023] Figure 3 Exemplary hardware schematics of a computing device provided according to some embodiments of this specification are shown;

[0024] Figure 4 An exemplary flowchart of a point cloud motion compensation method provided according to some embodiments of this specification is shown;

[0025] Figure 5 A schematic diagram illustrating a quaternion space description provided according to some embodiments of this specification is shown;

[0026] Figure 6 An exemplary flowchart of a method for determining target pose compensation data according to some embodiments of this specification is shown;

[0027] Figure 7 An exemplary flowchart of a method for determining second pose data at a first timestamp according to some embodiments of this specification is shown; and

[0028] Figure 8 An exemplary flowchart is shown, illustrating another method for determining target pose compensation data according to some embodiments of this specification. Detailed Implementation

[0029] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0030] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0031] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0032] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0033] Figure 1A This specification illustrates point cloud diagrams of static scenes according to some embodiments provided herein; Figure 1B A schematic diagram of point cloud distortion in a moving scene is shown, according to some embodiments of this specification. For example... Figure 1A As shown, in a static scene, where the lidar carrier itself is stationary and there are no moving objects in the environment, the lidar's starting and ending positions are both at position 1, and its scanning start and end points are both at position 2. Therefore, the point cloud data acquired by the lidar is undistorted, and the laser beam forms a closed circle when it hits the ground.

[0034] like Figure 1BAs shown, in a motion scenario, either the lidar carrier is moving or there are moving objects in the environment, the lidar data acquisition requires a certain time period. During the lidar scan, the lidar carrier moves the lidar simultaneously at a velocity v in the direction indicated by the arrow, resulting in a rigid body transformation of the lidar coordinate system containing the point cloud data. At this time, the lidar's starting position 3 at the beginning of a frame of point cloud data acquisition and its ending position 4 at the end of the acquisition result in displacement. Furthermore, the lidar's continuous movement during acquisition causes data lag, leading to a deviation between the lidar's scanning starting point 5 and scanning ending point 6. Therefore, the point cloud data acquired by the lidar exhibits distortion. The lidar's movement relative to its surrounding environment stretches and distorts the 3D environment information, failing to accurately reflect the environment around the lidar at a given moment. Therefore, when the object on which the lidar is mounted (e.g., an autonomous vehicle) is in motion, motion compensation is needed for the point cloud to eliminate the distortion caused by motion.

[0035] Figure 2 This is a schematic diagram illustrating application scenarios of the point cloud motion compensation system 001 (hereinafter referred to as system 001) according to some embodiments of this specification. For example... Figure 2 As shown, system 001 may include target carrier 100, lidar 200, velocity sensor 300 and computing device 600.

[0036] The target carrier 100 can be any form of movable object. For example, the target carrier 100 can be a moving car, a movable robotic arm, or other movable objects, such as a robot vacuum cleaner, a drone, etc. For ease of description, we will use a scenario where the target carrier 100 is a car and system 001 is applied to an autonomous vehicle as an example. Those skilled in the art should understand that other forms of movable objects, such as the target carrier 100, are also within the scope of protection of this specification.

[0037] The lidar 200 consists of a laser emitter, a detector, and an information processing system. When operating, the lidar 200 emits laser signals and receives echo signals reflected from targets, thereby generating point cloud data. A frame of point cloud data includes multiple laser point data, each point containing X, Y, and Z three-dimensional coordinate information. The coordinate system containing these X, Y, and Z coordinates is the lidar coordinate system O. L -X L Y L Z L .

[0038] The lidar 200 can be mounted on the target carrier 100 and moves with the target carrier 100 when the target carrier 100 moves. The lidar 200 can be mounted at any position on the target carrier 100. In some embodiments, one or more lidars 100 can be mounted on the target carrier 100. Figure 2 In the illustrated embodiment, a lidar 200 is mounted on the target carrier 100. In some embodiments, the relative position of the lidar 200 and the target carrier 100 is fixed. That is, at any given moment, the lidar coordinate system O of the lidar 200 is fixed. L -X L Y L Z L Relative to the carrier coordinate system O where the target carrier 100 is located C -X C Y C Z C Fixed. The lidar coordinate system O can be pre-stored in system 001. L -X L Y L Z L Relative to the carrier coordinate system O C -X C Y C Z C The calibration parameters. Regarding the lidar coordinate system O... L -X L Y L Z L Relative to the carrier coordinate system O C -X C Y C Z C The calibration parameters will be described in detail later. For ease of description, we will use the same timestamp for the lidar coordinate system O. L -X L Y L Z L Relative to the carrier coordinate system O C -X C Y C Z C The calibration parameters are defined as the first pose data.

[0039] A velocity sensor 300 can be mounted on the target carrier 100. When operating, the velocity sensor 300 can detect the carrier velocity of the target carrier 100. In some embodiments, the system 001 can calculate the pose data of the target carrier 100 based on its carrier velocity at different times. The pose data may include displacement data and attitude data. In some embodiments, the target carrier 100 can perform linear movement; in this case, the velocity sensor 300 may include a linear velocity sensor to detect the carrier linear velocity of the target carrier 100. In some embodiments, the linear velocity sensor may be a single-axis velocity sensor, a multi-axis velocity sensor, etc. In some embodiments, the target carrier 100 can rotate; in this case, the velocity sensor 300 may include an angular velocity sensor to detect the carrier angular velocity of the target carrier 100. The angular velocity sensor may be an inertial measurement unit, such as a gyroscope, an accelerometer, a magnetometer, etc. In some embodiments, the angular velocity sensor may be a single-axis angular velocity sensor, a multi-axis angular velocity sensor, etc. In some embodiments, the angular velocity sensor may be an Euler angular velocity sensor to measure Euler angular velocity. For ease of demonstration, this specification will use Euler angular velocity as an example. Those skilled in the art should understand that any other type of angular velocity is within the scope of this specification. In some embodiments, the carrier angular velocity can be an RPY Euler angular velocity, i.e., rotating sequentially around the X-axis (roll angle), Y-axis (pitch angle), and Z-axis (yaw angle) of an inertial coordinate system. In some embodiments, the carrier angular velocity can be an XYZ Euler angular velocity, i.e., rotating sequentially around the Z-axis (yaw angle), the rotated Y'-axis (pitch angle), and the X”-axis (roll angle) after two rotations of a rigid, fixed coordinate system. For ease of description, we will use an RPY Euler angular velocity as an example.

[0040] The speed sensor 300 can be mounted on the target carrier 100 and moves along with the target carrier 100 when the target carrier 100 moves. The speed sensor 300 can be mounted at any position on the target carrier 100. In some embodiments, the relative position of the speed sensor 300 and the target carrier 100 is fixed. That is, at any given moment, the sensor coordinate system of the speed sensor 300 is relative to the carrier coordinate system O of the target carrier 100. C -X C Y C Z C Fixed. The carrier velocity measured by the speed sensor 300 is in the sensor coordinate system O. S -X S Y S Z SThe speed of the carrier. System 001 can pre-store the sensor coordinate system relative to the carrier coordinate system O. C -X C Y C Z C The calibration parameters include calibration translation and calibration rotation. System 001 can be based on the sensor coordinate system relative to the carrier coordinate system O. C -X C Y C Z C The calibration parameters are used to convert the carrier velocity measured by the speed sensor 300 to the carrier coordinate system O. C -X C Y C Z C The carrier velocity is defined as the velocity of the target carrier 100 in the carrier coordinate system O. For ease of description, we define the carrier velocity of the target carrier 100 as the velocity of the carrier coordinate system O. C -X C Y C Z C The speed of the carrier.

[0041] The computing device 600 may store data or instructions for executing the point cloud motion compensation method described herein, and may execute or be used to execute said data and / or instructions. The computing device 600 may include hardware devices with data processing capabilities and the necessary programs to drive the hardware devices. Alternatively, the computing device 600 may simply be a hardware device with data processing capabilities, or simply a program running on the hardware device. During operation, the computing device 600 may communicate with the velocity sensor 300 and the lidar 200. When operating, the computing device 600 may acquire the carrier velocity of the target carrier 100 from the velocity sensor 300 and the raw point cloud data from the lidar 200, and perform motion compensation on the raw point cloud data based on the point cloud motion compensation method described herein to eliminate point cloud distortion caused by the motion of the target carrier 100. In some embodiments, the computing device 600 may be installed on the target carrier 100, for example, inside or outside the target carrier 100.

[0042] The communication connection refers to any form of connection capable of directly or indirectly receiving information. In some embodiments, the computing device 600 can transmit data to the speed sensor 300 and the lidar 200 via a wireless communication connection; in some embodiments, the computing device 600 can also transmit data to the speed sensor 300 and the lidar 200 via a direct wire connection; in some embodiments, the computing device 600 can also establish an indirect connection with the speed sensor 300 and the lidar 200 by directly connecting to other circuits via a wire, thereby enabling data transmission. The wireless communication connection can be a network connection, Bluetooth connection, NFC connection, etc.

[0043] In some embodiments, the computing device 600 may be local or remote. In some embodiments, the computing device 600 may run on a cloud platform. In some embodiments, the computing device 600 may access data from the speed sensor 300 and the lidar 200 via a network. In some embodiments, the computing device 600 may connect directly to the speed sensor 300 and the lidar 200 to access the data therein.

[0044] In some embodiments, computing device 600 may include mobile devices, tablet computers, laptop computers, built-in devices in motor vehicles or similar content, or any combination thereof.

[0045] Figure 3 An exemplary hardware schematic diagram of a computing device 600 provided according to some embodiments of this specification is shown. For example... Figure 3 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. In some embodiments, the computing device 600 may also include I / O components 660.

[0046] The internal communication bus 610 can connect different system components to enable data communication between them, including storage medium 630, processor 620, communication port 650, and I / O component 660. For example, processor 620 can send data to storage medium 630 or other hardware such as I / O component 660 via internal communication bus 610. In some embodiments, internal communication bus 610 can be a Controller Area Network (CAN) bus, Industry Standard (ISA) bus, Extended Industry Standard (EISA) bus, Video Electronics Standard (VESA) bus, External Component Interconnect (PCI) bus, etc.

[0047] I / O component 660 can be used to input or output signals, data, or information. I / O component 660 supports input / output between computing device 600 and other components. In some embodiments, I / O component 660 may include input devices and output devices. Exemplary input devices may include a camera, keyboard, mouse, display screen, and microphone, or any combination thereof. Exemplary output devices may include a display device, a voice playback device (e.g., a speaker), a printer, a projector, or any combination thereof. Exemplary display devices may include a liquid crystal display (LCD), a light-emitting diode (LED) based display, a flat panel display, a curved display, a television device, a cathode ray tube (CRT), or any combination thereof.

[0048] Communication port 650 enables computing device 600 to communicate with external data sources, such as speed sensor 300 and lidar 200. The connection can be wired, wireless, or a combination of both. Wired connections can include cables, fiber optic cables, or telephone lines, or any combination thereof. Wireless connections can include Bluetooth, Wi-Fi, WiMax, WLAN, ZigBee, mobile networks (e.g., 3G, 4G, or 5G), or any combination thereof. In some embodiments, communication port 650 can be a standardized port, such as RS232 or RS485. In some embodiments, communication port 650 can be a specially designed port.

[0049] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 also includes at least one instruction set stored in the data storage device. The at least one instruction set is used to perform a motion compensation method on point cloud data. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the point cloud motion compensation method provided in this specification.

[0050] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. The at least one processor 620 is used to execute the at least one instruction set described above. When the computing device 600 is running, the at least one processor 620 reads the at least one instruction set and executes the point cloud motion compensation method provided in this specification according to the instructions of the at least one instruction set. The processor 620 can execute all the steps included in the point cloud motion compensation method. The processor 620 can be in the form of one or more processors. In some embodiments, the processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in this specification for the computing device 600. However, it should be noted that the computing device 600 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0051] Figure 4 An exemplary flowchart of a point cloud motion compensation method P100 provided according to some embodiments of this specification is shown. As previously described, system 001 can execute the point cloud motion compensation method P100 described in this specification. Specifically, processor 620 in computing device 600 can read an instruction set stored in its local storage medium and then execute the point cloud motion compensation method P100 described in this specification according to the instructions in the instruction set. Computing device 600 can execute the point cloud motion compensation method P100 on each frame of raw point cloud data in lidar 200. Figure 4 As shown, when the computing device 600 executes the point cloud motion compensation method P100 on each frame of raw point cloud data, the method P100 may include:

[0052] S120: Obtain the original point cloud data.

[0053] The computing device 600 can acquire the raw point cloud data from the lidar 200. The raw point cloud data can be point cloud data generated from the echo signals received by the lidar 200. That is, raw single-frame point cloud data from the lidar without XYZ calculation and motion compensation. The computing device 600 can perform calculations on the raw point cloud data to obtain single-frame point cloud XYZ data without motion compensation. A frame of raw point cloud data includes multiple lidar point data, each point including X, Y, and Z three-dimensional coordinate information. The coordinate system of the X, Y, and Z three-dimensional coordinate information of the lidar point is the lidar coordinate system O where the lidar 200 is located at the corresponding moment. L -X L Y L Z L For ease of description, we label the X, Y, and Z 3D coordinates in the original point cloud data as [x0, y0, z0]. T .

[0054] As mentioned earlier, the acquisition of one frame of raw point cloud data requires a time period. For ease of description, we define the time period corresponding to one frame of raw point cloud data as the target time window. The length of the target time window can be arbitrary and can be adaptively modified and adjusted according to usage requirements. For example, the length of the target time window can be 200ms, 500ms, 800ms, 1000ms, or even longer; this specification does not limit this.

[0055] In some embodiments, in step S120, when the computing device 600 acquires the original point cloud data from the lidar 200, it can also acquire the timestamp corresponding to each point in the original point cloud data. Multiple point data points can exist at the same timestamp. When the target carrier 100 is in a moving scene, the lidar coordinate system O of the XYZ data in the point cloud data at different timestamps within the target time window in the original point cloud data... L -X L Y L Z L They may be different. The method P100 can convert XYZ data in point cloud data from different timestamps to the same lidar coordinate system.

[0056] In some embodiments, in step S120, the computing device 600 may further divide the original point cloud data within the target time window into N groups of point cloud data based on the timestamps corresponding to the point cloud data. The N groups of point cloud data correspond to N different first timestamps. Each group of point cloud data includes multiple laser point data points under its corresponding first timestamp. Wherein, N is an integer greater than 1. N can be any integer greater than 1, such as 2, 3, 4, 5, or even more. For ease of description, we define the N first timestamps as follows: Where i = 1, 2, ..., N. In some embodiments, the N first timestamps can be uniformly distributed within the target time window. In some embodiments, the N first timestamps can be non-uniformly distributed within the target time window. In some embodiments, the time interval between adjacent first timestamps can be any number. In some embodiments, the time interval between adjacent first timestamps can be set based on the accuracy of motion compensation. The smaller the time interval between adjacent first timestamps, the higher the accuracy of motion compensation. In some embodiments, the time interval between adjacent first timestamps can be set based on real-time requirements. The larger the time interval between adjacent first timestamps, the faster the motion compensation calculation speed and the better the real-time performance.

[0057] In some embodiments, the N first timestamps may include at least the frame header first timestamp at the start of the target time window and the frame tail first timestamp at the end of the target time window. For ease of description, we define the frame header first timestamp as... The first timestamp of the frame tail is defined as First timestamp of frame header The point cloud data below is the frame header point cloud data. The frame tail first timestamp... The point cloud data below is the point cloud data at the end of the frame. The lidar coordinate system where the point cloud data at the beginning of the frame is located is the lidar coordinate system at the beginning of the frame. The coordinate system of the lidar where the point cloud data at the end of the frame is located is the lidar coordinate system at the end of the frame. First timestamp The coordinate system of the LiDAR where the point cloud data is located is

[0058] like Figure 4 As shown, the method P100 may further include:

[0059] S140: Obtain the velocity of the target carrier 100 within the target time window corresponding to the original point cloud data.

[0060] As previously described, the velocity sensor 300 can measure the velocity of the target carrier 100. The computing device 600 can obtain the timestamped carrier velocity from the velocity sensor 300. In some embodiments, the computing device 600 can obtain the carrier velocity from the velocity sensor 300 in real time or at regular intervals. The detection frequency of the velocity sensor 300 can be configured according to actual conditions.

[0061] For ease of description, we will use the velocity sensor 300, which includes both a linear velocity sensor and an angular velocity sensor, as an example. That is, the computing device 600 can obtain the timestamped linear velocity and angular velocity of the target carrier 100 within the target time window from the velocity sensor 300. For ease of description, we define the carrier linear velocity as v and the carrier angular velocity as ω.

[0062] Step S140 may include: acquiring M carrier velocities of the target carrier 100 within the target time window and their corresponding M second timestamps based on the speed sensor 300 of the target carrier 100, where M is a positive integer. M can be determined based on the detection frequency of the speed sensor 300. The N first timestamps and the M second timestamps may be the same or different. The time interval between adjacent first timestamps may be greater than or less than the time interval between adjacent second timestamps. For ease of description, we define the M second timestamps as follows: Where j = 1, 2, ..., M. For ease of description, we will use the second timestamp. The target carrier 100 is located in the carrier coordinate system marked as follows: For ease of description, we define the M carrier linear velocities within the target time window as v1, v2, ..., v j ..., v M In some embodiments, the carrier linear velocity can be a three-axis velocity. In this case, the carrier linear velocity v j It can be represented as v j =[v jx v jy v jz ] T When the target carrier 100 is a car, we consider the carrier linear velocity v of the target carrier 100 to be... j The direction is the carrier coordinate system where the target carrier 100 is located. In The direction of the axis. At this moment, the carrier's linear velocity v j It can be represented as v j =[v jx ,0,0] TFor ease of description, we define the M carrier angular velocities within the target time window as ω1, ω2, ..., ω... j 、…、ω M In some embodiments, the carrier angular velocity can be an Euler angular velocity. In this case, the carrier angular velocity ω j It can be represented as ω j =[ω jx ω jy ω jz ] T The tables represent the target carrier 100 rotating sequentially around its own carrier coordinate system. of axis, axis, The roll angle, pitch angle, and yaw angle of the axis.

[0063] Because the real-time inputs of carrier linear velocity and carrier angular velocity may not be synchronized, a "first-come, first-served" principle is adopted when acquiring the carrier velocity at the target timestamp, recording the latest signal input. For example... Figure 4 As shown, the method P100 may further include:

[0064] S160: Based on the carrier speed, determine the target pose compensation data of the lidar coordinate system relative to the reference coordinate system at different timestamps where the original point cloud data is located.

[0065] The reference coordinate system may include the lidar coordinate system under the target timestamp. The target timestamp is located within the target time window. The target timestamp can be any timestamp within the target time window. For example, the target timestamp can be any one of N first timestamps. In some embodiments, to facilitate subsequent processing and calculation of the point cloud data, we can convert a frame of raw point cloud data to the first timestamp at the end of the frame. Corresponding frame tail lidar coordinate system It is also possible to convert a frame of raw point cloud data to the first timestamp of the frame header. Corresponding frame header lidar coordinate system The target timestamp may include the first timestamp in the frame header. and the first timestamp of the frame tail One of them. The reference coordinate system may include the lidar coordinate system of the frame header point cloud data. and the frame tail lidar coordinate system where the frame tail point cloud data is located One of them. For ease of demonstration, in the following description, we will use the reference coordinate system as the frame-tail lidar coordinate system. This will be described using an example. Those skilled in the art will understand that the reference coordinate system is any first timestamp within the target time window. The following lidar coordinate system All of these are within the scope of protection of this instruction manual.

[0066] In some embodiments, step S160 may calculate the pose change of the target carrier 100 in the carrier coordinate system within the target time window based on the carrier velocity of the target carrier 100, and determine the pose change of the lidar 200 in the lidar coordinate system within the target time window based on the calibration parameters between the carrier coordinate system and the lidar coordinate system, thereby determining the target pose compensation data. In some embodiments, step S160 may convert the carrier velocity of the target carrier 100 in the carrier coordinate system to the lidar velocity in the lidar coordinate system based on the calibration parameters between the carrier coordinate system and the lidar coordinate system, and calculate the pose change of the lidar 200 in the lidar coordinate system within the target time window, thereby determining the target pose compensation data. In some embodiments, since the relative pose of the lidar coordinate system and the carrier coordinate system is fixed at the same timestamp, step S160 may determine the pose change of the lidar coordinate system based on the pose change of the carrier coordinate system.

[0067] As mentioned earlier, pose data can include translation and rotation data of the motion coordinate system relative to the reference coordinate system. The translation data can be represented by the translation matrix T = [T...]. x T y T z ] T It indicates. T x T represents the displacement of the origin of the moving coordinate system along the x-axis of the reference coordinate system. y T represents the displacement of the origin of the moving coordinate system along the y-axis of the reference coordinate system. z This represents the displacement of the origin of the motion coordinate system along the z-axis of the reference coordinate system. The rotation data can be represented in various ways, such as a rotation matrix, an Euler angle rotation matrix, quaternions, etc. To avoid the Euler angle gimbal lock problem and to ensure computational simplicity and accuracy, we will describe the rotation data as quaternions. Those skilled in the art should understand that other forms of rotation matrices are also within the scope of this specification. The target pose compensation data can include rotation compensation data and translation compensation data of the lidar coordinate system relative to the reference coordinate system at different timestamps. The rotation compensation data can be represented using quaternions.

[0068] Figure 5 A schematic diagram of a quaternion space description is shown. For example... Figure 5 As shown, the motion coordinate system is O-X'Y'Z', and the reference coordinate system is O-XYZ. (Vector) Let α, β, and γ be the rotation axes of the moving coordinate system O-X'Y'Z' relative to the reference coordinate system O-XYZ. The angles between the axis of rotation and the X, Y, and Z axes in the reference coordinate system O-XYZ define the axis of rotation. The directions of rotation. cosα, cosβ, and cosγ are the axes of rotation, respectively. The direction cosines of the X, Y, and Z axes relative to the reference coordinate system O-XYZ. The parameters of a quaternion are functions of time. A quaternion can be represented as... in, Let the moving coordinate system O-X'Y'Z' rotate about the axis relative to the reference coordinate system O-XYZ. The rotation angle. They are the axes of rotation The direction cosines of the X, Y, and Z axes relative to the reference coordinate system O-XYZ are given. The quaternion components satisfy q1. 2 +q2 2 +q3 2 +q4 2 =1.

[0069] Figure 6 An exemplary flowchart of a method for determining target pose compensation data according to some embodiments of this specification is shown. Figure 6 This shows step S160. For example... Figure 6 As shown, step S160 may include:

[0070] S162: Based on the carrier speed, determine the second pose data of the carrier coordinate system relative to the first coordinate system of the target carrier 100 at different timestamps.

[0071] The first coordinate system can be the carrier coordinate system under the target timestamp. As mentioned earlier, the target timestamp is used as the first timestamp at the end of the frame. Let's take an example to describe this. The first coordinate system can be the first timestamp at the end of the frame. The target carrier 100 is located in the carrier coordinate system. For ease of description, we will label the first coordinate system as...

[0072] Specifically, step S162 may include: based on the M carrier velocities (v1, v2, ..., v...), j ..., v M And ω2, ..., ω j 、…、ω M ), determine the M second timestamps Below, the carrier coordinate system is relative to the first coordinate system. M second pose data. Due to the first coordinate system First timestamp at the end of the frame The carrier coordinate system is used below, therefore, M second timestamps are calculated. The M second pose data can be obtained from the first timestamp at the end of the frame. First timestamp of the frame header The direction is calculated. That is, in step S162, it can be based on M second timestamps. First timestamp from the end of the frame The time distance is calculated from nearest to farthest. That is, M second timestamps. The middle can be calculated sequentially. Specifically, step S162 can obtain the second pose data for the next moment based on the second pose data of the previous moment and the carrier velocity of the previous moment, thereby calculating the carrier coordinate system relative to the first coordinate system at each moment sequentially using the carrier velocity at each moment. The second pose data. First timestamp at the end of the frame. Below, the carrier coordinate system is the first coordinate system. Therefore, the first timestamp at the end of the frame Below, the carrier coordinate system is relative to the first coordinate system. The second pose data can be represented as a quaternion. Translation matrix

[0073] Since the velocity sensor 300 acquires the carrier velocity over a short period, it can be approximated that the change in pose of the target carrier 100 between adjacent second time stamps is a uniform change in both translation and rotation. Specifically, step S162 may include processing the M second time stamps. Each second timestamp Execution: Based on the current second timestamp The corresponding carrier velocity (v) j+1 ω j+1 ), determine the next second timestamp The carrier coordinate system below Relative to the current second timestamp The carrier coordinate system below Second pose change data The second pose change data With the current second timestamp The carrier coordinate system below Relative to the first coordinate system Second pose data Overlay to determine the next second timestamp The carrier coordinate system below Relative to the first coordinate system Second pose data

[0074] Among them, the second pose change data This can be expressed as the following formula:

[0075]

[0076]

[0077] in,

[0078] Second pose data This can be expressed as the following formula:

[0079]

[0080]

[0081] like Figure 6 As shown, step S160 may further include:

[0082] S164: Based on second pose data and reference coordinate system Relative to the first coordinate system The first pose data is used to determine the target pose compensation data.

[0083] As mentioned above, the first pose data is the lidar coordinate system O at the same timestamp. L -X L Y L Z L Relative to the carrier coordinate system O C -X C Y C Z C The calibration parameters. For ease of demonstration, we label the first pose data as the quaternion Q. L，C Translation matrix T L，C The computing device 600 pre-stores the first pose data Q. L，C T L，C Specifically, step S164 may include:

[0084] S164-2: Based on the M second pose data Determine the N first timestamps Below, the carrier coordinate system Relative to the first coordinate system N second pose data

[0085] In some embodiments, one or more first timestamps may exist under adjacent second timestamps. The carrier coordinate system is calculated under the first timestamp. When calculating the second pose data relative to the first coordinate system, it can be based on the previous second timestamp adjacent to the first timestamp. Figure 7 An exemplary flowchart of a method for determining second pose data at a first timestamp, according to some embodiments of this specification, is shown. Figure 7 Step S164-2 is shown. (As...) Figure 7 As shown, specifically, step S164-2 may include processing the N first timestamps. Each first timestamp implement:

[0086] S164-22: Based on the current first timestamp The adjacent previous second timestamp (let's assume it's...) The corresponding carrier velocity (v) j ω j Determine the current first timestamp. The carrier coordinate system below Relative to the adjacent previous second timestamp The carrier coordinate system below The second pose change data; and

[0087] S164-24: Combine the second pose change data with the adjacent previous second timestamp. The carrier coordinate system below Relative to the first coordinate system Second pose data Overlay to determine the current first timestamp The carrier coordinate system below Relative to the first coordinate system Second pose data The calculation method for the second pose change data is similar to the aforementioned calculation method for the second pose change data, and will not be repeated here.

[0088] S164-4: Based on the N second pose data and the first pose data Q L，C T L，C Determine the N first timestamps Below, the carrier coordinate system Relative to the reference coordinate system N third pose data

[0089] Specifically, step S164-4 may include N first timestamps Each first timestamp Execute: Set the current first timestamp The second pose data below With the first coordinate system Relative to the reference coordinate system The fourth pose data is superimposed to determine the current first timestamp. Below, the carrier coordinate system Relative to the reference coordinate system The third pose data First pose data Q L，C T L，C The inverse matrix is ​​the fourth pose data. The fourth pose data is in the carrier coordinate system O. C -X C Y C Z C Relative to the lidar coordinate system O L -X L Y L Z L The calibration parameters are denoted as Q. c，L T C，L Among them, Q c，L =Q L，c -1 T C，L =--Q L，C -1 ·T L，C .

[0090] Third pose data This can be expressed as the following formula:

[0091]

[0092]

[0093] S164-6: Based on the N third pose data and the first pose data Q L，C T L，C Determine the N first timestamps Below, the lidar coordinate system Relative to the reference coordinate system N target pose compensation data Q i T i .

[0094] Specifically, step S164-6 may include processing each first timestamp within the target time window. Execute: Set the current first timestamp The third pose data below With the first pose data Q L，C T L，C Overlay to determine the current first timestamp The following laser lidar coordinate system Relative to the reference coordinate system Target pose compensation data Q i T i .

[0095] Target pose compensation data Q i T i This can be expressed as the following formula:

[0096]

[0097]

[0098] In some embodiments, step S160 may also be implemented in other ways. Figure 8 An exemplary flowchart is shown, illustrating another method for determining target pose compensation data according to some embodiments of this specification. Figure 8 Step S160 is shown. (As...) Figure 8 As shown, step S160 may include:

[0099] S166: Based on the fourth pose data of the reference coordinate system relative to the first coordinate system, the carrier velocity of the target carrier 100 is converted into the lidar velocity of the lidar 200 in the reference coordinate system; and

[0100] S167: Based on the laser radar speed, determine the target pose compensation data of the laser radar coordinate system relative to the reference coordinate system at different timestamps.

[0101] like Figure 4 As shown, the method P100 may further include:

[0102] S180: Based on the target pose compensation data Q i T i The original point cloud data is converted to the reference coordinate system. The target point cloud data is obtained.

[0103] In step S160, system 001 can calculate N first timestamps within the target time window where the original point cloud data is located. Point cloud data relative to the reference coordinate system Target pose compensation data Q i T i In some embodiments, step S180 may also be based on N first timestamps. Point cloud data relative to the reference coordinate system Target pose compensation data Q i T i Point cloud data at other timestamps within the target time window are obtained relative to the reference coordinate system through interpolation. The target pose compensation data is used to determine the point cloud data relative to the reference coordinate system at each timestamp within the target time window. The target pose compensation data. For ease of description, we define each timestamp within the target time window as t. k Where k = 1, 2, ... We will consider the various timestamps t within the same target time window. k The target pose compensation data is defined as the quaternion Q. k Data on relocation and compensation T k .

[0104] Step S180 can be based on various timestamps t within the same target time window. k The target pose compensation data Q k T k Motion compensation is performed on the original point cloud data to obtain a reference coordinate system. The target point cloud data is as follows. As mentioned earlier, we label the X, Y, and Z 3D coordinates in the original point cloud data as [x0, y0, z0]. T We will use the timestamp t k The X, Y, and Z 3D coordinate information in the raw point cloud data is marked as [x 0，k y 0，k , z 0，k ] T We label the X, Y, and Z 3D coordinates in the target point cloud data as [x1, y1, z1]. T , timestamp t k The X, Y, and Z 3D coordinate information of the target point cloud data is marked as [x 1，k y 1，k , z 1，k ] T In step S180, the computing device 600 can process each timestamp t. k The raw point cloud data below [x 0，k y 0，k , z 0，k ] T Perform motion compensation. Specifically, the timestamp t k The target point cloud data [x]1，k y 1，k , z 1，k ] T This can be expressed as the following formula:

[0105] [x 1，k y 1，k , z 1，k ] T =Q k [x 0，k y 0，k , z 0，k ] T +T k Formula (9)

[0106] In some embodiments, target pose compensation data at each timestamp within the same target time window can be calculated first, and then motion compensation can be performed on the original point cloud data. That is, motion compensation can be performed synchronously on the point clouds at different timestamps in each frame of point cloud data. The motion-compensated target point cloud data can be obtained by performing one operation on each frame of point cloud data, which enables parallel computation of each frame of point cloud data, improves computational efficiency and motion compensation processing speed, and can meet the real-time requirements of scenarios such as autonomous driving.

[0107] The deviation between the target point cloud data after motion compensation of the original point cloud data by method P100 and the ideal point cloud data after motion compensation under ideal conditions is the compensation error. The maximum compensation error σ corresponding to method P100 is... max This can be expressed as the following formula:

[0108] σ max =(tΔω+Δv+ωvdt)Δt formula (10)

[0109] Where r is the maximum detection range of the point cloud. Different lidar 200s have different values ​​for r. In some embodiments, r can be any number, such as 50m, 100m, 200m, 300m, 500m, 1000m, or even smaller or larger, etc. ω is the average angular velocity (rad / s) of the velocity sensor 300 within the target time window. Δω is the angular velocity error (rad / s) of the velocity sensor 300. v is the average linear velocity (rad / s) of the velocity sensor 300 within the target time window. Δv is the linear velocity error (rad / s) of the velocity sensor 300. dt is the pose integration period. In some embodiments, dt = 50ms / 1200 ≈ 40μs. Δt is the point cloud data acquisition period. In some embodiments, Δt = 50ms. Those skilled in the art should understand that the above data are merely illustrative, and other data of any size are within the scope of this specification. As can be seen from formula (10), when the pose integration period is sufficiently small, the error accuracy of point cloud data motion compensation is almost only affected by the accuracy of the velocity sensor 300 of the target carrier 100. Therefore, the point cloud motion compensation method P100 and system 001 described in this specification can improve the motion compensation accuracy of point cloud data.

[0110] In summary, the point cloud motion compensation method P100 and system 001 provided in this specification acquire the carrier velocity of the target carrier 100 through a velocity sensor 300 mounted on the target carrier 100. This velocity sensor then calculates the pose information of the target carrier 100 relative to the target time stamp at different timestamps, thereby determining the target pose compensation data of the lidar coordinate system relative to the reference coordinate system at different timestamps. Based on this target pose compensation data, motion compensation is performed on the original point cloud data. The point cloud motion compensation method P100 and system 001 provided in this specification can use motion compensation algorithms to solve the point cloud distortion problem, motion compensating the point cloud data at different timestamps in the original point cloud data to the lidar coordinate system at the target time stamp. Therefore, the point cloud motion compensation method P100 and system 001 only need to calculate the target pose compensation data at different timestamps to obtain the pose compensation data of all point cloud data within the same time window. This allows for parallel calculation of each frame of point cloud data, thereby saving computing resources, improving computing efficiency, accelerating the calculation speed of motion compensation, meeting the real-time requirements of the system, and achieving the goal of real-time motion compensation. Meanwhile, when calculating target pose compensation data, the point cloud motion compensation method P100 and system 001 do not use global pose, but local pose, thus avoiding continuous accumulation of storage space and saving storage space. Furthermore, in the point cloud motion compensation method P100 and system 001, rotational attitude and relative displacement are not calculated separately, which ensures error accuracy and improves the accuracy of point cloud motion compensation.

[0111] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for motion compensation of point cloud data. When the executable instructions are executed by a processor, they instruct the processor to perform the steps of the point cloud motion compensation method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 600, the program code causes the computing device 600 to perform the steps of point cloud data motion compensation described herein. The program product for implementing the above method may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on the computing device 600. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system (e.g., processor 620). The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 600, partially on computing device 600, as a standalone software package, partially on computing device 600 and partially on a remote computing device, or entirely on a remote computing device.

[0112] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0113] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0114] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0115] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0116] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. The entire contents used for all purposes, except for any history of prosecution documents associated with it, that may be inconsistent with or conflict with this document, or that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0117] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A point cloud motion compensation method, characterized in that, The method includes performing the following steps on each frame of raw point cloud data acquired by a lidar mounted on a movable target carrier: The original point cloud data is obtained, which includes N sets of point cloud data within the corresponding target time window and their corresponding N first timestamps, wherein N is an integer greater than 1; Based on the speed sensor of the target carrier, M carrier velocities and their corresponding M second timestamps within the target time window are obtained, where M is a positive integer; Based on the carrier velocity, target pose compensation data is determined relative to a reference coordinate system in the lidar coordinate system of the original point cloud data at different timestamps. The reference coordinate system includes the lidar coordinate system at the target timestamp, and the target timestamp is located within the target time window. Based on the target pose compensation data, the original point cloud data is transformed into the reference coordinate system to obtain the target point cloud data.

2. The point cloud motion compensation method as described in claim 1, characterized in that, The determination of target pose compensation data based on the carrier velocity, where the original point cloud data is located in the lidar coordinate system relative to the reference coordinate system at different timestamps, includes: Based on the carrier velocity, determine the second pose data of the target carrier's carrier coordinate system relative to a first coordinate system at different timestamps, wherein the first coordinate system includes the carrier coordinate system at the target timestamp; and Based on the second pose data and the first pose data of the reference coordinate system relative to the first coordinate system, the target pose compensation data is determined.

3. The point cloud motion compensation method as described in claim 1, characterized in that, The N first timestamps include at least the frame header first timestamp at the start of the target time window and the frame tail first timestamp at the end of the target time window, and the target timestamp includes one of the frame header first timestamp and the frame tail first timestamp.

4. The point cloud motion compensation method as described in claim 1, characterized in that, The speed sensor includes a linear velocity sensor and an angular velocity sensor, and the carrier speed includes the carrier linear velocity and the carrier angular velocity.

5. The point cloud motion compensation method as described in claim 2, characterized in that, The step of determining the second pose data of the target carrier's coordinate system relative to the first coordinate system at different timestamps based on the carrier velocity includes: Based on the M carrier velocities, M second pose data of the carrier coordinate system relative to the first coordinate system are determined at the M second timestamps.

6. The point cloud motion compensation method as described in claim 5, characterized in that, The determination of M second pose data of the carrier coordinate system relative to the first coordinate system at the M second timestamps based on the M carrier velocities includes determining each of the M second timestamps as follows: Based on the carrier velocity corresponding to the current second timestamp, determine the second pose change data of the carrier coordinate system at the next second timestamp relative to the carrier coordinate system at the current second timestamp; as well as The second pose change data is superimposed with the second pose data of the carrier coordinate system relative to the first coordinate system at the current second timestamp to determine the second pose data of the carrier coordinate system relative to the first coordinate system at the next second timestamp.

7. The point cloud motion compensation method as described in claim 5, characterized in that, The step of determining the target pose compensation data based on the second pose data and the first pose data of the reference coordinate system relative to the first coordinate system includes: Based on the M second pose data, determine N second pose data of the carrier coordinate system relative to the first coordinate system under the N first timestamps; Based on the N second pose data and the first pose data, determine N third pose data of the carrier coordinate system relative to the reference coordinate system at the N first timestamps; and Based on the N third pose data and the first pose data, N target pose compensation data of the lidar coordinate system relative to the reference coordinate system are determined at the N first timestamps.

8. The point cloud motion compensation method as described in claim 7, characterized in that, The step of determining the N second pose data of the carrier coordinate system relative to the first coordinate system under the N first timestamps based on the M second pose data includes determining the N second pose data of the carrier coordinate system relative to the first coordinate system under the N first timestamps: Based on the carrier velocity corresponding to the previous second time stamp adjacent to the current first time stamp, determine the second pose change data of the carrier coordinate system under the current first time stamp relative to the carrier coordinate system under the previous adjacent second time stamp; as well as The second pose change data is superimposed with the second pose data of the carrier coordinate system relative to the first coordinate system at the adjacent previous second timestamp to determine the second pose data of the carrier coordinate system relative to the first coordinate system at the current first timestamp.

9. The point cloud motion compensation method as described in claim 7, characterized in that, The step of determining N third pose data of the carrier coordinate system relative to the reference coordinate system under the N first timestamps based on the N second pose data and the first pose data includes determining N third pose data of the carrier coordinate system relative to the reference coordinate system under the N first timestamps, including for each first timestamp within the target time window: The second pose data at the current first timestamp is superimposed with the fourth pose data of the first coordinate system relative to the reference coordinate system to determine the third pose data of the carrier coordinate system relative to the reference coordinate system at the current first timestamp.

10. The point cloud motion compensation method as described in claim 1, characterized in that, The target pose compensation data includes rotation compensation data and translation compensation data, and the rotation compensation data is represented by quaternions.

11. A point cloud motion compensation system for performing motion compensation on raw point cloud data collected by a lidar mounted on a movable target carrier, characterized in that, The point cloud motion compensation system includes: A speed sensor, mounted on the target carrier, detects the carrier speed during operation; and A computing device, communicatively connected to the velocity sensor and the lidar, acquires the carrier velocity from the velocity sensor and the raw point cloud data from the lidar during operation, including: At least one storage medium storing at least one instruction set for performing point cloud motion compensation on the original point cloud data; and At least one processor is communicatively connected to the at least one storage medium. When the point cloud motion compensation system is running, the at least one processor reads the at least one instruction set and implements the point cloud motion compensation method according to any one of claims 1 to 10.

Images