A laser point cloud de-distortion method and related device

Abstract

This application discloses a laser point cloud distortion correction method and related equipment, including: a controller acquiring laser point cloud data, target object detection data, and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time of all laser points and their coordinates in the vehicle coordinate system. The target object detection data includes the size and motion information of the target object, and the vehicle positioning data includes the vehicle's coordinates in the world coordinate system. Then, based on the target object detection data, a bounding box for the target object is obtained, and the attributes of the laser points are determined based on the bounding box and the data in the laser point cloud. Finally, based on the attributes of the laser points and the vehicle positioning data, the coordinates of the laser points in the vehicle coordinate system at the acquisition time are distorted to obtain the coordinates of the laser points in the vehicle coordinate system at the image exposure time. This eliminates the problem of compromised positional accuracy caused by the different reference coordinate systems of laser points acquired at different times due to the relative motion between the vehicle and the surrounding environment.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a laser point cloud distortion correction method, a laser point cloud distortion correction device, a controller, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the development of intelligent driving technology, environmental perception sensors are widely used. For example, radar, vehicle cameras, infrared sensors, and temperature sensors are used to extract information about the external environment, determine the vehicle's driving conditions, and thus automatically take driving strategies or assist the driver.

[0003] Among them, LiDAR (Light Detection and Ranging) features high resolution and multi-target tracking capabilities, enabling real-time construction of a 3D model of the vehicle's surrounding environment. This LiDAR-built 3D model not only supports scene modeling and dynamic obstacle tracking during vehicle movement but can also be combined with ground truth detection techniques to generate a large dataset of labeled images for training target detection models, thereby improving the performance of target detection technology.

[0004] However, due to the physical characteristics of lidar, the points in a single frame of lidar point cloud are usually not collected at the same time. Therefore, in scenarios where the vehicle is in relative motion with the surrounding environment, distortion will occur, which will affect the accuracy of the points projected from the lidar point cloud. Summary of the Invention

[0005] In view of this, this application provides a laser point cloud distortion correction method and related equipment to solve the problem that the laser point cloud collected by lidar is distorted when the vehicle is in relative motion with the surrounding environment, resulting in inaccurate measurement results.

[0006] In a first aspect, this application provides a laser point cloud distortion correction method, the method comprising:

[0007] The system acquires laser point cloud data, target detection data, and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time and coordinates of all laser points collected by the vehicle's lidar within the unit time period. The target detection data includes the size and motion information of the target within the unit time period. The vehicle positioning data includes the coordinates of the vehicle in the world coordinate system within the unit time period.

[0008] Based on the size and motion information of the target object within the unit time period, obtain the identification frame of the target object within the unit time period;

[0009] The attributes of the laser point are determined based on the data of the identification box and at least one laser point in the laser point cloud;

[0010] Based on the attributes of the laser point and the vehicle positioning data, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are distorted to obtain the coordinates of the laser point in the vehicle coordinate system at the image exposure time. The image exposure time and the acquisition time belong to the unit time period.

[0011] In some possible implementations, the identification box further includes a target object attribute identifier, and determining the attribute of the laser point based on the identification box and data of at least one laser point in the laser point cloud includes:

[0012] When the laser point is located on or within the marking frame, the laser point is determined to be the target object laser point, and the target object laser point has the same target object attribute identifier as the marking frame.

[0013] When the laser point is located outside the marking frame, the laser point is determined to be a background laser point.

[0014] In some possible implementations, the laser point is a target object laser point, and the step of correcting the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle positioning data to obtain the laser point's coordinates in the vehicle coordinate system at the image exposure time includes:

[0015] Based on the target object attribute identifier of the laser point, query the target object detection data to obtain the first pose change of the target object corresponding to the laser point in the vehicle coordinate system between the acquisition time and the image exposure time.

[0016] Based on the first pose change and the rotation parameters at the acquisition time, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are distorted to obtain the coordinates of the laser point in the world coordinate system at the image exposure time.

[0017] Based on the coordinates of the laser point in the world coordinate system at the time of image exposure and the rotation parameters at the time of image exposure, the coordinates of the laser point in the vehicle coordinate system at the time of image exposure are obtained.

[0018] In some possible implementations, the laser point is a background laser point. The step of correcting the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle positioning data, to obtain the laser point's coordinates in the vehicle coordinate system at the image exposure time, includes:

[0019] Based on the rotation parameters at the acquisition time, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are converted to the coordinates in the world coordinate system.

[0020] Based on the vehicle positioning data, the second pose change of the vehicle in the world coordinate system between the acquisition time and the image exposure time is obtained;

[0021] Based on the coordinates of the laser point in the world coordinate system at the acquisition time, the second pose change, and the rotation parameters at the image exposure time, the coordinates of the laser point in the vehicle coordinate system at the image exposure time are obtained.

[0022] In some possible implementations, the vehicle positioning data is obtained based on the vehicle's magnetic declination, speed, and acceleration within the unit time period.

[0023] In some possible implementations, the motion information of the target object within the unit time period includes the target object's coordinates in the vehicle coordinate system, its heading angle, and its speed relative to the vehicle.

[0024] Secondly, this application provides a laser point cloud distortion correction apparatus, which includes modules for performing the laser point cloud distortion correction method in the first aspect or any possible implementation of the first aspect, specifically including:

[0025] The acquisition module is used to acquire laser point cloud data, target detection data, and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time and coordinates of all laser points acquired by the vehicle's lidar within the unit time period. The target detection data includes the size and motion information of the target within the unit time period. The vehicle positioning data includes the coordinates of the vehicle in the world coordinate system within the unit time period.

[0026] The identification module is used to obtain the identification frame of the target object within the unit time period based on the size information and movement information of the target object within the unit time period;

[0027] The classification module is used to determine the attributes of the laser point based on the data of the identification box and at least one laser point in the laser point cloud;

[0028] The distortion correction module is used to correct the distortion of the coordinates of the laser point in the vehicle coordinate system at the acquisition time based on the attributes of the laser point and the vehicle positioning data, so as to obtain the coordinates of the laser point in the vehicle coordinate system at the image exposure time, wherein the image exposure time and the acquisition time belong to the unit time period.

[0029] The device can also be used to perform the information processing method as described in any of the implementations of the first aspect.

[0030] Thirdly, this application provides a controller. The vehicle controller includes a processor and a memory. The memory stores computer instructions; the processor executes the methods described in the first aspect of this application or any possible implementation thereof, according to the computer instructions.

[0031] Fourthly, this application provides a computer-readable medium storing instructions that, when executed on a computer device, cause the computer device to perform the method described in the first aspect of this application or any possible implementation thereof.

[0032] Fifthly, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect of this application or any possible implementation thereof.

[0033] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods.

[0034] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0035] This application provides a laser point cloud distortion correction method. On one hand, this method corrects the distortion of laser point cloud data collected per unit time, eliminating the problem of compromised positional accuracy caused by the different coordinate systems referenced by laser points collected at different times due to the relative motion of the vehicle and the surrounding environment. On the other hand, this method directly utilizes target object detection data based on the original point cloud, eliminating the need for target object detection on the distorted laser point cloud, thus improving image annotation efficiency. Attached Figure Description

[0036] Figure 1 This is a schematic diagram illustrating laser point cloud distortion as disclosed in an embodiment of this application;

[0037] Figure 2 This is a flowchart of a laser point cloud distortion correction method disclosed in an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of the structure of a laser point cloud distortion correction device disclosed in an embodiment of this application. Detailed Implementation

[0039] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0040] The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. The terms "first" and "second" in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, chronological order of operations, or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0041] First, some technical terms involved in the embodiments of this application will be introduced.

[0042] Environmental perception sensors are devices used to sense and acquire information about the surrounding environment. Common environmental perception sensors include: radar, such as millimeter-wave radar, ultrasonic radar, lidar (light detection and ranging, LiDAR), and front-corner radar; vehicle cameras, such as monocular cameras, 360-degree surround view cameras, and surround-view fisheye cameras; infrared sensors; and temperature sensors. As tools for vehicles to extract information about the external environment, the performance of environmental perception sensors directly affects the quality of intelligent driving technology.

[0043] LiDAR, with its high resolution and multi-target tracking capabilities, can acquire depth information in real time and construct a 3D model of the vehicle's surrounding environment. This 3D model not only supports scene modeling and dynamic obstacle tracking during vehicle movement but can also be combined with ground truth detection techniques. By projecting the data onto images captured by an onboard camera, it generates a large dataset of labeled images to train object detection models, thereby improving the performance of object detection technology.

[0044] However, since lidar typically acquires laser point clouds by periodically rotating and scanning the surrounding environment, the points in a single frame of the laser point cloud are usually not acquired simultaneously. Therefore, distortion occurs in scenarios where the vehicle is in relative motion with its surroundings, affecting the accuracy of the projected points from the laser point cloud. Furthermore, when the vehicle is equipped with multiple panoramic cameras to acquire images of the surrounding environment more comprehensively and accurately, the degree of distortion may be exacerbated due to accumulated errors.

[0045] In some possible situations, when there are static targets in the surrounding environment, such as stationary cars or pedestrians, road signs, obstacles, etc., the collected laser point cloud may be distorted due to vehicle movement, resulting in errors when projected onto the image captured by the vehicle-mounted camera.

[0046] In some possible scenarios, there are dynamic objects in the surrounding environment; these objects could be moving cars, electric vehicles, bicycles, or pedestrians. See also Figure 1 The diagram illustrates the distortion of laser point clouds. In the upper image, the vehicle is stationary. During the lidar scan, the target object moves downwards, causing the laser beam to strike the target at a different position at the start of the scan compared to the end. For example, at the start of the scan, the laser beam might strike the upper part of the target, while at the end, due to the target's downward movement, it strikes the lower part, resulting in spatial distortion of the collected point cloud data. If the vehicle is in motion, such as in the lower image where it is moving towards the target, the target's downward movement causes the laser beam's strike position to change continuously at different times. Furthermore, the lidar's own movement alters the relative scanning position. The lidar's scanning path is fixed (from the start to the end of the scan), but because both the target and the lidar are moving, their paths do not match, leading to a significant change in the laser beam's strike position compared to the start, resulting in point cloud data distortion.

[0047] In view of this, this application provides a laser point cloud distortion correction method and related apparatus to solve the problem that the laser point cloud collected by lidar is distorted when the vehicle is in relative motion with the surrounding environment, resulting in inaccurate measurement results.

[0048] Specifically, this method involves a controller acquiring laser point cloud data, target detection data, and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time and coordinates in the vehicle coordinate system for all laser points collected by the vehicle's LiDAR within the unit time period. The target detection data includes the size and motion information of the target within the unit time period. The vehicle positioning data includes the vehicle's coordinates in the world coordinate system within the unit time period. Then, based on the target's size and motion information within the unit time period, a bounding box for the target is obtained. The attributes of the laser point are determined based on the bounding box and data from at least one laser point in the laser point cloud. Finally, the controller corrects the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle positioning data, obtaining the laser point's coordinates in the vehicle coordinate system at the image exposure time.

[0049] This method corrects distortion of the laser point cloud data collected per unit time, eliminating the problem of compromised positional accuracy caused by the different coordinate systems referenced by laser points collected at different times due to the relative motion of the vehicle and the surrounding environment. On the other hand, this method directly utilizes the target detection data based on the original point cloud, eliminating the need to perform target detection on the distorted laser point cloud, thus improving the efficiency of image annotation.

[0050] The method provided in this application will be described below with reference to specific embodiments.

[0051] Figure 2 This is a flowchart illustrating a laser point cloud distortion correction method disclosed in an embodiment of this application. The method includes:

[0052] S202: The controller acquires laser point cloud data, target detection data, and vehicle positioning data within a unit time period.

[0053] The laser point cloud data includes the acquisition time and coordinates in the vehicle coordinate system of all laser points collected by the vehicle's LiDAR within a unit time period. The vehicle's LiDAR is typically a mechanical LiDAR that collects laser point clouds and corresponding data by rotating and scanning the surrounding environment within a certain unit time (e.g., at a frequency of 10 Hz, with a rotation acquisition cycle of 100 milliseconds). The collected laser point cloud and corresponding data include the coordinates in the vehicle coordinate system and the acquisition time; that is, the collected laser point cloud data includes the position information and timestamp of each laser point.

[0054] Target detection data can be provided by the vehicle's target detection module and may include the target's size and motion information within a unit time period. The motion information may include the target's coordinates in the vehicle's coordinate system, the heading angle between the target and the vehicle, and the target's velocity relative to the vehicle. In some possible implementations, the input to the target detection module can be laser point cloud data collected by the vehicle's LiDAR, thus allowing direct combination of the laser point cloud data to obtain target detection data including time information.

[0055] Vehicle positioning data can be provided by the vehicle's positioning module, including the vehicle's coordinates in the world coordinate system within a unit time period. In some possible implementations, the vehicle positioning data can be obtained based on the vehicle's magnetic declination, speed, and acceleration within a unit time period. For example, the vehicle's positioning module can be a GNSS / INS integrated navigation system, achieving precise positioning through satellite signals. It uses an inertial measurement unit (IMU) to obtain the vehicle's magnetic declination, speed, and acceleration, and then uses this data to obtain the vehicle's coordinates in the world coordinate system within a unit time period. For instance, it can combine a piecewise uniform acceleration motion model and a trajectory calculation algorithm to obtain the vehicle's position change in the world coordinate system, and then determine the vehicle's coordinates in the world coordinate system within a unit time period based on the position change and the origin of the world coordinate system. The origin of the world coordinate system can be the vehicle's position information corresponding to the earliest timestamp (e.g., the time corresponding to the first valid data point within the unit time period). Among them, valid data refers to data that has been filtered out due to unstable data collection (such as data segments displaying "NA") and has undergone format conversion under preset rules (such as unified units, unified timestamp length, etc.).

[0056] In some possible implementations, when the controller obtains the image exposure time of the vehicle-mounted camera, the controller can obtain the corresponding vehicle positioning data at that time, for example, by looking up a table or interpolating to obtain the vehicle's coordinates in the world coordinate system at the image exposure time.

[0057] It should be noted that, in the embodiments of this application, the controller can periodically send data collection requests to the lidar, target detection module, or positioning module to obtain laser point cloud data, target detection data, and vehicle positioning data within a unit time period. Alternatively, the aforementioned modules can autonomously collect data and send it to the controller.

[0058] S204: The controller obtains the bounding box of the target object within a unit time period based on the size and motion information of the target object within a unit time period.

[0059] Bounding boxes are used to describe the location and orientation of a target. For example, a bounding box can be a two-dimensional (2D) bounding box or a three-dimensional (3D) bounding box. A 2D bounding box can identify the size and position of the target in the image to be detected, while a 3D bounding box can identify the size, position, and orientation of the target in space.

[0060] The controller can obtain the bounding box (e.g., the vertex coordinates of the bounding box) of the target object within a unit time period based on the target object's size information, its coordinates in the vehicle coordinate system (e.g., the coordinates of the target object's edge points or key points), the heading angle between the target object and the vehicle, and the target object's speed relative to the vehicle. The selection of the target object's edge points or key points should be based on the accuracy of recognition or the ease of acquisition; for example, it could be based on the bottom surface of the bounding box.

[0061] S206: The controller determines the attributes of the laser point based on the data of the identification box and at least one laser point in the laser point cloud.

[0062] The controller can determine whether a laser point belongs to the background or the target object based on the relationship between the data in the laser point cloud and the bounding box. The background laser point corresponds to static elements in the surrounding environment, such as street signs and obstacles. In other words, the distortion of the background laser point comes from the movement of the vehicle itself. The target object laser point corresponds to the dynamic target object. The distortion of the target object laser point comes from both the movement of the vehicle itself and the movement of the target object. That is, the distortion of the target object laser point comes from the pose change of the target object in the vehicle coordinate system.

[0063] In some possible implementations, the controller can determine the attributes of the laser point in the following ways: When the laser point is located on or within the marker frame, the controller can determine that the laser point is a target object laser point, wherein the target object laser point and the marker frame have the same target object attribute identifier; that is, the controller can assign the attribute identifier on the marker frame to the target object laser point. When the laser point is located outside the marker frame, the controller can determine that the laser point is a background laser point and assign a preset background identifier to the background laser point. The attribute identifier or preset background identifier can be, for example, a numerical value, or other forms; this application embodiment does not impose any limitations on this.

[0064] It should be noted that the comparison in this application embodiment is between the positional relationship between the laser point within a unit time period and the corresponding identifier box within that unit time period, that is, the time information (e.g., timestamp) between the two is matched.

[0065] S208: The controller corrects the distortion of the laser point's coordinates in the vehicle coordinate system at the time of acquisition based on the laser point's attributes and the vehicle's positioning data, and obtains the laser point's coordinates in the vehicle coordinate system at the time of image exposure.

[0066] As mentioned earlier, when the laser point belongs to the background laser point, its distortion originates from the movement of the vehicle itself; when the laser point belongs to the target object laser point, its distortion originates from the relative movement between the target object and the vehicle. The two cases will be described separately below.

[0067] In some possible implementations, when the laser point is the target object's laser point, the coordinates of the laser point in the vehicle coordinate system at the time of acquisition are distorted based on the laser point's attributes and the vehicle's positioning data. The coordinates of the laser point in the vehicle coordinate system at the time of image exposure can be obtained in the following way:

[0068] The controller first queries the target object detection data based on the target object attribute identifier of the laser point to obtain the first pose change of the target object in the vehicle coordinate system between the acquisition time and the image exposure time. The acquisition time and the image exposure time belong to the same time unit, and the first pose change represents the displacement and / or directional change of the target object relative to the vehicle during this time. Of course, when the target object is stationary relative to the vehicle, the first pose change is 0. Then, the controller uses the first pose change and the rotation parameters at the acquisition time to correct the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time, obtaining the laser point's coordinates in the world coordinate system at the image exposure time. Finally, the controller uses the laser point's coordinates in the world coordinate system at the image exposure time and the rotation parameters at the image exposure time to obtain the laser point's coordinates in the vehicle coordinate system at the image exposure time. In this way, distortion correction of the laser point can be performed based on the relative motion between the target object and the vehicle. Meanwhile, since this method directly utilizes the target object detection data based on the original point cloud for distortion correction, meaning that the converted data will include target object detection data and time information, there is no need to perform target object detection on the distorted laser point cloud, thereby improving the image annotation efficiency.

[0069] It should be noted that the rotation parameters can be determined based on the actual direction of the vehicle and the parameters of the environmental perception sensor. Therefore, the rotation parameters at the acquisition time and the rotation parameters at the image exposure time can be the same or different. This application embodiment does not impose any limitation on how to obtain the above rotation parameters.

[0070] In some possible implementations, when the laser point is a background laser point, the coordinates of the laser point in the vehicle coordinate system at the time of acquisition are distorted based on the laser point's attributes and the vehicle's positioning data. The coordinates of the laser point in the vehicle coordinate system at the time of image exposure can be obtained in the following way:

[0071] The controller first converts the coordinates of the laser point in the vehicle coordinate system at the time of acquisition to coordinates in the world coordinate system based on the rotation parameters at the acquisition time. Then, based on the vehicle positioning data, it obtains the second pose change of the vehicle in the world coordinate system between the acquisition time and the image exposure time. Finally, based on the coordinates of the laser point in the world coordinate system at the acquisition time, the second pose change, and the rotation parameters at the image exposure time, it obtains the coordinates of the laser point in the vehicle coordinate system at the image exposure time. In this way, the coordinate system transformation can be performed through rotation parameters, thereby combining the vehicle positioning data in the world coordinate system and the coordinates of the laser point acquired by the vehicle's LiDAR in the vehicle coordinate system. Together, they can correct the distortion of the laser point's coordinates based on the pose change of the vehicle between the acquisition time and the image exposure time, thus improving image accuracy.

[0072] Based on the above description, this application provides a laser point cloud distortion correction method. On the one hand, this method corrects the distortion of laser point cloud data collected per unit time, eliminating the problem that the coordinate system referenced by laser points collected at different times is different due to the relative motion of the vehicle and the surrounding environment, thus causing damage to the positional accuracy. On the other hand, this method directly utilizes the target object detection data based on the original point cloud, eliminating the need to perform target object detection on the distorted laser point cloud, thereby improving the image annotation efficiency.

[0073] Based on the foregoing, this application also provides a laser point cloud distortion correction device. The device of this application will be described in detail below with reference to the accompanying drawings.

[0074] See Figure 3 The diagram shown illustrates the structure of a laser point cloud distortion correction device 300, which may include:

[0075] The acquisition module 302 is used to acquire laser point cloud data, target detection data and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time and coordinates of all laser points acquired by the vehicle's lidar within a unit time period. The target detection data includes the size and motion information of the target within a unit time period. The vehicle positioning data includes the coordinates of the vehicle in the world coordinate system within a unit time period.

[0076] The identification module 304 is used to obtain the identification frame of the target object within a unit time period based on the size information and movement information of the target object within a unit time period;

[0077] The classification module 306 is used to determine the attributes of the laser points based on the data of the identification box and at least one laser point in the laser point cloud;

[0078] The distortion correction module 308 is used to correct the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle's positioning data, so as to obtain the laser point's coordinates in the vehicle coordinate system at the image exposure time. The image exposure time and the acquisition time belong to a unit time period.

[0079] In some possible implementations, the bounding box also includes target object attribute identifiers. When the classification module 306 determines the attribute of the laser point based on the data of the bounding box and at least one laser point in the laser point cloud, it is specifically used for:

[0080] When the laser point is located on or inside the marking frame, the laser point is identified as the target object laser point, and the target object laser point and the marking frame have the same target object attribute markings.

[0081] When the laser point is outside the marking frame, it is identified as a background laser point.

[0082] In some possible implementations, the laser point is the target object's laser point. The distortion correction module 308 corrects the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle's positioning data. Specifically, when obtaining the laser point's coordinates in the vehicle coordinate system at the image exposure time, it is used for:

[0083] Based on the target object attribute identifier of the laser point, query the target object detection data to obtain the first pose change of the target object corresponding to the laser point in the vehicle coordinate system between the acquisition time and the image exposure time.

[0084] Based on the first pose change and the rotation parameters at the acquisition time, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are distorted to obtain the coordinates of the laser point in the world coordinate system at the image exposure time.

[0085] Based on the coordinates of the laser point in the world coordinate system at the time of image exposure and the rotation parameters at the time of image exposure, the coordinates of the laser point in the vehicle coordinate system at the time of image exposure are obtained.

[0086] In some possible implementations, the laser point is a background laser point. The distortion correction module 308 corrects the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle's positioning data. Specifically, when obtaining the laser point's coordinates in the vehicle coordinate system at the image exposure time, it is used for:

[0087] Based on the rotation parameters at the acquisition time, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are converted to the coordinates in the world coordinate system.

[0088] Based on the vehicle's positioning data, the second pose change of the vehicle in the world coordinate system between the acquisition time and the image exposure time is obtained;

[0089] Based on the coordinates of the laser point in the world coordinate system at the acquisition time, the second pose change, and the rotation parameters at the image exposure time, the coordinates of the laser point in the vehicle coordinate system at the image exposure time are obtained.

[0090] Based on the aforementioned laser point cloud distortion correction method and apparatus, this application also provides a controller. This controller may be, for example, a vehicle control unit (VCU) or an electronic control unit (ECU). The controller includes a processor and a memory. The memory stores computer-readable instructions, and the processor executes these computer-readable instructions to perform the aforementioned laser point cloud distortion correction method. In some examples, the controller is used to implement the functions of the aforementioned laser point cloud distortion correction apparatus.

[0091] This application provides a computer-readable storage medium storing instructions that, when executed on a computer device, cause the computer device to perform the aforementioned laser point cloud distortion correction method.

[0092] This application provides a computer program product containing instructions that, when run on a computer device, causes the computer device to perform the aforementioned laser point cloud distortion correction method.

[0093] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0095] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0096] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A laser point cloud distortion correction method, characterized in that, The method includes: The system acquires laser point cloud data, target detection data, and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time and coordinates of all laser points collected by the vehicle's lidar within the unit time period. The target detection data includes the size and motion information of the target within the unit time period. The vehicle positioning data includes the coordinates of the vehicle in the world coordinate system within the unit time period. Based on the size and motion information of the target object within the unit time period, obtain the identification frame of the target object within the unit time period; The attributes of the laser point are determined based on the data of the identification box and at least one laser point in the laser point cloud; Based on the attributes of the laser point and the vehicle positioning data, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are distorted to obtain the coordinates of the laser point in the vehicle coordinate system at the image exposure time. The image exposure time and the acquisition time belong to the unit time period.

2. The method according to claim 1, characterized in that, The identification frame further includes a target object attribute identifier. Determining the attribute of the laser point based on the identification frame and data from at least one laser point in the laser point cloud includes: When the laser point is located on or within the marking frame, the laser point is determined to be the target object laser point, and the target object laser point has the same target object attribute identifier as the marking frame. When the laser point is located outside the marking frame, the laser point is determined to be a background laser point.

3. The method according to claim 2, characterized in that, The laser point is the target object laser point. The step of correcting the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle positioning data, to obtain the laser point's coordinates in the vehicle coordinate system at the image exposure time, includes: Based on the target object attribute identifier of the laser point, query the target object detection data to obtain the first pose change of the target object corresponding to the laser point in the vehicle coordinate system between the acquisition time and the image exposure time. Based on the first pose change and the rotation parameters at the acquisition time, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are distorted to obtain the coordinates of the laser point in the world coordinate system at the image exposure time. Based on the coordinates of the laser point in the world coordinate system at the time of image exposure and the rotation parameters at the time of image exposure, the coordinates of the laser point in the vehicle coordinate system at the time of image exposure are obtained.

4. The method according to claim 2, characterized in that, The laser point is a background laser point. The step of correcting the distortion of the laser point's coordinates in the vehicle coordinate system at the acquisition time based on the laser point's attributes and the vehicle positioning data, to obtain the laser point's coordinates in the vehicle coordinate system at the image exposure time, includes: Based on the rotation parameters at the acquisition time, the coordinates of the laser point in the vehicle coordinate system at the acquisition time are converted to the coordinates in the world coordinate system. Based on the vehicle positioning data, the second pose change of the vehicle in the world coordinate system between the acquisition time and the image exposure time is obtained; Based on the coordinates of the laser point in the world coordinate system at the acquisition time, the second pose change, and the rotation parameters at the image exposure time, the coordinates of the laser point in the vehicle coordinate system at the image exposure time are obtained.

5. The method according to claim 1, characterized in that, The vehicle positioning data is obtained based on the vehicle's magnetic declination, speed, and acceleration within the unit time period.

6. The method according to any one of claims 1 to 5, characterized in that, The motion information of the target object within a unit time period includes the target object's coordinates in the vehicle coordinate system, heading angle, and speed relative to the vehicle.

7. A laser point cloud distortion correction device, characterized in that, The device includes: The acquisition module is used to acquire laser point cloud data, target detection data, and vehicle positioning data within a unit time period. The laser point cloud data includes the acquisition time and coordinates of all laser points acquired by the vehicle's lidar within the unit time period. The target detection data includes the size and motion information of the target within the unit time period. The vehicle positioning data includes the coordinates of the vehicle in the world coordinate system within the unit time period. The identification module is used to obtain the identification frame of the target object within the unit time period based on the size information and movement information of the target object within the unit time period; The classification module is used to determine the attributes of the laser point based on the data of the identification box and at least one laser point in the laser point cloud; The distortion correction module is used to correct the distortion of the coordinates of the laser point in the vehicle coordinate system at the acquisition time based on the attributes of the laser point and the vehicle positioning data, so as to obtain the coordinates of the laser point in the vehicle coordinate system at the image exposure time, wherein the image exposure time and the acquisition time belong to the unit time period.

8. A controller, characterized in that, The vehicle controller includes: Memory is used to store computer programs or computer instructions; A processor for executing a computer program or computer instructions stored in the memory, causing the vehicle controller to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed, is used to implement the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 6.

Images