Dense LiDAR point cloud

Abstract

The present invention relates to a computer-implemented method for generating a dense LiDAR point cloud. A plurality of LiDAR point clouds consisting of a reference LiDAR point cloud and a residual LiDAR point cloud are received, wherein the plurality of LiDAR point clouds are obtained by measurements of a LiDAR device of a vehicle at subsequent measurement time points. The reference LiDAR point cloud and the residual LiDAR point cloud are combined to generate a dense LiDAR point cloud. Each residual LiDAR point cloud is transformed by correcting for motion of the LiDAR device in a time range between the measurement time points of the residual LiDAR point cloud and the measurement time points of the reference LiDAR point cloud. Only those points of the residual LiDAR point cloud that are within a predefined neighborhood around a point of the reference LiDAR point cloud are incorporated into the dense LiDAR point cloud. The dense LiDAR point cloud is further enhanced by including a plurality of additional points selected based on a statistical distribution around the points of the dense LiDAR point cloud.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for generating a dense Light Detection and Ranging (LiDAR) point cloud, a computer-implemented method for training an artificial neural network, a computer program product, a non-transitory computer-readable storage medium, an apparatus for generating a dense LiDAR point cloud, and a vehicle driver assistance system. Background Art

[0002] A LiDAR (Light Detection and Ranging) imaging system consists of a high-energy laser device that emits a laser beam. This laser beam is pulsed and reflected by an optical device to illuminate the scene. Furthermore, an optical detector is provided to detect the laser beam after it interacts with a target object in the surrounding environment. The emission of the laser pulse triggers a circuit that measures the laser beam's time of flight. Based on this measured laser beam flight time, the respective distances to target objects in the LiDAR system's surroundings can be determined.

[0003] To obtain a 3D LiDAR (laser radar) point cloud, distance information is generated for multiple target object points. Based on the LiDAR point cloud, information about the target object, such as its size and distance, can be determined. Further data analysis can determine the target object's type. This data can be used, for example, in driver assistance systems.

[0004] US 2018 / 341022 A1 relates to a LiDAR (laser radar)-based mapping method. First and second initial data frames are acquired, including LiDAR point cloud data scanned by a LiDAR and navigation data detected by an inertial measurement unit (IMU). Point features, line features, and surface features are extracted from the data frames according to a preset algorithm, and the data frames are logged based on the extracted features. The local coordinate system is then converted to a global coordinate system, and the map is updated.

[0005] EP 2 272 045 B1 relates to the logging of LiDAR (laser radar) point cloud data. It addresses situations where a target is partially obscured by another, possibly porous, object. The sensor can identify the portion of the target visible through the porous areas of the obstructing material. Data aggregation is achieved by collecting data from different sensor locations.

[0006] The article "Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite" published by Geiger et al. at the Conference on Computer Vision and Pattern Recognition (CVPR) in 2012 provides benchmarks for tasks such as stereo, optical flow, visual odometry / simultaneous localization and mapping (SLAM), and 3D object recognition on autonomous driving platforms.

[0007] The article “Vision meets Robotics: The KITTI Dataset” published by Geiger et al. in the International Journal of Robotics Research (IJRR) in 2013 provides a dataset for mobile robotics and autonomous driving research.

[0008] LiDAR (Light Detection and Ranging) data can be further analyzed using machine learning methods such as artificial neural networks. However, LiDAR sensors mounted on moving vehicles do not have sufficient access points for deep learning. Therefore, to obtain high-quality ground truth (GT), the number of points must be increased. As shown in the article "Sparsity Invariant Convolutional Neural Networks (CNNs)" by Uhrig et al., published on arXiv:1708.06500, traditional convolutional networks perform relatively poorly when applied to sparse data, even when the network is provided with the locations of missing data. We propose a sparse convolutional layer that explicitly takes the locations of missing data into account. Using the proposed setup, a dataset of 93K RGB images from the KITTI benchmark with depth annotations is obtained.

[0009] Subsequent LiDAR point clouds can be stacked on top of each other using a rigid body transformation (rotation and translation) defined by an inertial measurement unit (IMU) sensor, which also provides Global Positioning System (GPS) measurements. However, this type of point densification approach is problematic because moving objects are obscured by their LiDAR point clouds being logged at different points in time.

[0010] Another source of problems can also occur with static objects. Since points originate from different time instances and are pasted onto an intermediate frame, some objects may become transparent. For example, a vehicle may move behind a given object, with the LiDAR recording covered points. Consequently, densification on the intermediate frames may even make static objects transparent. To address this issue, a stereo camera can be mounted on the vehicle. Software-based stereo matching depth estimation can be implemented. The depth measured with the LiDAR can be compared with the estimated stereo depth. All matching points are retained, and the remaining points are ignored.

[0011] Stereo matching-based filtering of blurred dynamic and / or transparent objects is costly due to the need for an additional stereo camera setup. Furthermore, the stereo camera setup must be properly installed and calibrated, which is a potential source of error. Furthermore, stereo matching is less accurate than LiDAR (laser radar) measurements. Stereo matching depth is inherently quite incomplete. Furthermore, precise tuning of the algorithm parameters is difficult. It is also possible that useful LiDAR point clouds are discarded during the filtering process. Summary of the Invention

[0012] In view of the above situation, an object of the present invention is to provide an improved dense LiDAR (laser radar) point cloud that overcomes the above problems.

[0013] As described in the independent claims, the present invention provides a computer-implemented method for generating a dense LiDAR (laser radar) point cloud, a computer-implemented method for training an artificial neural network, a computer program product, a non-transitory computer-readable storage medium, an apparatus for generating a dense LiDAR (laser radar) point cloud, and a vehicle driver assistance system. The dependent claims recite various preferred features of the invention.

[0014] Thus, according to a first aspect, the present invention provides a computer-implemented method for generating a densified LiDAR point cloud. A plurality of LiDAR point clouds consisting of a reference LiDAR point cloud and a residual LiDAR point cloud are received, wherein the plurality of LiDAR point clouds are obtained based on measurements of a LiDAR device of a vehicle at subsequent measurement time points. A densified LiDAR point cloud is generated by combining the reference LiDAR point cloud and the residual LiDAR point clouds, wherein each residual LiDAR point cloud is transformed by correcting for motion of the LiDAR device in a time range between the measurement time points of the residual LiDAR point cloud and the measurement time points of the reference LiDAR point cloud, wherein only those points of the residual LiDAR point cloud that are within a predefined neighborhood around points of the reference LiDAR point cloud are combined into the densified LiDAR point cloud. The dense LiDAR point cloud is further enhanced by including a plurality of additional points selected based on a statistical distribution around the points of the dense LiDAR point cloud.

[0015] According to a second aspect, the present invention provides a computer-implemented method for training an artificial neural network. Output training data is provided by generating a plurality of dense LiDAR point clouds according to the method of any of the preceding claims. Input training data is provided by a reference LiDAR point cloud corresponding to the plurality of dense LiDAR point clouds. The artificial neural network is trained using the input training data as input and the output training data as output.

[0016] According to a third aspect, the present invention provides a computer program product comprising executable program code, the executable program code being configured to, when executed by a computing device, implement the method according to the first aspect or implement the method according to the second aspect.

[0017] According to a fourth aspect, the present invention provides a non-transitory computer-readable storage medium comprising executable program code, the executable program code being configured to, when executed by a computing device, implement the method according to the first aspect or implement the method according to the second aspect.

[0018] According to a fifth aspect, the present invention provides an apparatus for generating a dense LiDAR point cloud, comprising an interface configured to receive a plurality of LiDAR point clouds consisting of a reference LiDAR point cloud and a remaining LiDAR point cloud, wherein the plurality of LiDAR point clouds are obtained based on measurement results obtained by a LiDAR device of a vehicle at subsequent measurement time points; and further comprising an interface configured to generate a dense LiDAR point cloud by combining the reference LiDAR point cloud and the remaining LiDAR point cloud. A device for computing LiDAR point clouds, wherein the computing device is adapted to transform each of the remaining LiDAR point clouds by correcting for motion of the LiDAR device between the measurement time points of the remaining LiDAR point clouds and the measurement time points of the reference LiDAR point cloud, wherein the computing device is further configured to combine into the dense LiDAR point cloud only those points of the remaining LiDAR point clouds that are within a predefined neighborhood around the points of the reference LiDAR point cloud. The computing device is further configured to further enhance the dense LiDAR point cloud by including a plurality of additional points selected based on a statistical distribution around the points of the dense LiDAR point cloud.

[0019] According to a sixth aspect, the present invention provides a driver assistance system for a vehicle, comprising a LiDAR device configured to generate LiDAR measurement data; a device for generating a dense LiDAR point cloud according to the fifth aspect based on the LiDAR measurement data generated by the LiDAR device; and a control unit configured to control at least one function of the vehicle based on the generated dense LiDAR point cloud.

[0020] The present invention relates to generating a dense LiDAR point cloud by stitching (merging) subsequent LiDAR point clouds together. All LiDAR point clouds are merged into a common coordinate system. Any movement of the LiDAR device between different measurement times is corrected. Furthermore, points that are not in the neighborhood of points in the reference LiDAR point cloud are excluded or not included in the dense LiDAR point cloud.

[0021] By using this type of neighborhood filtering method, static target objects and moving target objects can be treated differently. For static target objects, basically all corresponding points of each corresponding LiDAR point cloud are included. In contrast, for moving target objects, points of the LiDAR point cloud whose measurement time points are far before and after the measurement time points of the reference LiDAR point cloud may be outside the neighborhood of the points of the reference point cloud. Therefore, these points are omitted and not included in the dense LiDAR point cloud. The points in the dense LiDAR point cloud (i.e., the target objects in the depth map) are no longer blurred or transparent.

[0022] The present invention does not require the use of a stereo camera, making it less expensive and easier to install. Furthermore, no camera calibration is required. Many useful points in the LiDAR point cloud are preserved. The depth values ​​in the dense LiDAR point cloud (depth map) are very accurate. Because no camera is required, neighborhood filtering can be performed on the entire LiDAR point cloud (e.g., 360 degrees) and is not limited to the windshield area.

[0023] According to another embodiment of the method for generating a densified LiDAR point cloud, the number of points in the densified LiDAR point cloud is at least 5 times higher than the number of points in the reference LiDAR point cloud, preferably at least 10 times higher. This factor depends on the number of LiDAR point clouds used in the method and on the number of additional points added according to the method.

[0024] According to another embodiment of the method for generating a densified LiDAR point cloud, the spatial extension of a predefined neighborhood around a point of a reference LiDAR point cloud used to determine whether to merge a point of the remaining LiDAR point cloud into the densified LiDAR point cloud depends on the difference between the measurement time of the remaining LiDAR point cloud and the measurement time of the reference LiDAR point cloud. For example, the larger the temporal distance between each respective remaining LiDAR point cloud and the reference LiDAR point cloud, the smaller the neighborhood.

[0025] According to another embodiment of the method for generating a dense LiDAR point cloud, the spatial expansion depends linearly on the difference between the measurement time of the remaining LiDAR point cloud and the measurement time of the reference LiDAR point cloud. For example, the radius c of the neighborhood around a point of the reference LiDAR point cloud can be given by the following formula:

[0026] c=17–3·d,

[0027] The measurements are in centimeters, where d gives the temporal distance relative to the frame of the reference LiDAR point cloud in the form of a timestamp. Thus, for example, a point in the remaining LiDAR point cloud that is at a temporal distance d=3 from the frame of the reference LiDAR point cloud has a neighborhood radius c=8 cm. This means that if there is a point in the remaining LiDAR point cloud that is within a neighborhood of 8 cm of any point in the reference LiDAR point cloud, said point will be retained and added to the densified LiDAR point cloud. In general, the neighborhood can be set manually and depends on the dataset, i.e., on multiple LiDAR point clouds.

[0028] According to another embodiment of the method for generating a dense LiDAR point cloud, the spatial extension of a predefined neighborhood around a point of a reference LiDAR point cloud depends on the depth of the point in the reference LiDAR point cloud. Here, the depth of a point is related to the distance from the LiDAR device. Errors at closer points are penalized more severely than errors at more distant points.

[0029] According to another embodiment of the method for generating a dense LiDAR point cloud, the spatial extension depends exponentially on the depth of the point of the reference LiDAR point cloud. For example, the spatial dependence of the radius of the neighborhood can be given by the following function:

[0030] exp(f(depth)),

[0031] Where f is a function that depends on the depth of a given point. Its specific form can be determined based on each corresponding data set, for example, f = depth / 10-1.

[0032] According to another embodiment of the method for generating a dense LiDAR point cloud, a plurality of additional points used to enhance the dense LiDAR point cloud are selected based on a Gaussian distribution around the points of the dense LiDAR point cloud. For example, the additional random points can be added to a small neighborhood of the dense LiDAR point cloud, such as a small neighborhood of a spherically symmetric Gaussian distribution with a radius of, for example, 1 to 2 centimeters.

[0033] According to another embodiment of the method for generating a dense LiDAR point cloud, the plurality of additional points used to enhance the dense LiDAR point cloud are selected based on a uniform distribution around the points of the dense LiDAR point cloud. However, the present invention is not limited to any particular statistical distribution.

[0034] According to another embodiment of the method for generating a dense LiDAR point cloud, the plurality of LiDAR point clouds includes 2N+1 LiDAR point clouds, wherein the measurement time points of the N LiDAR point clouds are before the measurement time points of the reference LiDAR point cloud, and the measurement time points of the N LiDAR point clouds are after the measurement time points of the reference LiDAR point cloud. In other words, with respect to the measurement time points, the reference LiDAR point cloud is the central LiDAR point cloud.

[0035] According to another embodiment of the method for generating a dense LiDAR point cloud, the step of correcting the motion of the LiDAR device between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud is performed using sensor data obtained from the vehicle's inertial measurement unit (IMU). Based on the sensor data from the inertial measurement unit (IMU), the motion of the vehicle and, from this, the motion of the LiDAR device can be calculated. The transformation (translation and rotation) of the remaining LiDAR point cloud is determined based on this calculated motion.

[0036] According to another embodiment of the method for generating a densified LiDAR point cloud, the step of correcting for the motion of the LiDAR device between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud is performed using an iterative closest point algorithm. The iterative closest point algorithm is used to map the LiDAR point clouds to each other.

[0037] According to another embodiment, the driver assistance system further comprises an inertial measurement unit (IMU) configured to generate data; wherein the device for generating a dense LiDAR point cloud is further configured to implement a motion correction step of the LiDAR device between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud using sensor data originating from the inertial measurement unit (IMU). BRIEF DESCRIPTION OF THE DRAWINGS

[0038] For a more complete understanding of the present invention and its advantages, in the following description, exemplary embodiments of the present invention are explained in more detail with reference to the accompanying drawings, in which like reference characters refer to like parts, and in which:

[0039] Figure 1 A block diagram is provided showing an apparatus for generating a dense LiDAR (laser radar) point cloud according to one embodiment of the present invention;

[0040] Figure 2 A vehicle driver assistance system according to an embodiment of the present invention is shown in the form of a block diagram;

[0041] Figure 3 A computer-implemented method for generating a dense LiDAR (laser radar) point cloud according to one embodiment of the present invention is presented in the form of a flowchart;

[0042] Figure 4 A computer-implemented method for training an artificial neural network according to one embodiment of the present invention is presented in the form of a flowchart;

[0043] Figure 5 A computer program product according to an embodiment of the present invention is shown in a block diagram form; and

[0044] Figure 6 A non-transitory computer-readable storage medium according to one embodiment of the present invention is shown in block diagram form. DETAILED DESCRIPTION

[0045] The accompanying drawings are provided to facilitate a better understanding of the present invention and are an integral part of this specification. They illustrate specific embodiments of the present invention and, together with the accompanying description, explain the principles of the invention. The present invention will be better understood by reference to the following detailed description, and further embodiments and numerous advantages associated therewith will become apparent.

[0046] It will be apparent that common and generally accepted parts that may be useful or necessary for a commercially feasible embodiment are not necessarily depicted in order to provide an abstract view of the embodiments. The parts in the drawings are not necessarily illustrated in exact proportion to one another. It will also be understood that in a method embodiment, specific actions and / or steps may be described or depicted in a particular order, while those skilled in the art will understand that such specificity with respect to order is not actually required. It will also be understood that the terms and expressions used in this specification have the ordinary meaning associated with their respective areas of investigation and study unless otherwise specified herein.

[0047] Figure 1 A schematic block diagram shows an apparatus 1 for generating a dense LiDAR point cloud. Apparatus 1 includes an interface 2, such as a wireless or wired connection, for receiving multiple LiDAR point clouds. LiDAR point clouds are generated by a vehicle's LiDAR device 5 at subsequent measurement points in time. The LiDAR point clouds are stored in a storage device 8 of apparatus 1.

[0048] The computing device 3 (e.g., a microprocessor, integrated circuit, or similar device) retrieves the stored LiDAR point cloud from the storage device 8 and divides the LiDAR point cloud into a reference LiDAR point cloud and remaining LiDAR point clouds. The number of remaining LiDAR point clouds can be a predefined number, such as an even number of 2, 4, 8, or 10. The reference LiDAR point cloud can be the center LiDAR point cloud of an odd number of LiDAR point clouds, relative to a time sequence.

[0049] The computing device 3 combines the reference LiDAR point cloud and the remaining LiDAR point cloud to generate a dense LiDAR point cloud.

[0050] Before combining the LiDAR point clouds, the computing device 3 transforms the remaining LiDAR point clouds by correcting for the motion of the LiDAR device 5 between the time of measurement of the remaining LiDAR point clouds and the time of measurement of the reference LiDAR point clouds. The correction can be based on data retrieved from the vehicle's inertial measurement unit 7 or on an iterative closest point algorithm.

[0051] The computing device 3 also implements a neighborhood filtering method. According to this neighborhood filtering method, only those points in the remaining LiDAR point cloud that are within any predefined neighborhood relative to each point of the reference LiDAR point cloud are merged into the dense LiDAR point cloud. The neighborhood can be a sphere or a cube with the point at its center.

[0052] The size of the neighborhood (eg, the radius of the sphere) may depend (eg, in a linear fashion) on the temporal distance between points in the remaining LiDAR point cloud and points in the reference LiDAR point cloud.

[0053] Additionally or alternatively, the size may also depend on the depth of the point in a reference LiDAR point cloud (eg to an index).

[0054] The generation of dense LiDAR point clouds can be performed offline. However, the generation of dense LiDAR point clouds can also be performed online, that is, during vehicle operation.

[0055] The computing device 3 may be further configured to further enhance the dense LiDAR point cloud by including a plurality of additional points selected based on a statistical distribution around the dense LiDAR point cloud point. The statistical distribution may be, for example, a Gaussian distribution or a uniform distribution.

[0056] Figure 2 A schematic block diagram of a driver assistance system 4 for a vehicle is shown. The driver assistance system 4 comprises a LiDAR device 5 which generates measurement data.

[0057] The driver assistance system 4 also includes Figure 1 The apparatus 1 for generating a dense LiDAR point cloud shown and described above is provided. The apparatus 1 generates a dense LiDAR point cloud based on measurement data generated by a LiDAR device 5. The apparatus 1 further includes a control unit 6 for controlling at least one vehicle function based on the generated dense LiDAR point cloud. The control unit 6 may, for example, provide at least one of the following functions: a parking assistance function, a lane keeping function, an emergency braking function, an overtaking assistance function, or the like. The control unit 6 may steer the vehicle or accelerate or decelerate the vehicle.

[0058] Figure 3 A computer-implemented method for generating a dense LiDAR (laser radar) point cloud is presented in the form of a flowchart. The method can be used Figure 1 The device 1 shown in FIG.

[0059] In a first method step S1 , the vehicle's LiDAR device 5 generates a plurality of LiDAR point clouds at respective measurement points in time.

[0060] In a second method step S2, the device 1 (e.g. Figure 1 The apparatus 1) shown receives a plurality of LiDAR point clouds divided into a reference LiDAR point cloud and a remaining LiDAR point cloud.

[0061] In a third method step S3, a dense LiDAR point cloud is generated by combining the reference LiDAR point cloud and the remaining LiDAR point cloud, wherein each remaining LiDAR point cloud is transformed by correcting for the motion of the LiDAR device 5 between the measurement time of the remaining LiDAR point cloud and the measurement time of the reference LiDAR point cloud, wherein only those points of the remaining LiDAR point cloud that are within a predefined neighborhood around a point of the reference LiDAR point cloud are combined into the dense LiDAR point cloud. The neighborhood may depend on the spatial position or the temporal position of the points of the reference LiDAR point cloud.

[0062] The size of the neighborhood (eg the radius of the sphere) may for example depend (eg in a linear fashion) on the temporal distance between points in the remaining LiDAR point cloud and points in the reference LiDAR point cloud.

[0063] Additionally or alternatively, the size may also depend (eg exponentially) on the depth of the point in a reference LiDAR point cloud.

[0064] In a fourth method step S4 , the dense LiDAR point cloud is enhanced by including a plurality of further points selected based on a statistical distribution (eg Gaussian distribution or uniform distribution) around the points of the dense LiDAR point cloud.

[0065] As described below, a method for generating dense LiDAR point clouds using a neighborhood filtering algorithm can be part of a data generation pipeline that generates input for a deep neural network.

[0066] Figure 4 A computer-implemented method for training an artificial neural network is presented in the form of a flowchart.

[0067] In the first step S11, according to Figure 3 The method described in the present invention provides output training data by generating multiple dense LiDAR (laser radar) point clouds. Here, the output training data refers to a set of ground truth for training artificial neural networks.

[0068] In a second step S12, input training data is provided, which is given by a reference LiDAR point cloud corresponding to a plurality of dense LiDAR point clouds.

[0069] In a third step S13, the artificial neural network is trained using the input training data as input and the output training data as output.

[0070] The ANN is trained in a supervised manner. For each training input (a reference LiDAR point cloud), a corresponding dense LiDAR point cloud is provided as the ground truth. The ANN is trained by generating a dense LiDAR point cloud (i.e., an enhanced depth map) from the sparse raw LiDAR point cloud as input.

[0071] The artificial neural network can be any known network such as a convolutional neural network (CNN).

[0072] Figure 5A schematic block diagram shows a computer program product P comprising an executable program code PC. The executable program code PC is configured to, when executed (e.g. by a computing device), implement the method for generating a dense LiDAR point cloud or the method for training an artificial neural network as described above.

[0073] Figure 6 A schematic block diagram of a non-transitory computer-readable storage medium M containing executable program code MC is shown, wherein the program code MC is configured to implement the method for generating a dense LiDAR (laser radar) point cloud or the method for training an artificial neural network as described above when the program code is executed (for example by a computing device).

[0074] Although specific embodiments of the present invention have been illustrated and described herein, those of ordinary skill in the art will appreciate that there are numerous alternative and / or equivalent embodiments. It should be understood that exemplary embodiments or exemplary embodiments are merely examples and are not intended to limit the scope, applicability, or configuration in any way. Rather, the foregoing summary and detailed description will provide those of skill in the art with a convenient roadmap for implementing at least one exemplary embodiment, wherein it is understood that various changes may be made to the features and devices described in the exemplary embodiments without departing from the scope of application set forth in the appended claims and their legal equivalents. In general, this patent application is intended to cover all adaptations or variations of the specific embodiments discussed herein.

[0075] As used herein, the terms "include," "comprising," "having," and their derivatives, should be understood as being inclusive (i.e., non-exclusive), such that the processes, methods, devices, apparatus, or systems described herein are not limited to those features or parts or elements or steps described, but may include other elements, features, parts, or steps not expressly listed or inherent to the process, method, article, or apparatus. In addition, the terms "one" and "an" as used herein should be understood as meaning one or more unless expressly stated otherwise. Furthermore, the terms "first," "second," "third," etc., are used merely as labels and do not impose numerical requirements on their objects or specify a certain order of importance.

[0076] Reference Mark List

[0077] 1 Device for generating dense LiDAR point clouds

[0078] 2 Interfaces

[0079] 3 Computing devices

[0080] 4 Driver Assistance Systems

[0081] 5. LiDAR (laser radar) device

[0082] 6 Control Unit

[0083] 7 Inertial Measurement Unit

[0084] 8 Storage devices

[0085] S1 to S4 method steps

[0086] S11 to S13 method steps

[0087] M Storage Media

[0088] MC program code

[0089] P Computer program product

[0090] PC program code

Claims

1. A computer-implemented method for generating a dense LiDAR point cloud, wherein: The method comprises the following steps: receiving ( S2 ) a plurality of LiDAR point clouds consisting of a reference LiDAR point cloud and a remaining LiDAR point cloud, wherein the plurality of LiDAR point clouds are obtained based on measurement results of a LiDAR device ( 5 ) of the vehicle at subsequent measurement time points; generating (S3) a dense LiDAR point cloud by combining the reference LiDAR point cloud and the residual LiDAR point cloud, wherein each residual LiDAR point cloud is transformed by correcting for motion of the LiDAR device (5) between the measurement time points of the residual LiDAR point cloud and the measurement time points of the reference LiDAR point cloud, wherein only those points of the residual LiDAR point cloud that are within a predefined neighborhood around points of the reference LiDAR point cloud are combined into the dense LiDAR point cloud; and The dense LiDAR point cloud is further enhanced (S4) by including a plurality of additional points selected based on a statistical distribution around the points of the dense LiDAR point cloud.

2. The method according to claim 1, wherein The spatial extension of the predefined neighborhood used to determine whether to incorporate points of the remaining LiDAR point cloud into the dense LiDAR point cloud depends on the difference between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud.

3. The method according to claim 2, wherein: The spatial expansion depends in a linear manner on the difference between the measurement time points of the remaining LiDAR point clouds and the measurement time points of the reference LiDAR point cloud.

4. The method according to claim 1, wherein The spatial extension of the predefined neighborhood around a point of the reference LiDAR point cloud depends on the depth of said point of the reference LiDAR point cloud.

5. The method according to claim 4, wherein The spatial extension depends exponentially on the depth of the point in the reference LiDAR point cloud.

6. The method according to claim 1, wherein A plurality of additional points for enhancing the dense LiDAR point cloud are selected based on a Gaussian distribution around a point of the dense LiDAR point cloud.

7. The method according to claim 1, wherein The multiple LiDAR point clouds include 2N+1 LiDAR point clouds, wherein measurement time points of the N LiDAR point clouds are before the measurement time point of the reference LiDAR point cloud, and measurement time points of the N LiDAR point clouds are after the measurement time point of the reference LiDAR point cloud.

8. The method according to claim 1, wherein The step of correcting the movement of the LiDAR device (5) between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud is performed using sensor data obtained from the vehicle's inertial measurement unit (7).

9. The method according to claim 1, wherein The step of correcting the motion of the LiDAR device (5) between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud is performed using an iterative closest point algorithm.

10. A computer-implemented method for training an artificial neural network, the method comprising the steps of: Providing (S11) output training data by generating a plurality of dense LiDAR point clouds according to the method of any one of the preceding claims; providing (S12) input training data given by a reference LiDAR point cloud corresponding to the plurality of dense LiDAR point clouds; as well as The artificial neural network is trained (S13) using the input training data as input and the output training data as output.

11. A computer program product (P) comprising an executable program code (PC) configured for, when executed by a computing device (3), carrying out the method according to any one of claims 1 to 10.

12. A non-transitory computer-readable storage medium (M) comprising an executable program code (MC) configured for, when executed by a computing device (3), implementing the method according to any one of claims 1 to 10.

13. A device (1) for generating a dense LiDAR point cloud, comprising: An interface (2) configured to receive a plurality of LiDAR point clouds consisting of a reference LiDAR point cloud and a remaining LiDAR point cloud, wherein the plurality of LiDAR point clouds are obtained based on measurement results of a LiDAR device (5) of the vehicle at subsequent measurement time points; and a computing device (3) configured to generate a dense LiDAR point cloud by combining the reference LiDAR point cloud and the residual LiDAR point cloud, wherein the computing device (3) is adapted to transform each residual LiDAR point cloud by correcting for a motion of the LiDAR device (5) between a measurement time point of the residual LiDAR point cloud and a measurement time point of the reference LiDAR point cloud, wherein the computing device (3) is further configured to combine into the dense LiDAR point cloud only those points of the residual LiDAR point cloud that are within a predefined neighborhood around points of the reference LiDAR point cloud; The computing device (3) is further configured to further enhance the dense LiDAR point cloud by including a plurality of other points selected based on a statistical distribution around the points of the dense LiDAR point cloud.

14. A driver assistance system (4) for a vehicle, comprising: a LiDAR device (5) configured to generate LiDAR measurement data, The device (1) for generating a dense LiDAR point cloud according to claim 13 based on LiDAR measurement data generated by a LiDAR device (5); as well as A control unit (6) is configured to control at least one function of the vehicle based on the generated dense LiDAR point cloud.

15. The driver assistance system (4) according to claim 14, further comprising an inertial measurement unit (7) configured to generate data; in, The apparatus (1) for generating a dense LiDAR point cloud is further configured to perform a step of correcting the movement of the LiDAR device (5) between the measurement time points of the remaining LiDAR point cloud and the measurement time points of the reference LiDAR point cloud using sensor data from an inertial measurement unit (7).

Images