Method for training an AI of a LIDAR sensor and LIDAR sensor
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- DR ING H C F PORSCHE AG
- Filing Date
- 2024-03-27
- Publication Date
- 2026-07-30
AI Technical Summary
There is a need for low-cost LIDAR sensors capable of detecting obstacles in autonomous driving vehicles, focusing on distance and relative speed rather than precise 3D shape, to prevent collisions effectively.
A voxel-based training method for AI in LIDAR sensors, utilizing self-supervised learning without human intervention, where voxels are used to quickly and economically train the AI to detect obstacles, followed by further training with pixel-based methods.
This approach enables a cost-effective LIDAR sensor for autonomous vehicles by efficiently training the AI to detect obstacles, minimizing human oversight and enhancing scanning efficiency.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for training an AI of a LIDAR sensor and a corresponding LIDAR sensor with which obstacles in front of an autonomously driving motor vehicle can be automatically detected.
[0002] EP 3 745 158 A1 discloses a vehicle with a LIDAR sensor whose AI is trained with voxel data of an environment calculated based on a LIDAR point cloud.
[0003] There is a constant need for cost-effective LIDAR sensors in the automotive sector.
[0004] It is an object of the invention to provide a cost-effective LIDAR sensor.
[0005] The object is achieved by a method according to the features of claim 1 and a LIDAR sensor according to the features of claim 10. Preferred embodiments of the invention are specified in the dependent claims and the following description, which may each individually or in combination represent an aspect of the invention, the scope of protection being determined by the claims.
[0006] One aspect of the invention relates to a method for training an AI of a LIDAR sensor, wherein the AI of the LIDAR sensor is intended for use in autonomous driving of a motor vehicle, the AI is designed to detect obstacles in a direction of travel of the motor vehicle based on the data provided by the LIDAR sensor, wherein a 3D detection of the obstacles of the AI is trained or at least pre-trained using voxel-based data, wherein the voxel-based data is provided by scanning an environment using a scanning unit of the LIDAR sensor that provides a plurality of detection points within a predefined 3D grid, and converting the plurality of detection points to voxels before the voxels are input into the AI for the voxel-based training, wherein the voxel-based training comprises self-monitoring performed on the basis of the detection points.
[0007] The invention is based, among other things, on the realization that in the technical field of autonomously driving motor vehicles it is more important to detect the distance and / or the relative speed of an obstacle than the exact three-dimensional shape of the obstacle. The main task is to prevent a collision between the motor vehicle and the obstacle, whereby the area of the motor vehicle in which the collision would occur and the exact size of the obstacle are of lesser importance. Since, for safety reasons, it should also be prevented that the motor vehicle drives very close to the obstacle, the exact shape of the surface of the obstacle is of lesser importance due to the intended safety distance when driving past the obstacle. It is therefore sufficient to at least pre-train the AI based on voxels rather than based on a pixel cloud.Instead of representing three-dimensional objects using polygon lines within a pixel cloud, the AI can be trained based on voxels within the scanned 3D space. The multitude of voxels, especially block-like ones, can be processed more easily and with less effort, so that the voxel-based (pre-)training of the AI can be carried out faster and more cost-effectively. It is even possible to dispense with human supervision during voxel-based training, as the machine learning process can be carried out in self-monitored mode without human intervention by using the pixel information of the detection points already available after the environment has been scanned by the LIDAR sensor's scanning unit. Self-supervised voxel-based (pre-)training of the AI makes it possible to provide a cost-effective LIDAR sensor for an autonomously driving vehicle.
[0008] The AI ("artificial intelligence") is provided by a neural network. Object detection is a common field for the use of AI that analyzes images or other data. The AI backbone, in particular, is trained using voxel-based (pre-)training, which creates a good foundation for further training to refine obstacle detection for specific application areas.
[0009] A LIDAR (Light Imaging, Detection, and Ranging) sensor is capable of scanning a 3D space and detecting the distance to a point where the emitted (laser) light is reflected. Specifically, the LIDAR sensor has two or more respective light sources for emitting a light beam, allowing the specific scanned point within the 3D space to be determined. "Light" in this context refers to emitted electromagnetic waves with a specific wavelength, which may or may not be visible light or laser light, but can also be within the bandwidth of radar waves, infrared light, or ultraviolet light.
[0010] The obstacles are three-dimensional objects that could collide with the vehicle. In most cases, it is sufficient to scan the road in front of the vehicle using the LIDAR sensor. The road and / or road edges can usually be easily detected by the LIDAR sensor's AI. It is possible to limit the LIDAR sensor's scanning to the roadway for obstacle detection. However, certain situations, such as crossing pedestrians, road intersections, or the like, can also be detected in which the LIDAR sensors scan a larger area than just the road, so that an area near the edges of the road outside the road is also scanned for obstacles.
[0011] The 3D space scanned by the LIDAR sensor can comprise a plurality of detection points, the distance between which is measured by the LIDAR sensors. This plurality of detection points is also referred to as a LIDAR point cloud. Additionally, the 3D space is divided into a plurality of voxels. For example, the 3D space is divided into cubes or cuboids arranged by means of a 3D grid along the axis of a Cartesian coordinate system, with each cube or cuboid defining a voxel of the 3D space. In a preferred embodiment, a plurality of voxels, in particular each voxel, comprises the same number of detection points, with each voxel being assigned a single detection point within the volume of the voxel.
[0012] Voxel-based (pre-)training is a machine learning algorithm in which the AI of the LIDAR sensor is trained to detect obstacles. A suitable voxel-based machine learning algorithm is disclosed in EP 3 745 158 A1 and US 10 970 518 B1, the contents of which are hereby incorporated by reference as part of the invention, but modified such that the AI is not monitored by a human, but rather self-monitored based on the detection points provided by the scanning unit of the LIDAR sensor. During voxel-based (pre-)training, the AI should assess whether a particular voxel is occupied by part of an obstacle or not, with the AI's assessment being monitored and verified. Voxel-based training can be the only machine learning algorithm used to train the AI of the LIDAR sensor.Preferably, the voxel-based training can be only one of several machine learning algorithms applied to train the AI of the LIDAR sensor, wherein the voxel-based training is not the last, but one of the previously applied machine learning algorithms, in particular the first machine learning algorithm applied. In this case, the voxel-based training is voxel-based pre-training. The term "voxel-based training" is intended to encompass the case in which the voxel-based training is the only machine learning algorithm applied, the case in which the voxel-based training is one of several machine learning algorithms applied, and the case in which the voxel-based training is the first of several machine learning algorithms applied.
[0013] The plurality of neighboring voxels classified as occupied define a three-dimensional surface of an obstacle pointing toward the motor vehicle and the motor vehicle's LIDAR sensor. Based on this information, and in particular additional information such as the speed and direction of the motor vehicle and the obstacle, it is possible to calculate whether a collision between the motor vehicle and the obstacle is likely to occur and whether a driver assistance system should intervene.
[0014] In particular, it is intended that a classification of a specific voxel by the AI as occupied is considered true if the corresponding specific detection point within that voxel is also classified as occupied. If a specific detection point within the LIDAR pixel cloud is classified as occupied due to sufficient opacity, this information can be automatically used as a reference for an AI assessment for the corresponding voxel. Human supervision can be dispensed with.
[0015] Preferably, the AI's classification of a particular voxel as occupied is considered incorrect if the corresponding specific detection point within that voxel is classified as unoccupied. It is possible not only to confirm the AI's assessment regarding a particular voxel, but also to correct it. Thus, the AI learns not only from correct decisions, but also from incorrect decisions.
[0016] Particularly preferably, a classification of a specific voxel by the AI as unoccupied is assessed as true if the corresponding specific detection point within this voxel is also classified as unoccupied, and / or a classification of a specific voxel by the AI as unoccupied is assessed as false if the corresponding specific detection point within this voxel is classified as occupied. Even the AI's decisions in which the AI has not detected an obstacle can be monitored and used for training the AI, with the AI learning, in particular, from both correct and incorrect decisions.
[0017] In particular, one or more voxels located in front of an occupied voxel along a detection beam of the scanning unit are set to an unoccupied reference during voxel-based training. Since the scanning beam can only reach an occupied voxel with sufficiently high opacity if the voxels in front of the occupied voxel are primarily transparent, it is clear that every voxel in front of the occupied voxel must be unoccupied. The correct decision by the AI to classify these voxels as unoccupied can be made without comparing the voxel's classification as occupied or unoccupied with the classification of the corresponding detection point.
[0018] This makes it possible to avoid sampling detection points within the corresponding voxels before a voxel already known to be occupied. Sampling using the LIDAR sensor's sampling unit can be performed very quickly by focusing the sampling on the most important areas without providing false information about unsampled voxels.
[0019] Preferably, voxels located along a detection beam of the scanning unit where no detection point is present are set to an unoccupied reference during voxel-based training. If the scanning beam can reach the farthest detection point, it is clear that no voxel occupied by an obstacle can be present in the beam, so each additional detection point and / or each voxel within the beam can be set to unoccupied without having to perform a separate scan of the additional detection point within the beam. Sampling by the LIDAR sensor's scanning unit can again be performed very quickly by focusing the scan on the most important areas without providing false information about unsampled voxels.
[0020] Particularly preferred are voxels located behind an occupied voxel along a detection beam of the scanning unit, which are neglected during voxel-based training. Since the voxels occupied by the obstacle block the scanning of the area behind these occupied voxels, no unambiguous information can be received. These voxels are not used for voxel-based (pre-)training, thereby preventing unnecessary, arbitrary, unconfirmed results during training. In particular, it is provided that if no further detection point can be identified behind a specific detection point along the detection beam of the scanning unit, the respective detection point is classified as occupied and / or neglected during voxel-based (pre-)training. The data on the missed detection points can also be processed more simply and quickly.
[0021] In particular, further training of the AI takes place after voxel-based training, in particular using a training method using a pixel-based 3D space and taking into account a binary cross-entropy between neighboring pixels and / or using a training method using additional labeling or scene augmentation of a 3D space. The combination of different machine learning algorithms leads to better AI quality. In particular, voxel-based pre-training provides a well-trained backbone of the AI's neural network, which simplifies the subsequent machine learning algorithm(s).
[0022] Preferably, further training takes place under human supervision. Due to the well-trained backbone of the AI's neural network provided by voxel-based pretraining, human supervision can be focused on cases where automated decision-making would be difficult and error-prone. The extent of human supervision can be minimized, resulting in cost-effective machine learning with high reliability.
[0023] Another aspect relates to a LIDAR sensor for an autonomously driven motor vehicle, comprising a scanning unit for scanning a 3D space and an AI for identifying obstacles within the 3D space, wherein the AI is trained using the method as described above. The LIDAR sensor can be designed as described above with respect to the method. The self-supervised voxel-based (pre-)training of the AI makes it possible to provide a cost-effective LIDAR sensor for an autonomously driven motor vehicle.
[0024] The invention will now be described by way of example with reference to the accompanying drawings, which illustrate preferred embodiments. The features set forth below may constitute an aspect of the invention, both individually and in combination, with the scope of protection being determined by the claims. It shows: Fig.1 a schematic diagram of a LIDAR sensor according to the invention, which was trained by means of the method according to the invention.
[0025] The LIDAR sensor 10 can be used for autonomous driving of a motor vehicle. The LIDAR sensor 10 comprises a scanning unit 12, which includes at least one scanner 14, preferably at least two scanners 14, for scanning a plurality of detection points 16 of a LIDAR pixel cloud. Using the scanner(s) 14, the distance of each detection point 16 within a scanned 3D space can be measured. Furthermore, it can be detected whether a specific scanned detection point 16 is occupied by an obstacle, for example, by detecting the opacity of this specific detection point 16.
[0026] The scanned 3D space is divided into a plurality of voxels 18. When a camera 20 of the LIDAR sensor captures an image of the 3D space, the image can be analyzed using an AI 22 of the LIDAR sensor and other data available in the LIDAR sensor. The AI decides whether a specific voxel 18 of the 3D space is occupied by an obstacle. During (pre-)training of the AI, this decision is automatically self-monitored by classifying the specific detection point 16 within the corresponding voxel 18 as occupied or unoccupied, without the need for human intervention. Voxels 18 located behind an occupied voxel 18 that cannot be reached by a beam of the scanner 14 are neglected during the voxel-based (pre-)training. QUOTES CONTAINED IN THE DESCRIPTION
[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature
[0000] EP 3 745 158 A1 [0002, 0012] US 10 970 518 B1
[0012]
Claims
[1] Method for training an AI (22) of a LIDAR sensor (10), wherein the AI (22) of the LIDAR sensor (10) is intended for use in autonomous driving of a motor vehicle, the AI (22) is designed to detect obstacles in a direction of travel of the motor vehicle based on the data provided by the LIDAR sensor (10), where a 3D detection of the obstacles by the AI (22) is trained or at least pre-trained using voxel-based data, wherein the voxel-based data is provided by scanning an environment by means of a scanning unit (12) of the LIDAR sensor (10) providing a plurality of detection points (16) within a predefined 3D grid, and converting the plurality of detection points (16) to voxels (18) before the voxels (18) are input to the AI (22) for voxel-based training, wherein the voxel-based training comprises self-monitoring performed on the basis of the detection points (16). [2] Method according to claim 1, wherein a classification of a specific voxel (18) by the AI (22) as occupied is judged to be true if the corresponding specific detection point (16) within this voxel (18) is also classified as occupied. [3] Method according to claim 1 or 2, wherein a classification of a specific voxel (18) by the AI (22) as occupied is judged to be incorrect if the corresponding specific detection point (16) within this voxel (18) is classified as unoccupied. [4] Method according to one of claims 1 to 3, wherein a classification of a specific voxel (18) by the AI (22) as not occupied is assessed as true if the corresponding specific detection point (16) within this voxel (18) is also classified as not occupied, and / or a classification of a specific voxel (18) by the AI (22) as not occupied is assessed as false if the corresponding specific detection point (16) within this voxel (18) is classified as occupied. [5] Method according to one of claims 1 to 4, wherein one or more voxels (18) located in front of an occupied voxel (18) along a detection beam of the scanning unit (12) are set to an unoccupied reference during the voxel-based training. [6] Method according to one of claims 1 to 5, wherein voxels (18) located along a detection beam of the scanning unit (12) where no occupied detection point (16) is present are set to an unoccupied reference during the voxel-based training. [7] Method according to one of claims 1 to 6, wherein voxels (18) located behind an occupied voxel (18) along a detection beam of the scanning unit (12) are neglected in the voxel-based training. [8] Method according to one of claims 1 to 7, wherein after the voxel-based training, further training of the AI (22) takes place, in particular by means of a training method using a pixel-based 3D space and taking into account a binary cross entropy between neighboring pixels and / or by means of a training method using an additional labeling or scene extension of a 3D space. [9] A method according to claim 8, wherein the further training is carried out under human supervision. [10] LIDAR sensor (10) for an autonomously drivable motor vehicle, comprising a scanning unit (12) for scanning a 3D space and an AI (22) for identifying obstacles within the 3D space, wherein the AI (22) is trained by means of the method according to one of claims 1 to 9.