Method for object detection and corresponding vehicle

Abstract

Method for detecting objects (12), having the following features: - a three-dimensional point cloud (10) is captured using a lidar sensor (11), - for each object (12) among the objects (12), a corner point (13) of the object (12) that is closest to the sensor (11) is located in the point cloud (10) and certain properties of the object (12) are inferred from this corner point (13) and - the objects (12) are identified based on their respective corner points (13), characterized by the following features: - the vertices (13) are located in a grid of isotropic voxels and - the properties are regressed to the voxel of the grid to which the respective vertex (13) is assigned.

Description

[0001] The present invention relates to a method for recognizing objects. The present invention further relates to a corresponding vehicle, a corresponding computer program, and a corresponding machine-readable storage medium.

[0002] The detection of three-dimensional (3D) objects, particularly by advanced driver assistance systems (ADAS) and autonomous vehicles, is the subject of intensive research. One of the primary techniques used in this field is based on lidar, a remote sensing method that uses light in the form of pulsed laser beams to measure distances. A lidar sensor captures 3D point clouds, with each point characterized by its Cartesian spatial coordinates and the strength of its backscatter signal. Using these points, located and described by their reflectance, the goal is to identify objects—whether moving or stationary—within the sensor's range in real time.

[0003] Recently, artificial intelligence (AI), particularly in the form of deep neural networks, has proven to be a promising approach to solving this problem.

[0004] The VoxelNet architecture for lidar-based object detection is well-known to experts. It relies on decomposing three-dimensional point clouds into a grid of isotropic volume elements (called "voxels"), each with a discrete value—typically the reflectance. This so-called voxelization enables effective data reduction and abstraction for subsequent processing. The VoxelNet framework comprises two essential steps: feature extraction and object suggestion generation. Feature extraction utilizes three-dimensional convolutional neural networks (CNNs), which are responsible for extracting significant features from the voxels. Object suggestion generation employs two-dimensional convolutional neural networks to identify potential objects in the space under consideration and to suggest regions where these objects might be located.VoxelNet is trained on the basis of a large amount of lidar data in which the positions and classes of the objects were manually marked.

[0005] Since point clouds acquired using VoxelNet, for example, have a high spatial resolution, making it difficult to determine the exact boundaries of objects or their bounding boxes, center-based approaches to object detection have been proposed. These methods focus on locating the center of an object rather than outlining its entire perimeter. The voxel to which the located center is assigned is typically evaluated to infer the object's class—for example, a road user.

[0006] One example of this type of method is "CenterPoint," described in YIN, Tianwei; ZHOU, Xingyi; KRAHENBUHL, Philipp. Center-based 3D Object Detection and Tracking. In: Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2021. pp. 11784-11793.

[0007] DE 102018 127 865 A1 discloses the subject matter of the preamble of claim 1.

[0008] DE 10 2020 215 504 A1 relates to a computer-implemented method for detecting objects in the vicinity of a vehicle. A three-dimensional point cloud is captured by means of a radar sensor, and an object frame in the form of a rectangle is generated using a trained classifier. The position and orientation of this frame in a coordinate system are defined by a specific number of vertices.

[0009] KR 10 2023 0 072 584 A discloses a method for predicting the movement of a so-called measurement point on a vehicle using the Kalman filter algorithm. In this method, the vertex closest to the measurement point is selected, and the position values ​​of both the selected vertex and the measurement point are input into the Kalman filter. The resulting prediction includes the velocity, position, and acceleration of the measurement point. The selection process for the measurement point can also include selecting the vertex closest to a specific sensor.

[0010] EP3644276A1 discloses a method for fitting a three-dimensional bounding box around any point cloud cluster representing an object. The process begins by estimating the center of the point cloud cluster as the center of the 3D bounding box. Next, the extreme positions of the bounding box along the x, y, and z axes are determined. In the final step, the positions of eight 3D vertices are estimated and configured such that the bounding rectangle has the smallest possible volume and area, resulting in a tight 3D frame around the object.

[0011] EP4174611A1 discloses a mobile robot configured to use an object detection neural network (ODNN) on camera and lidar data to detect objects. The ODNN outputs the position of the bounding box of a detected object in several ways. One method provides the position as a combination of a central point and offsets in height, length, and width from the central point within the lidar sensor's coordinate system. Alternatively, the position can be output as a combination of the coordinates of the bounding box's corners within the same coordinate system.

[0012] US20190385457A1 discloses a system for identifying and analyzing obstacles. It extracts feature points from images from a camera or a three-dimensional image generated by a radar sensor. For example, if the obstacle is a truck, the system could identify and extract distinctive features such as corners or vertices. Furthermore, the processor can create a boundary field for the detected obstacle based on the three-dimensional image. This process involves identifying the obstacle by compressing the point cloud using MPEG-I standard technology, followed by generating a boundary frame that encloses it.

[0013] WO2019060125A1 discloses a CNN used to determine parameters from a 3D point cloud. To do this, an offset is calculated for each corner of a three-dimensional bounding box with respect to each point in the cloud. The algorithm also determines a confidence value for the offset of each point. The point and associated offset with the highest confidence value are then selected to define the smallest possible cuboid bounding box.

[0014] WO2021094398A1 discloses another CNN used for image processing tasks, particularly for the construction of enclosing objects. The model projects the corners of the predicted 3D frame onto an image plane and determines its minimum surrounding rectangle (MUR). The accuracy of the prediction is evaluated by minimizing the distance between this MUR and the original. The CNN accounts for consistency and geometry losses, as well as losses related to the position, orientation, and size of the object's center.

[0015] One problem is that, unlike a camera, for example, a lidar sensor often fails to capture information from the center of an object. This is because the lidar signal is primarily reflected on the surface of the object facing the sensor. However, object detection using the CenterPoint method relies precisely on its (inadequately detectable by lidar) center point. Thus, the prediction ultimately uses an anchor point that may contain little information about the object as a whole.

[0016] Furthermore, it should be considered that while the second stage of the CenterPoint model may be faster than other two-stage approaches, it still requires time and computing power to recognize an object's surfaces. This can be problematic in scenarios requiring real-time recognition and processing, such as autonomous driving.

[0017] Against this background, the object of the invention is to create a method for lidar-based object recognition which utilizes the available point cloud information with the greatest possible accuracy and efficiency, taking into account the specific properties of the lidar sensor used for its acquisition.

[0018] The described problem is solved by a method for detecting objects, a corresponding vehicle, a corresponding computer program and a corresponding machine-readable storage medium according to the independent claims.

[0019] This approach offers the advantage of using the information acquired by the lidar sensor more efficiently by representing objects based on their vertices rather than their centers. This enables more accurate and realistic object detection, as the lidar sensor often detects the vertices more precisely than the centers of the objects. In this way, a method according to the invention allows for the precise detection and classification of objects, which can be particularly beneficial in safety-critical applications such as autonomous driving.

[0020] Furthermore, the proposed method allows for improved object tracking. Because the information is processed directly from the objects' vertices rather than their centers, object tracking can be more efficient and accurate. This could contribute to improving the performance of object tracking systems, especially in dynamic and complex environments.

[0021] Furthermore, the proposed approach can contribute to reducing computational complexity and thus improving processing speed. Since the need for a second processing phase to refine the results, as required by the CenterPoint model, is eliminated, the described method enables faster and more efficient object recognition.

[0022] Ultimately, one embodiment of the invention offers greater flexibility in object recognition by being able to detect and process multiple vertices of an object, thus providing a more comprehensive and accurate representation of the object. This capability could contribute to improving the occupancy structure of the representational space, which also benefits the efficiency and accuracy of object recognition.

[0023] Further advantageous embodiments of the invention are specified in the dependent patent claims.

[0024] The figure shows the two-dimensional map view of a point cloud, which is captured by the lidar sensor of a vehicle turning left at an intersection.

[0025] An embodiment will now be explained using an exemplary scene captured by a lidar sensor (11) in the form of a spatial point cloud (10). This point cloud (10) represents all stationary and moving objects within the range of the lidar sensor (11) and is represented – for example, within the framework of VoxelNet – by a grid of voxels, each of which carries information about a specific area in the real world obtained from the lidar impulse response.

[0026] To enable effective object recognition, this three-dimensional point cloud (10) is first transformed into a top-down view. This can be achieved by projecting the points in the point cloud (10) onto a two-dimensional plane that corresponds to the height of the ground or the road surface.

[0027] In this top view, several objects (12) are visible, which have been detected by the lidar sensor (11). For each of these objects (12), the vertex (13) closest to the lidar sensor (11) is determined. The laser signals emitted by the lidar sensor (11) must travel the shortest distance to the respective object (12) at the aforementioned vertex (13) so that this point provides the most reliable information about the object (12).

[0028] Once the relevant vertex (13) for an object (12) has been located, its properties are regressed onto its associated voxel. In this way, the object's height, width, length, and orientation relative to the lidar sensor (11) can be derived to gain a comprehensive understanding of each object and to effectively detect and track it. It is understood that further vertices of the objects (12) may be located and included in the analysis to further improve the accuracy of object detection and tracking.

[0029] Finally, each object (12) can be assigned to a specific class of road users based on its derived properties. This classification can help predict the behavior of the detected objects and thus improve the safety and efficiency of autonomous driving. Reference symbol list 10 point cloud 11 Lidar sensor (of the Ego vehicle) 12 objects 13 Corner point

Claims

[1] Method for detecting objects (12), having the following features: - a three-dimensional point cloud (10) is captured using a lidar sensor (11), - for each object (12) among the objects (12), a corner point (13) of the object (12) that is closest to the sensor (11) is located in the point cloud (10) and certain properties of the object (12) are inferred from this corner point (13) and - the objects (12) are recognized based on their respective corner points (13), characterized by the following characteristics: - the vertices (13) are located in a grid of isotropic voxels and - the properties are regressed to the voxel of the grid to which the respective vertex (13) is assigned. [2] Method according to claim 1, characterized by the following characteristics: - the point cloud (10) is first converted into a top view after it has been captured and - the corner points (13) are located using the top view. [3] Method according to claim 1 or 2, characterized by at least one of the following characteristics: - the properties include dimensions of the respective object (12) or - the properties include an orientation of the respective object (12). [4] Method according to any one of claims 1 to 3, characterized by the following characteristics: - at least one further corner point (13) of the respective object (12) is located and - the regression is performed on all voxels to which at least one of the vertices (13) is assigned. [5] Method according to any one of claims 1 to 4, characterized by the following characteristic: - the respective object (12) is assigned to a specific class of road users based on its properties. [6] vehicle, characterized by the following characteristics: - the vehicle has a lidar sensor (11) and - the vehicle is equipped to carry out a method according to any one of claims 1 to 5. [7] Computer program which is configured to perform all steps of a method according to any one of claims 1 to 5. [8] Machine-readable storage medium with a computer program stored thereon according to claim 7.

Images