Method for capturing a road surface and sensing system

Abstract

A method for capturing a road surface of a roadway to be traversed by an autonomous vehicle using a sensing system (10). Wherein at least one image of the roadway is captured S1 for geometric scene re

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the field of automobiles. More specifically, the present invention relates to a method for capturing a road surface according to claim 1. Furthermore, the present invention relates to a sensing system for operating the method, a corresponding computer program product and a corresponding non-transitory computer-readable storage medium. BACKGROUND INFORMATION

[0002] It is important to ensure that roads are drivable for autonomous vehicles, for example to prevent collisions or damage. SUMMARY OF THE INVENTION

[0003] The objective of the invention is to develop a method through which autonomous driving becomes safer.

[0004] This objective is accomplished through a method with the features of claim 1, by a sensing system according to the invention, by utilizing a corresponding computer program product, as well as by a corresponding non-transitory computer-readable storage medium. Advantages embodiments of the invention can be found in the dependent claims.

[0005] The invention relates to a method for capturing a road surface of a roadway to be traversed by an autonomous vehicle using a sensing system. In other words, the invention relates to a method for capturing precise information about the road surface on which an autonomous vehicle is expected to travel. This method relies on a sensing system, primarily employing passive sensing techniques, such as camera sensors, to achieve its objectives. It is envisaged that the method for capturing the road surface uses at least one image of the roadway, which is captured for geometric scene reconstruction by at least one optical sensing device of the sensing system. Furthermore, at least one estimation of the ground plane is performed based on the at least one image using an electronic computing device coupled to the sensing device. The method also comprises an underdrivability estimation of the road surface, which is calculated based on the estimation of the ground plane. Therefore, the method is structured into several steps, each of which plays its role in achieving a comprehensive understanding of the road environment and ensuring the safety of the autonomous vehicle.

[0006] In a first step of this method a scene is reconstructed. Geometric scene reconstruction is utilized to create a detailed representation of the road and its surroundings. This involves capturing at least one image of the road using optical sensing devices. These images can then be analyzed by using a neural-network-based approach for example, which is capable of estimating the depth of projected image pixels. The neural network also performs the task of generating a 3D point cloud of detection. This 3D point cloud is a representation of the object in the scene. This step is beneficial for its robustness in low-light scenarios. It can effectively adjust light settings in challenging environments, such as tunnels and parking structures, where traditional sensing methods may struggle due to low light conditions.

[0007] In a second step, a ground plane estimation takes places. This involves further analysis of the captured images, specifically using semantic segmentation. Semantic segmentation is employed to identify and classify objects and elements within the camera images. Recognizing the road surface plays a vital role in assessing whether the road is drivable.

[0008] After that, in a third step, a mapping may be performed. The information obtained from the semantic segmentation is mapped to a 3D point cloud generated in the previous step. This process links the 2D image data with the 3D information, allowing for a comprehensive understanding of the road environment. The conversion of 2D data into 3D data can be accomplished through various techniques, such as the use of stereo cameras, LiDAR scanners, or computer vision algorithms. These technologies capture additional depth information, which is then integrated into the 2D data to create a 3D representation of the environment. This allows a more precise analysis and modeling of objects and structures and can also be achieved by using vectors to capture depth information and depict the spatial position of objects.

[0009] It is possible to use an interpolation in this final step, which is used to fine-tune the estimation of the ground plane. By interpolating data points, the system can achieve precise adjustment, ensuring that the ground plane representation is as accurate as possible.

[0010] The next step focuses on drivability estimation, a critical aspect of autonomous vehicle safety. Given the accurate representation, of the ground plane and the 3D point cloud of detections, the system performs a geometric check. This check is based on the dimensions of the autonomous vehicle. The system assesses whether there are any obstacles on the road that could potentially hinder the vehicle’s safe passage. This step for avoiding a collision or?? any other incident lowers the risk of harm or damage to the vehicle, passengers, or other road users.

[0011] An extension of the method involves the detection of signs and horizontal bars that provide information about upcoming vertical clearance. The system utilizes optical character recognition, OCR, to identify a numeric value relating to vertical clearance from these signs or bars. The numeric values are then compared with an estimated vertical clearance calculated based on the ground plane estimation. If the posted value falls within an acceptable threshold of the estimated clearance, it is considered safe. Otherwise, warnings or actions can be triggered to ensure the vehicle’s safe passage by creating a signal sent into the environment of the electronic computing device.

[0012] This method can find application in various real-world scenarios, such as parking structures, tunnels, bridges, road bridges, and for example areas with overhanging obstacles. It serves to signal the vehicle’s operator or autonomous driving system that the upcoming vertical clearance may be insufficient for safe passage. The applications of this method are diverse. It can provide immediate feedback to the vehicle’s operator signaling the need for adjustment. For instance, the system can recommend reducing the vehicle’s height by adapting air suspension or reducing tire pressure to ensure safe passage under low clearances. Additionally, the information obtained through this method can be sent to a mapping backend, allowing for the identification and flagging of roads with potential vertical clearance issues. This data can contribute to safer road planning for autonomous vehicles and improve overall road safety.

[0013] In summary, this method for capturing road surface information through a sensing system, uses passive optical sensors and advanced data processing techniques that enable enhancing the safety and efficiency of autonomous vehicles. Its ability to accurately estimate the ground plane, detect obstacles, and assess vertical clearance provides a valuable guidance for safe navigation in various road environments.

[0014] The invention also relates to a sensing system for capturing a road surface of a roadway to be traversed by an autonomous vehicle, comprising at least one optical sensing device for capturing at least one image of the roadway for geometric scene reconstruction, an electronic computing device coupled to the sensing device for performing estimation of the ground plane based on the at least one image and a further computing device for calculating the underdrivability estimation of the road surface depending on the estimated ground plane.

[0015] This sensing system can also comprise any components needed to perform the method for capturing the road surface of the roadway to be traversed by the autonomous vehicle using this sensing system.

[0016] Further advantages, features, and details of the invention derive from the following description of preferred embodiments as well as from the drawing. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figure and / or shown in the figure alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention. BRIEF DESCRIPTION OF THE DRAWING

[0017] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0018] The drawing shows in:

[0019] Fig. 1 various images for a comparison of scenes of road surface to illustrate the method according to the invention. DETAILED DESCRIPTION

[0020] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0021] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0022] The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0023] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0024] Fig. 1 shows in a first image X1 a scene displaying various road sections in an environment. Two heights, H1 and H2, are marked to illustrate the disparity between the image's simple estimate and the actual height of the road sections. A first road section A with height H1 is visible in the scene and is smaller than the other road section B with height H2. This demonstrates that in reality the road section A is lower than the road section B.

[0025] A second image X2 shows a more advanced method for height checking based on 3D scene reconstruction, as described in the following procedure. The invention relates to a procedure for capturing a road surface by for example interpolation 20. This procedure aims at accurately capturing the height of the road surface of a roadway to be traversed by an autonomous vehicle. It utilizes a sensing system 10 comprising at least one optical sensing device. The procedure includes the following steps:

[0026] In a first step S1 an image is captured, wherein at least one optical sensing device captures the images of the road on which the vehicle will travel.

[0027] In a second step S2 a geometric scene reconstruction is envisaged, wherein the captured images are used to create a 3D scene reconstruction. Another network is employed to estimate the depth of projected image pixels and to generate a 3D point cloud of detection.

[0028] In a third step S3 a ground plane estimation is envisaged, wherein based on the 3D scene reconstruction, the estimation of the ground plane is performed by the electronic computing device of the sensing system 10.

[0029] In a fourth step S4 an underdrivability assessment is envisaged, using the estimation of the ground plane, wherein the underdrivability of the road surface is calculated, taking into account the given vehicle dimensions.

[0030] In addition to precise height measurements through 3D scene reconstruction, interpolation 20 can be carried out in a fifth step S5. This interpolation 20 considers information from the 3D reconstruction and adjusts the ground plane accurately to obtain precise height estimates, wherein the Fig. 1 shows interpolated objects C1, C2 for better visual explanation.

[0031] Overall, images X1 and X2 in this comparison show the progression from a 2D height check to a more advanced method based on the precise 3D scene reconstruction and the option of interpolation 20 for even more accurate results. This advancement enhances the safety and efficiency of the autonomous vehicle in for example complex road environments. List of Reference Signs 10 sensing system 20 interpolation A road section B road section H1 height H2 height C1 object C2 object X1 image X2 image S1-S5 steps

Claims

1. A method for capturing a road surface of a roadway to be traversed by an autonomous vehicle using a sensing system (10), wherein at least one image of the roadway is captured for geometric scene reconstruction by at least one optical sensing device of the sensing system (10), and at least one estimation of the ground plane is performed based on the at least one image using an electronic computing device coupled to the sensing device, and wherein an underdrivability estimation of the road surface is calculated based on the estimation of the ground plane.

2. The method according to claim 1, characterized in thata neural network is applied for scene reconstruction.

3. The method according to claim 1 or 2, characterized in thata 3D space is created based on the at least one image for the calculation of the underlying road surface.

4. The method according to any one of the preceding claims, characterized in thattraffic signs and horizontal bars with information about the upcoming road surface are detected by a sensing device.

5. The method according to any one of the preceding claims, characterized in thatcalculated values are compared with digitally downloaded values by the electronic computing device.

6. Sensing system (10) for capturing a road surface of a roadway to be traversed byan autonomous vehicle, comprising:- at least one optical sensing device for capturing at least one image of the roadway for geometric scene reconstruction,- an electronic computing device coupled to the sensing device for performing estimation of the ground plane based on the at least one image, anda further computing device for calculating the underdrivability estimation of the road surface depending on the estimated ground plane.

7. A computer program product comprising program code means for performing a method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium comprising at least the computer program product according to claim 7.

Images