METHOD AND DISPLAY DEVICE FOR VISUALIZING AN ARRANGEMENT AND OPERATION OF ENVIRONMENTAL SENSORS OF A MOTOR VEHICLE

Description

[0001] The invention relates to a method and a display device for visualizing the arrangement and operation of environmental sensors of a motor vehicle. Furthermore, the invention relates to a motor vehicle.

[0002] With the increasing use of autonomous driving capabilities in motor vehicles, it will become very important to explain how autonomous vehicles work and the resulting high level of reliability.

[0003] DE 10 2016 214 916 A1 describes a method for operating a motor vehicle in which the motor vehicle drives autonomously and in which a maneuver currently being performed or about to be performed by the motor vehicle is shown on a display of the motor vehicle. Sensory capabilities of the motor vehicle can be visibly displayed for a driver using a display.

[0004] DE 10 2013 021 853 A1 discloses a device and a method for operating a vehicle's autopilot. Data glasses arranged outside the vehicle are used to display the vehicle's sensor data if an error occurs during an autonomous driving maneuver.

[0005] DE 10 2011 121 948 A1 describes a method for providing a preview of actions of an autonomous driving system. A vehicle display serves to depict a planned autonomous driving maneuver of the vehicle.

[0006] The visualization of sensor operating information is known from DE 10 2016 210611 A1, US 2017 / 129426 A1, US 2017 / 076606 A1 and DE 10 2008 036009 A1.

[0007] DE 10 2014 214506 A1 shows that in a motor vehicle, the electromagnetic radar waves detected by a radar sensor can be visualized for a user in order to demonstrate the correct functioning of the radar sensor.

[0008] DE 10 2016 211 646 A1 deals with the idea that, with automatic longitudinal and lateral guidance, the vehicle suggests several possible driving maneuvers to the driver, from which the driver can select one and this selected driving maneuver is then carried out by the vehicle.

[0009] It is the object of the present invention to provide a solution by means of which the functioning of an at least partially autonomous vehicle can be visualized in a particularly easy-to-understand manner.

[0010] This object is achieved by a method for operating a motor vehicle and by a display device having the features of the independent patent claims. Advantageous embodiments with expedient and non-trivial refinements of the invention are specified in the dependent claims.

[0011] In the method according to the invention for operating a motor vehicle, during at least partially autonomous driving of the motor vehicle, the arrangement and mode of operation of the motor vehicle's environmental sensors is visualized by means of at least one display device arranged in the motor vehicle, the sensor data of which are used during at least partially autonomous driving. Partially autonomous driving means that the motor vehicle acts autonomously at least with regard to longitudinal or lateral guidance, without a driver having to intervene in this regard. However, it is equally possible with the method according to the invention for the arrangement and mode of operation of the motor vehicle's environmental sensors to be visualized by means of the display device arranged in the motor vehicle during fully autonomous driving of the motor vehicle, i.e. during driving in which a driver intervenes neither in the longitudinal nor in the lateral guidance.

[0012] The invention is based on the realization that, with the increasing use of autonomous driving, it will become increasingly important to explain to customers, employees, and even public figures such as journalists how autonomous vehicles work and the resulting high level of reliability. By using the display device arranged in the motor vehicle to visualize both the arrangement and the mode of operation of the motor vehicle's environmental sensors, whose sensor data is used in at least partially autonomous driving, a particularly high level of acceptance for partially autonomous or even fully autonomous driving can be created among the vehicle occupants. A spatially correct arrangement of the vehicle's environmental sensors is thus visualized, allowing a vehicle occupant to recognize where the vehicle's environmental sensors are located.Since the operation of these environmental sensors is also visualized, the vehicle occupant can easily see what the vehicle is detecting in its surroundings. This also makes it possible to use the display device to convey how the vehicle processes the sensor data during at least partially autonomous driving or even fully autonomous driving and translates it into partially or fully autonomous driving movements. For this purpose, permanent communication or data exchange is provided between the respective display device and the vehicle, for example via a suitable interface, e.g., in the form of Bluetooth or similar.

[0013] An advantageous embodiment of the invention provides that the display device is augmented reality glasses, an augmented reality contact lens, virtual reality glasses, or a display device of the motor vehicle, in particular a contact-analog display device. Using augmented reality glasses, the vehicle occupant can continue to perceive their surroundings, with information on the arrangement and operation of the environmental sensors being simply displayed. If the vehicle occupant looks forward while wearing the augmented reality glasses, for example, the augmented reality glasses display a sensor arranged in the front area of ​​the vehicle as an icon or similar. In addition, the augmented reality glasses can also visualize the operation of the respective sensor by appropriately overlaying reality with virtual content.If the vehicle occupant turns their head to the left while wearing augmented reality glasses, they will see an icon of the same kind, indicating a sensor and its position, which is located on the side of the vehicle to monitor the vehicle's surroundings. The same can also be achieved with an augmented reality contact lens or a contact-analog display device. The contact-analog display device can be, for example, a head-up display or display devices integrated into the vehicle's windshield.Using augmented reality glasses, augmented reality contact lenses, and contact-analog display devices, it is possible for the vehicle occupant to still perceive their real surroundings. These display devices can be used to visualize the arrangement and operation of the vehicle's environmental sensors as a superimposed view of reality. It is also possible for the vehicle occupant, especially if they are a passenger and not the driver, to wear virtual reality glasses, which can be used to visualize the arrangement and operation of the vehicle's environmental sensors.

[0014] A further advantageous embodiment of the invention provides that the display device indicates the respective positioning of the environmental sensors on the motor vehicle and visualizes their respective detection ranges. A vehicle occupant can therefore easily identify where the respective environmental sensors of the motor vehicle are located and what detection range they can actually cover. Particularly when the detection ranges are relatively large, this can contribute to the vehicle occupant being particularly reassured when the motor vehicle is driving semi-autonomously or autonomously, since the vehicle occupant is shown how much of its surroundings the motor vehicle can actually detect using sensors.

[0015] According to a further advantageous embodiment of the invention, it is provided that a respective sensor operating principle of the environmental sensors is visualized by means of the display device. This allows the vehicle occupant to easily recognize whether the respective environmental sensors are, for example, cameras, lidar sensors, radar sensors, infrared sensors, and the like. This can be done, for example, by displaying suitable symbols corresponding to the respective sensor operating principle at the location where the respective environmental sensor is actually located. This makes it particularly easy for the vehicle occupant to understand which sensor operating principle underlies the respective environmental sensors, which can also contribute to the vehicle occupant's particularly good acceptance of the semi-autonomous or fully autonomous driving operation of the motor vehicle.

[0016] In a further advantageous embodiment of the invention, if at least one of the environmental sensors is a radar sensor, the respective radar waves are visualized by means of the display device. In particular, it is also possible for the radar waves reflected back to the radar sensor from an object external to the vehicle to be visualized by means of the display device. In the case of one or more radar sensors for environmental detection, the vehicle occupant can thus recognize both a field of view of the relevant sensor system and an interaction with the environment in the form of the returned or reflected radar waves. The mode of operation of the relevant environmental sensors can thus be particularly well understood by the vehicle occupant.In general, it is possible, regardless of the type of environmental sensor, to convey the respective mode of operation of the environmental sensor, particularly with the help of further animations in an external area of ​​the motor vehicle, so that the vehicle occupant can see how the respective sensor data is processed and converted into a driving movement during semi-autonomous or fully autonomous driving.

[0017] A further advantageous embodiment of the invention provides that the visualization of the arrangement and operation of the motor vehicle's environmental sensors is automatically started as soon as the motor vehicle switches from a manual driving mode to at least a semi-autonomous driving mode. As soon as a driver no longer fully controls the motor vehicle, be it the longitudinal and / or lateral control, the visualization of the arrangement and operation of the environmental sensors starts. This gives the driver and possibly other vehicle occupants a particularly safe feeling, since the vehicle occupants can directly see how the motor vehicle's environmental sensors are arranged and operate when the motor vehicle is no longer manually controlled.Of course, it is also possible to prevent the automatic start of this visualization, for example via a corresponding menu in a vehicle's infotainment system, so that a vehicle occupant can select whether or not he wants the automatic start of the visualization of the arrangement and operation of the environmental sensors.

[0018] The display device according to the invention can be arranged in a motor vehicle and is configured to visualize, during at least partially autonomous driving of the motor vehicle, an arrangement and mode of operation of environmental sensors of the motor vehicle, the sensor data of which are used during at least partially autonomous driving. The display device can be augmented reality glasses, an augmented reality contact lens, virtual reality glasses, or a display device that can be installed in the motor vehicle. The latter can, in particular, be a contact-analog display device. Advantageous embodiments of the method according to the invention are to be regarded as advantageous embodiments of the display device according to the invention and vice versa, wherein the display device in particular has means for carrying out the method steps.

[0019] An advantageous embodiment of the display device provides that the display device has a data interface for wirelessly receiving data from the motor vehicle, which characterizes the arrangement and operation of the motor vehicle's environmental sensors. The interface can be, for example, a radio module, by means of which communication between the motor vehicle and the display device is possible via Bluetooth or, for example, WLAN. The interface makes it particularly easy to transmit said data from the motor vehicle, which characterizes the arrangement and operation of the motor vehicle's environmental sensors, to the display device.In particular, when it comes to a mobile display device, such as augmented reality data glasses or the like, the display device does not have to be laboriously wired or connected to the motor vehicle in order to receive the data.

[0020] The motor vehicle according to the invention is designed to drive at least partially autonomously and has a data interface for wirelessly transmitting data to the above-mentioned display device, wherein the data characterize an arrangement and mode of operation of environmental sensors of the motor vehicle, the sensor data of which the motor vehicle uses during at least partially autonomous driving.

[0021] Further advantages, features, and details of the invention will become apparent from the following description of a preferred embodiment and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the single figure, can be used not only in the respective combinations specified, but also in other combinations or on their own, without departing from the scope of the invention.

[0022] The drawing shows in the only figure a highly schematic representation of a motor vehicle in which a vehicle occupant sits wearing augmented reality glasses, by means of which the arrangement and mode of operation of the vehicle's environmental sensors are visualized.

[0023] A motor vehicle 1 is shown in a highly schematic representation in the single figure. A passenger 2 sits in the motor vehicle 1 wearing augmented reality glasses 3. The motor vehicle 1 comprises several environment sensors 4, indicated only schematically, as well as a vehicle-side data interface 5 for wirelessly transmitting data to the augmented reality glasses 3, wherein the data characterizes an arrangement and mode of operation of the environment sensors 4 of the motor vehicle 1, whose sensor data the motor vehicle 1 uses during semi-autonomous and fully autonomous driving. The augmented reality glasses 3 also have a data interface (not shown in detail here), which can communicate wirelessly with the data interface 5 of the motor vehicle 1 in order to receive the said data.

[0024] As soon as the motor vehicle 1 is driving semi-autonomously or fully autonomously, both the arrangement and the mode of operation of the environmental sensors 4 of the motor vehicle 1 are visualized using the augmented reality glasses 3. For example, the augmented reality glasses 3 indicate the respective positioning of the environmental sensors 4 on the motor vehicle 1. If the vehicle occupant 2 looks forward through the augmented reality glasses 3, the position of the front environmental sensor 4 is indicated, for example, via a corresponding virtual symbol displayed by the augmented reality glasses 3.

[0025] If the vehicle occupant 2 looks backward by tilting their head, they will be shown the position of the rear environment sensor 4, for example, by a different symbol. Furthermore, the augmented reality glasses 3 can also indicate the respective detection areas 6 of the environment sensors 4. If the vehicle occupant 2 looks forward through a windshield of the motor vehicle 1, the augmented reality glasses 3 display the detection area 6, so that the vehicle occupant 2 can see, for example, how far forward the environment sensor 4 can see and detect the vehicle's surroundings. The same applies to the rear environment sensor 4.

[0026] Of course, the motor vehicle 1 can also have additional environmental sensors not shown here, which, for example, detect and cover a lateral environmental area of ​​the motor vehicle 1. The respective sensor operating principle of the environmental sensors 4 can also be visualized using the augmented reality glasses 3. If the environmental sensors 4 are radar sensors, for example, animated radar waves can be visualized using the augmented reality glasses 3. These are displayed using the augmented reality glasses 3, for example, in such a way that they identify the detection area 6. Alternatively, it is also possible for the detection area 6 to be displayed as a type of cone, with the radar waves also being displayed.In addition, it can also be provided that the radar waves which are reflected back from an object external to the vehicle to the relevant environment sensor 4 designed as a radar sensor are visualized by means of the augmented reality glasses 3.

[0027] Using the augmented reality glasses 3, the vehicle occupant 2 can thus view the spatially correct location of the environmental sensors 4 required for semi- or fully autonomous driving while driving. Furthermore, it is also possible to visually illustrate the mode of operation, and further animations on the exterior of the motor vehicle 1 can be used to convey how the motor vehicle 1 processes the available sensor data and translates it into semi- or fully autonomous driving.

[0028] Above all, it can also be provided that the visualization of the arrangement and mode of operation of the environmental sensors 4 of the motor vehicle 1 is started automatically as soon as the motor vehicle 1 changes from a manual driving mode to a semi-autonomous or fully autonomous driving mode.

[0029] The functionality explained above is not limited to the augmented reality glasses 3. Instead of the augmented reality glasses 3, augmented reality contact lenses, virtual reality glasses, or various display devices installed in the motor vehicle 1 can also be used. In the latter case, contact-analog display devices can be particularly helpful. For example, a contact-analog head-up display can be used to visualize the arrangement and operation of the environmental sensors 4. Contact analog is a special form of augmented reality. Here, the displayed information merges with the environment. The displayed information appears perspectively correct at the location to which it refers and virtually adheres to objects in the environment.By means of contact-analog vehicle-side display devices, it is therefore possible to visualize the arrangement and mode of operation of the environmental sensors 4 of the motor vehicle 1 even without the need for additional aids.

[0030] Overall, the described method improves the sense of safety of vehicle occupants 2 during semi-autonomous or fully autonomous driving, since vehicle occupants 2 can recognize both the arrangement and the mode of operation of the environmental sensors 4 of the motor vehicle 1, the sensor data of which are used during semi-autonomous or fully autonomous driving.

Claims

1. A method for operating a motor vehicle (1), in which, during an at least partially autonomous driving operation of the motor vehicle (1), by means of at least one display device (3) arranged in the motor vehicle (1), an arrangement and a mode of action of surround sensors (4) of the motor vehicle (1) is visualized, the sensor data from which are used in the at least partially autonomous driving operation, characterized in that the display device (3) is augmented reality glasses, an augmented reality contact lens, or virtual reality glasses, and by means of the display device (3), the respective positioning and mode of action of the surround sensors (4) on the motor vehicle (1) are identified and their respective detection regions (6) are visualized, such that a vehicle occupant (2) looking forward or backward can see how far the surround sensor (4) in question can view forward or backward and detect the vehicle surroundings.

2. The method according to claim 1, characterized in that a relevant sensor operating principle of the surround sensors (4) is visualized by means of the display device (3).

3. The method according to any of the preceding claims, characterized in that if at least one of the surround sensors (4) is a radar sensor, respective radar waves are visualized by means of the display device (3).

4. The method according to claim 3, characterized in that the radar waves which are reflected back from an object outside the vehicle to the radar sensor are visualized by means of the display device (3).

5. The method according to any of the preceding claims, characterized in that the visualization of the arrangement and mode of action of the surround sensors (4) of the motor vehicle (1) is started automatically once the motor vehicle (1) switches from a manual driving mode into at least a partially autonomous driving mode.

6. A display device (3), which can be arranged in a motor vehicle (1) and is configured to, during an at least partially autonomous driving operation of the motor vehicle (1), visualize an arrangement and a mode of action of surround sensors (4) of the motor vehicle (1), the sensor data from which are used in the at least partially autonomous driving operation.

7. The display device (3) according to claim 6, characterized in that the display device (3) comprises a data interface (5) for wirelessly receiving data from the motor vehicle (1), which data characterize the arrangement and mode of action of the surround sensors (4) of the motor vehicle (1).

8. A motor vehicle (1), which is configured to drive at least partially autonomously and comprises a data interface (5) for wirelessly transmitting data to a display device (3) according to claim 6 or 7, wherein the data characterize an arrangement and mode of action of surround sensors (4) of the motor vehicle (1), the sensor data from which are used by the motor vehicle (1) in the at least partially autonomous driving operation.