Method and display device for visualizing the arrangement and mode of operation of motor vehicle surroundings sensors

Abstract

The present invention relates to a method for operating a motor vehicle (1), wherein, during at least a semi-automatic driving mode of the motor vehicle (1), the arrangement and operation of environmental sensors (4) of the motor vehicle (1) are visualized by means of at least one display device (3) arranged in the motor vehicle (1), and sensor data of said environmental sensors are used in the at least semi-automatic driving mode. The invention also relates to a display device (3) that can be arranged in the motor vehicle (1) and is configured to visualize the arrangement and operation of environmental sensors (4) of the motor vehicle (1) during at least a semi-automatic driving mode. Furthermore, the invention relates to a motor vehicle (1) having a data interface (5) for wirelessly transmitting data to the display device (3).

Description

Technical Field

[0001] The invention relates to a method and a display device for visualizing the arrangement and mode of operation of motor vehicle environment sensors. The invention also relates to a motor vehicle. Background Art

[0002] As automated driving concepts are increasingly used in motor vehicles, it is important to understand how automated vehicles work and the resulting high reliability.

[0003] DE 10 2016 214 916 A1 shows a method for operating a motor vehicle, wherein the motor vehicle is driven automatically and during this process a maneuver being performed or about to be performed by the motor vehicle is displayed on a display of the motor vehicle. The performance of sensors of the motor vehicle can be displayed in a manner visible to the driver by means of the display.

[0004] DE 10 2013 021 853 A1 discloses a device and a method for operating an automatic driving system of a vehicle. Data glasses arranged outside the vehicle are used to display sensor data of the vehicle in the event of an error in the automatic driving maneuver of the vehicle.

[0005] DE 10 2011 121 948 A1 shows a method in which a preview of the behavior of an automated driving system is given. A display of the vehicle is used to show a planned automated driving maneuver of the vehicle. Summary of the invention

[0006] The object of the present invention is to provide a solution by means of which the operating mode of an at least partially automated vehicle can be visualized in a particularly simple and understandable manner.

[0007] In the method for operating a motor vehicle according to the invention, during an at least semi-automatic driving mode of the motor vehicle, the arrangement and mode of operation of the motor vehicle's environmental sensors whose sensor data are used in the at least semi-automatic driving mode are visualized by means of at least one display device arranged in the motor vehicle. A semi-automatic driving mode means that the motor vehicle is operated automatically at least with respect to longitudinal guidance or transverse guidance without the driver having to personally control and intervene in this. Likewise, in the method according to the invention, it is also possible to visualize the arrangement and mode of operation of the motor vehicle's environmental sensors by means of a display device arranged in the motor vehicle in a fully automatic driving mode of the motor vehicle, i.e., in a driving mode in which the driver does not intervene in either longitudinal guidance or transverse guidance.

[0008] The invention is based on the recognition that, with the increasing use of automated driving, it is very important to inform customers, employees and members of the public (e.g. journalists) about the way automated vehicles work and the resulting high reliability. By visualizing the arrangement and mode of operation of the environmental sensors of the motor vehicle, which use their sensor data in an at least semi-automatic driving mode, with the aid of a display device arranged in the motor vehicle, a particularly high acceptance of the semi-automatic driving mode or fully automatic driving mode by the occupants of the motor vehicle can be achieved. The spatially exact arrangement of the environmental sensors of the motor vehicle is thus visualized, so that the occupants can see where the environmental sensors of the motor vehicle are arranged. Since the mode of operation of the environmental sensors is also visualized, the occupants can easily see what the motor vehicle detects from its environment. It is therefore also possible to inform, with the aid of a display device, how the motor vehicle processes the sensor data in the at least semi-automatic driving mode or in the fully automatic driving mode and uses them for the semi-automatic or fully automatic driving movement. For this purpose, it is provided that a permanent communication or data exchange takes place between the relevant display device and the motor vehicle, for example via a suitable interface, for example in the form of Bluetooth or the like.

[0009] An advantageous embodiment of the present invention proposes that the display device is an augmented reality glasses, an augmented reality contact lens, a virtual reality glasses or a display device of a motor vehicle, in particular a contact simulation display device. With the aid of the augmented reality glasses, the occupants can still perceive their surroundings while simply displaying information about the arrangement and working mode of the environmental sensors. If the occupants look forward when wearing the augmented reality glasses, for example, the augmented reality glasses show the sensors arranged in the front area of ​​the motor vehicle in the form of icons, etc. In addition, the augmented reality glasses can also visualize the working mode of the relevant sensors by correspondingly superimposing virtual content on reality. If the occupants turn their heads to the left when wearing the augmented reality glasses, for example, the occupants see icons or the like, which indicate the sensors and their positioning arranged laterally on the motor vehicle to monitor the vehicle surroundings in the side direction. This can also be achieved using augmented reality contact lenses or contact simulation display devices. The contact simulation display device can be, for example, a head-up display or a display device integrated in the corresponding window glass of the motor vehicle. With the help of augmented reality glasses, augmented reality contact lenses and contact simulation display devices, the relevant passengers can still perceive their real surroundings, while at the same time, with the help of these display devices, the arrangement and function of the vehicle's environmental sensors can be visualized in a superimposed manner on reality. The passenger (especially when the passenger is a co-pilot instead of the driver) can also wear virtual reality glasses, so as to visualize the arrangement and function of the vehicle's environmental sensors with the help of virtual reality glasses.

[0010] Another advantageous embodiment of the invention provides that the respective positioning of the surroundings sensors on the motor vehicle is shown by means of a display device and the respective detection range of the surroundings sensors is visualized. The occupants can thus very easily see where the respective surroundings sensors of the motor vehicle are arranged and which detection ranges they can cover. In particular, if the detection ranges are large, this can help to give the occupants a particular peace of mind during semi-automatic or automatic driving of the motor vehicle, because the occupants are clearly shown how much of the surroundings the motor vehicle can actually detect with its sensors.

[0011] According to another advantageous embodiment of the invention, it is provided that the respective sensor operating principle of the environmental sensors is visualized by means of a display device. Thus, the occupants can easily see whether the respective environmental sensor is, for example, a camera, a lidar sensor, an infrared sensor, etc. This can be achieved, for example, by displaying a suitable symbol according to the respective sensor operating principle at the location where the respective environmental sensor is actually arranged. Thus, the occupants can understand the sensor operating principle on which the respective environmental sensor is based particularly easily, which can also help the occupants to accept the semi-automatic or fully automatic driving mode of the motor vehicle particularly well.

[0012] In another advantageous embodiment of the present invention, it is provided that if at least one of the environmental sensors is a radar sensor, the corresponding radar waves are visualized by means of a display device. In particular, it is also possible to visualize the radar waves reflected back to the radar sensor by objects outside the vehicle by means of a display device. In the case of one or more radar sensors for environmental recognition, the occupants can therefore not only see the field of view of the relevant sensor device but also the interaction with the environment in the form of reflected or reflected radar waves. The mode of action of the relevant environmental sensor can thus be particularly well understood by the occupants. In general, it is possible to inform the occupants of the corresponding mode of action of the environmental sensors, in particular by means of continuous animations in the external area of ​​the motor vehicle, regardless of the respective type of environmental sensor, so that the occupants can see how the corresponding sensor data is processed and how it is applied to the driving movement in semi-automatic or fully automatic driving mode.

[0013] Another advantageous embodiment of the invention provides that the visualization of the arrangement and mode of operation of the vehicle's environmental sensors is automatically started as soon as the vehicle is switched from a manual driving mode to an at least semi-automatic driving mode. The visualization of the arrangement and mode of operation of the environmental sensors is started as soon as the driver no longer fully controls the vehicle himself (regardless of whether this is longitudinal control and / or lateral control). This provides the driver and, if necessary, other passengers with a particularly safe feeling, since the passengers can directly see how the vehicle's environmental sensors are arranged and how they function when the vehicle is no longer manually controlled. Of course, the automatic start of this visualization can also be disabled, for example by means of a corresponding menu in the vehicle's infotainment system, so that the passenger can select whether he wants to automatically start the visualization of the arrangement and mode of operation of the environmental sensors.

[0014] The display device according to the invention can be arranged in a motor vehicle and is configured to visualize the arrangement and mode of operation of the following environmental sensors of the motor vehicle during an at least semi-automatic driving mode of the motor vehicle: the sensor data of the environmental sensors are used in the at least semi-automatic driving mode. The display device can be augmented reality glasses, augmented reality contact lenses, virtual reality glasses or a display device installed in a motor vehicle. The latter can be a display device for contact simulation in particular. An advantageous embodiment of the method according to the invention can be regarded as an advantageous embodiment of the display device according to the invention, and vice versa, wherein the display device has in particular means for executing the method steps.

[0015] An advantageous embodiment of the display device provides that the display device has a data interface for wirelessly receiving data from the motor vehicle, the data characterizing the arrangement and mode of operation of the motor vehicle's environmental sensors. The interface may be, for example, a radio module, with the aid of which communication between the motor vehicle and the display device via Bluetooth or, for example, WLAN can be realized. The motor vehicle data characterizing the arrangement and mode of operation of the motor vehicle's environmental sensors can be transmitted to the display device particularly easily via the interface. In particular, when the display device is a mobile display device, such as augmented reality data glasses or the like, the display device does not need to be connected to the motor vehicle by complex wiring or wires in order to receive the data.

[0016] The motor vehicle according to the invention is designed for at least semi-automatic driving and has a data interface for wirelessly transmitting data to the aforementioned display device, wherein the data characterize the arrangement and mode of operation of environmental sensors of the motor vehicle, and the motor vehicle uses sensor data of the environmental sensors in the at least semi-automatic driving mode.

[0017] Further advantages, features and details of the invention are apparent from the following description of preferred exemplary embodiments and from the drawings. The features and feature combinations mentioned above in the description and the features and feature combinations mentioned below in the description of the figures and / or shown individually in the single figure can be used not only in the respectively indicated combination but also in other combinations or individually without departing from the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Attached picture Figure 1 , a very schematic illustration of a motor vehicle is shown, in which an occupant is seated who wears augmented reality glasses, with the aid of which the arrangement and mode of operation of the motor vehicle's surrounding sensors are visualized. DETAILED DESCRIPTION

[0019] exist Figure 1 1 shows a motor vehicle 1 in a very schematic illustration. In the motor vehicle 1 sits an occupant 2 who is wearing augmented reality glasses 3. The motor vehicle 1 comprises a plurality of only schematically shown environmental sensors 4 and a vehicle-side data interface 5 for wirelessly transmitting data to the augmented reality glasses 3, wherein the data characterize the arrangement and mode of operation of the environmental sensors 4 of the motor vehicle 1, the sensor data of the environmental sensors being used by the motor vehicle 1 in semi-automatic and fully-automatic driving modes. The augmented reality glasses 3 also have a data interface (not shown in detail here) which can communicate wirelessly with the data interface 5 of the motor vehicle 1 in order to receive the data.

[0020] As soon as the motor vehicle 1 is driven semi-automatically or fully automatically, the arrangement and also the mode of operation of the environment sensors 4 of the motor vehicle 1 can be visualized with the aid of the augmented reality glasses 3. For example, the respective position of the environment sensors 4 on the motor vehicle 1 can be shown with the aid of the augmented reality glasses 3. If the occupant 2 looks forward through the augmented reality glasses 3, he or she can see the position of the environment sensors 4 in front, for example, by means of corresponding virtual symbols (which are displayed by the augmented reality glasses 3).

[0021] If the occupant 2 looks back by turning his head, he can see the position of the rear environment sensor 4 again, for example, through another symbol. In addition, the augmented reality glasses 3 also show the detection range 6 of the respective environment sensor 4. If the occupant 2 looks forward through the windshield of the motor vehicle 1, the augmented reality glasses 3 show the detection range 6, so that the occupant 2 can see when looking forward: how far the environment sensor 4 can see forward and how far it can detect the vehicle surroundings. This also applies to the rear environment sensor 4.

[0022] Of course, the motor vehicle 1 can also have other environmental sensors not shown here, which, for example, detect and cover the lateral surrounding area of ​​the motor vehicle 1. The sensor operating principle of each environmental sensor 4 can also be visualized with the aid of the augmented reality glasses 3. If the environmental sensor 4 is, for example, a radar sensor, the radar waves can be visualized in an animated manner with the aid of the augmented reality glasses 3. This is displayed with the aid of the augmented reality glasses 3, for example, in such a way that the detection area 6 can be seen. Alternatively, it is also possible that the detection area 6 is shown as a cone, wherein the radar waves are also shown in addition. It can also be provided that the radar waves reflected by objects outside the vehicle back to the relevant environmental sensor 4 designed as a radar sensor are visualized with the aid of the augmented reality glasses 3.

[0023] The occupant 2 can thus obtain a spatially correctly positioned display of the surrounding sensors 4 required for semi-automatic or fully automated driving during driving by means of the augmented reality glasses 3. Furthermore, the mode of operation can also be shown in a visualized diagram and a continuous animation can also be used in the outer area of ​​the motor vehicle 1 to indicate how the motor vehicle 1 processes the current sensor data and uses it for semi-automatic or fully automated driving movements.

[0024] In particular, it may also be provided that the visualization of the arrangement and mode of operation of environmental sensors 4 of motor vehicle 1 is automatically initiated as soon as motor vehicle 1 switches from a manual driving mode to a semi-automatic or fully automatic driving mode.

[0025] The above-described working method is not limited to augmented reality glasses 3. Instead of augmented reality glasses 3, for example, augmented reality contact lenses, virtual reality glasses or various display devices installed in the motor vehicle 1 can also be used. In the last case, especially the contact-simulated display device can be very helpful. Thus, for example, a contact-simulated head-up display can be used to visualize the arrangement and operation of the environmental sensors 4. Contact simulation is a special form of augmented reality. The information displayed here is integrated with the environment. The displayed information appears at the correct perspective at the positions to which they are respectively referenced and corresponds to objects in the surrounding environment to a certain extent. With the help of a contact-simulated display device on the vehicle side, the arrangement and operation of the environmental sensors 4 of the motor vehicle 1 can also be visualized without the need for additional wearing auxiliary equipment.

[0026] Overall, the described method improves the sense of safety of occupant 2 during semi-automatic or fully automatic driving, since occupant 2 can recognize not only the arrangement of environmental sensors 4 of motor vehicle 1 but also the mode of operation of environmental sensors 4 whose sensor data are used during the semi-automatic or fully automatic driving mode.

Claims

1. A method for operating a motor vehicle (1), wherein: During an at least semi-automatic driving mode of a motor vehicle (1), the arrangement and mode of operation of the following environmental sensors (4) of the motor vehicle (1) are visualized by means of at least one display device (3) arranged in the motor vehicle (1): sensor data of the environmental sensors are used in the at least semi-automatic driving mode, wherein the display device (3) is an augmented reality pair of glasses, a virtual reality pair of glasses or a contact simulation display device of the motor vehicle (1), by means of which the respective positioning of the environmental sensors (4) on the motor vehicle (1) is shown and the respective detection range (6) of the environmental sensors is visualized, wherein the occupant (2) can visualize the respective detection range (6) of the environmental sensors by means of the augmented reality pair of glasses or the contact simulation display device. The vehicle (1) can still perceive its real surrounding environment, wherein the arrangement and functioning mode of the environmental sensors (4) of the motor vehicle (1) can be visualized by means of these display devices in a manner superimposed on reality, or the occupant (2) wears virtual reality glasses, and the arrangement and functioning mode of the environmental sensors (4) of the motor vehicle (1) can be visualized by means of virtual reality glasses, wherein the sensor working principle of each environmental sensor (4) is visualized by means of the display device (3), and by displaying appropriate symbols at the actual arrangement position of the corresponding environmental sensor according to the corresponding sensor working principle, the occupant (2) can see whether each environmental sensor (4) is a camera, a lidar sensor or an infrared sensor.

2. The method according to claim 1, It is characterized in that AR glasses are augmented reality contact lenses.

3. The method according to claim 1 or 2, It is characterized in that If at least one of the surrounding sensors (4) is a radar sensor, the corresponding radar waves are visualized by means of the display device (3).

4. The method according to claim 3, It is characterized in that Radar waves reflected by objects outside the vehicle back to the radar sensor are visualized by means of a display device (3).

5. The method according to claim 1 or 2, It is characterized in that As soon as the motor vehicle (1) is switched from a manual driving mode to an at least semi-automatic driving mode, a visualization of the arrangement and mode of operation of the surroundings sensors (4) of the motor vehicle (1) is automatically initiated.

6. A display device (3) which can be arranged in a motor vehicle (1) and is designed to visualize the arrangement and mode of operation of the following environmental sensors (4) of the motor vehicle (1) during an at least semi-automatic driving mode of the motor vehicle (1): the sensor data of the environmental sensors are used in the at least semi-automatic driving mode, wherein: The display device (3) is an augmented reality glasses, a virtual reality glasses, or a contact simulation display device of the motor vehicle (1), and the display device (3) is used to display the respective positioning of the environmental sensors (4) on the motor vehicle (1) and visualize the respective detection areas (6) of the environmental sensors, wherein the occupant (2) can still perceive his real surroundings with the help of the augmented reality glasses or the contact simulation display device, wherein the arrangement and function of the environmental sensors (4) of the motor vehicle (1) can be visualized in a superimposed manner on reality with the help of these display devices, or the occupant (2) wears virtual reality glasses, and the arrangement and function of the environmental sensors (4) of the motor vehicle (1) can be visualized with the help of the virtual reality glasses, wherein the sensor working principle of each environmental sensor (4) is visualized with the help of the display device (3), and by displaying appropriate symbols at the actual arrangement position of the corresponding environmental sensor according to the corresponding sensor working principle, the occupant (2) can see whether each environmental sensor (4) is a camera, a lidar sensor or an infrared sensor.

7. The display device (3) according to claim 6, It is characterized in that The display device (3) has a data interface (5) for wirelessly receiving data from the motor vehicle (1), which data interface characterizes the arrangement and mode of operation of an environmental sensor (4) of the motor vehicle (1).

8. The display device (3) according to claim 6, characterized in that AR glasses are augmented reality contact lenses.

9. A motor vehicle (1) which is designed for at least semi-automatic driving and has a data interface (5) for wirelessly transmitting data to a display device (3) according to any one of claims 6 to 8, wherein: The data characterize the arrangement and mode of operation of an environmental sensor (4) of the motor vehicle (1), the sensor data of which are used by the motor vehicle (1) in an at least semi-automatic driving mode.

Images