Method and control device for operating a head-mounted display device in a motor vehicle

Abstract

Method for operating a head-mounted display device in a motor vehicle (1) wherein at least the condition is checked whether there is no other vehicle occupant (9) in the motor vehicle (1) besides a driver (2) when the motor vehicle (1) is not operated in an autonomous driving mode, and if this condition is met, the following restriction is applied to the display device with regard to the display of content: - the display of any content is prevented if the display device is a virtual reality headset (3) by preventing data transmission of displayable content from the motor vehicle (1) and / or from another data source to the virtual reality headset (3); - The display device enables only the display of traffic-related content if the display device is an augmented reality headset, by enabling the transmission of traffic-related content on the headset side and / or on the vehicle side, whereby, if another vehicle occupant (9) is in the motor vehicle (1), as a further condition, it is checked whether this vehicle occupant (9) is in the passenger seat (6), and if this further condition is met, the display device enables the display of any information.

Description

[0001] The invention relates to a method and a control device for operating a head-mounted display device in a motor vehicle. Furthermore, the invention relates to a motor vehicle with such a control device.

[0002] In the foreseeable future, it will be possible to wear virtual reality or augmented reality headsets in motor vehicles for entertainment purposes and to enjoy various content through them. Such entertainment options are intended more for passengers than for the driver themselves—especially before the availability of autonomous driving.

[0003] DE 10 2013 221 858 A1 discloses a method for operating smart glasses in a motor vehicle. If it is determined that a seat assigned to the smart glasses is the driver's seat of the vehicle, the transmission of user data, in particular graphic or video data, to the smart glasses is prevented. This occurs particularly if it cannot be reliably determined that the driver is not using the smart glasses in a manner adapted to a driving task. This is intended to reduce the risk of distraction caused by the smart glasses.

[0004] DE 10 2014 206 623 A3 describes a method for operating smart glasses in a vehicle. A display on the smart glasses is controlled depending on the seat position of the user.

[0005] DE 10 2016 213 916 A1 shows an infotainment system of a motor vehicle which has data glasses for displaying a virtual reality, whereby the data glasses can be assigned to a very specific seat.

[0006] DE 10 2016 003 019 A1 relates to a display system for a motor vehicle, comprising a display device for displaying contents on an at least partially transparent display surface and comprising a control device which is designed to adapt the transparency of the displayed contents depending on a detected operating mode and / or driving situation of the motor vehicle.

[0007] DE 10 2013 021 137 A1 relates to a method for operating a data interface of a motor vehicle, by means of which the motor vehicle and at least one pair of electronic data glasses are coupled to one another for exchanging data, comprising the steps of: determining, based on a detected position of the data glasses, where the data glasses are arranged in the motor vehicle; controlling the data interface of the motor vehicle depending on the determined position of the data glasses in the motor vehicle.

[0008] DE 10 2016 006 382 A1 describes a method for operating an emergency call device for a motor vehicle, wherein a transmission of an emergency call signal to a remote station outside the motor vehicle is triggered manually or automatically, wherein during or after the transmission of the emergency call signal, upon receipt of a signal from the remote station by the emergency call device and / or upon detection of a vehicle occupant exiting the motor vehicle by evaluating sensor data from at least one sensor device of the motor vehicle, immediately or after fulfilling an additional condition, location information relating to a location of the and / or at least one other vehicle occupant is transmitted from the emergency call device to the remote station and / or a configuration of a human-machine interface of the motor vehicle,which comprises at least one input device and / or at least one output device and which serves for communication between the remote station and the and / or the other vehicle occupant, is changed.,

[0009] DE 10 2009 008 492 A1 describes a motor vehicle with a reproduction device for reproducing image information, comprising a control device and a display, wherein a first and a second image representation, which each occupy the display area of ​​the display, can be displayed simultaneously via the control device, wherein the first image representation can only be recognized from a first distinguished viewing angle range directed towards the driver, and the second image representation can only be recognized from a second viewing angle range directed towards the passenger, and wherein at least one detection device assigned to the control device is provided, via which the position of the driver can be detected directly or indirectly, wherein the reproduction of the second image representation can be controlled as a function of the position detection result.

[0010] It is the object of the present invention to provide a solution by means of which it can be ensured in a particularly simple manner that a driver of a motor vehicle is not distracted from his driving task by a display device worn on the head.

[0011] This object is achieved by a method and a control device for operating a head-mounted display device in a motor vehicle having the features of the independent patent claims. Advantageous embodiments with expedient and non-trivial further developments of the invention are specified in the dependent claims.

[0012] In the method according to the invention for operating a head-mounted display device in a motor vehicle, at least the condition is checked whether there are no other vehicle occupants in the motor vehicle other than a driver when the motor vehicle is not operated in an autonomous driving mode. For the purposes of the invention, this means that the driver drives the motor vehicle entirely themselves. This can also be understood to mean that certain assistance systems help with vehicle operation, for example adaptive cruise control. This can also be understood to mean that a certain degree of partial automation is present in the fulfillment of the driving task. Functions such as automatic parking, lane keeping, general steering guidance, acceleration, and braking can also be taken over by assistance systems, for example by a traffic jam assistant.

[0013] The fact that the motor vehicle is not operated in an autonomous driving mode can, for example, relate to autonomy levels 0, 1 and 2 according to the SAE J3016 standard. An autonomous driving mode within the meaning of the invention is therefore to be understood as, for example, an automated operation. The driver, for example, does not have to constantly monitor the system of the motor vehicle. The motor vehicle independently performs functions such as activating the turn signal, changing lanes and keeping in lane. The driver can attend to other things but, if necessary, will be prompted by the system within a pre-warning period to take over driving. High automation within the meaning of the invention can also include the motor vehicle being operated in an autonomous driving mode. Driving of the motor vehicle is permanently taken over by a suitable system. If the system can no longer handle the driving tasks, the driver can be prompted to take over driving.Autonomous driving mode can also be understood as full automation, where no driver is required at all. No human intervention is required other than setting the destination and starting the system.

[0014] The motor vehicle can generally be a motor vehicle that can drive itself autonomously or cannot drive autonomously. If the motor vehicle is not operated in an autonomous driving mode - either because the motor vehicle is not capable of doing so or because a corresponding driving mode has been selected - and the condition is met that there are no other vehicle occupants in the motor vehicle other than the driver, one of the following restrictions is applied to the head-mounted display device with regard to the display of content: the display of any content is prevented if the display device is virtual reality glasses; only the display of traffic-relevant content is enabled via the display device if the display device is augmented reality glasses.

[0015] To reduce the risk of the driver accidentally or intentionally using their virtual reality or augmented reality glasses while driving alone, thereby exposing themselves to potentially avoidable risks, the display of any content is prevented if the display device is virtual reality glasses. Only traffic-relevant content can be displayed using the display device if the display device is augmented reality glasses. In the case of augmented reality glasses, this is controlled in such a way that it can only be used in a mode adapted to the driving task. In this case, the augmented reality glasses can display, for example, navigation instructions, speed information, and the like; the display of graphics and / or videos that serve purely for entertainment purposes is then prevented.In the case of virtual reality glasses, it makes sense to prevent the display of any content if the above condition is met. Virtual reality glasses completely isolate the wearer visually from their surroundings.

[0016] According to the invention, it is therefore provided that if there is another vehicle occupant in the motor vehicle, a further condition is checked as to whether this vehicle occupant is in the passenger seat, and if this further condition is met, the display of any information by the display device is made possible.

[0017] The invention is based on the finding that if a passenger or front passenger is detected, it can be assumed that they are using the head-mounted display device themselves or would prevent the driver from using it. With the method according to the invention, it is not necessary to use complex sensors to determine exactly where the head-mounted display device is located in the motor vehicle and which vehicle occupant is wearing it. By means of the method according to the invention, it is therefore possible in a particularly simple yet reliable manner to prevent a driver from being distracted from their actual driving task by wearing the head-mounted display device, provided the motor vehicle is not operated in an autonomous driving mode.

[0018] The invention provides that the display of any content in the case of virtual reality glasses is prevented by preventing data transmission of displayable content from the motor vehicle and / or from another data source to the virtual reality glasses. Image data and / or video data can, for example, be transmitted from an in-vehicle infotainment system to the virtual reality glasses. If the display of any content on the virtual reality glasses is to be prevented, a corresponding data connection between the infotainment system and the virtual reality glasses can, for example, be interrupted as long as the condition is met that no other vehicle occupant is in the vehicle other than the driver.In the case of augmented reality glasses, for example, it is possible for the glasses or the vehicle to only allow the transmission of traffic-relevant content to the augmented reality glasses, as long as the aforementioned restriction is in place. It is also possible for the head-mounted display device to receive the data to be displayed, for example, from a smartphone or another device not permanently installed in the vehicle. In this case, the data transmission of displayable content can also be prevented in the case of virtual reality glasses or filtered accordingly in the case of augmented reality glasses.

[0019] A further advantageous embodiment provides that if another vehicle occupant is present in the motor vehicle, a further condition is checked to determine whether this vehicle occupant is located behind the respective front seats of the motor vehicle, in particular on a rear seat of the motor vehicle. Only if this further condition is met is the display of any content enabled by the display device. If the further condition is not met, the restriction is applied. The latter therefore means that the display of any content is prevented in the case of virtual reality glasses, and in the case of augmented reality glasses, it is enabled exclusively for the display of traffic-relevant content.If, for example, the use of the head-mounted display is only desired or permitted in the back seat, a check is carried out if there are other vehicle occupants besides the driver to see whether they are, for example, in the back seat. The restriction regarding the playback or display of content via the display device therefore occurs if the at least one other vehicle occupant is not in the back seat and instead, for example, is in the front passenger seat. If, on the other hand, it is determined that the at least one other vehicle occupant is sitting in the back seat, any content can be displayed using the head-mounted display. This measure makes it possible to ensure in a particularly reliable way that the driver cannot be distracted from his driving task by the head-mounted display device.In this case, it is simply only possible to use the head-mounted display device to display any content if at least one other vehicle occupant is seated behind the front seats of the vehicle, for example on a back seat.

[0020] According to a further advantageous embodiment of the invention, the display of any content is prevented in the case of virtual reality glasses by deactivating the virtual reality glasses. In this way, it can be particularly reliably ensured that no content can be displayed using the virtual reality glasses in the case of the aforementioned restriction. If the driver has put on the virtual reality glasses while the motor vehicle is not operating in an autonomous driving mode, they would most likely remove or put off the virtual reality glasses, since they are then deactivated.

[0021] In a further advantageous embodiment of the invention, a seat occupancy detection system is used to check whether there are any other passengers in the vehicle other than the driver. Appropriate seat occupancy sensors in the vehicle can be used for this purpose. The seat occupancy detection system can also be used to check whether there is another passenger behind the respective front seats of the vehicle. Seat occupancy detection makes it easy to check whether and where there are any passengers in the vehicle.

[0022] A further advantageous embodiment of the invention provides for monitoring the opening and closing of vehicle doors of the motor vehicle and checking, based on this, whether there are no other vehicle occupants in the vehicle other than the driver. This also makes it possible to check whether another vehicle occupant has taken a seat, for example, in the back seat or a passenger seat. By monitoring the opening and closing movements of the vehicle doors, it is easy to determine where certain vehicle occupants are most likely to have taken their seats.

[0023] According to a further advantageous embodiment of the invention, the opening and closing of seat belt buckles of the motor vehicle is monitored and checked based on whether there are no other vehicle occupants in the vehicle other than the driver. In general, by checking or monitoring the seat belt buckles of the motor vehicle, it is possible to determine quite reliably whether and where certain vehicle occupants are seated in the vehicle.

[0024] A further advantageous embodiment of the invention provides for a camera-based check to determine whether there are any other vehicle occupants in the motor vehicle besides the driver. It is also possible to use a camera to check or monitor the location of any vehicle occupants in the motor vehicle. In general, sensors installed in the motor vehicle for vehicle occupancy and / or driver status detection, for example, in the form of cameras, can be used to detect the driver themselves or other vehicle occupants and their positioning in the motor vehicle.

[0025] The control device according to the invention for operating a head-mounted display device in a motor vehicle is configured to carry out the method according to the invention or an advantageous embodiment of the method according to the invention. Advantageous embodiments of the method according to the invention are to be regarded as advantageous embodiments of the control device according to the invention, and vice versa, wherein the control device in particular has means for carrying out the method steps. The control device can, for example, have one or more data interfaces via which the control device can be connected to a wide variety of sensors or detection devices that can be used to check the above-mentioned conditions.

[0026] The motor vehicle according to the invention comprises the control device according to the invention or an advantageous embodiment of the control device according to the invention. According to a further independent aspect of the invention, it can also be provided that the head-mounted display device itself comprises the control device.

[0027] Further advantages, features, and details of the invention will become apparent from the following description of preferred embodiments and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combinations, but also in other combinations or on their own, without departing from the scope of the invention.

[0028] The drawing shows: Fig. 1 a schematic representation of a motor vehicle in which a driver sits wearing virtual reality glasses; Fig. 2 a further schematic representation of the motor vehicle, in which, in addition to the driver, another vehicle occupant has taken a seat on a passenger seat; and in Fig. 3 a further schematic representation of the motor vehicle, wherein the further vehicle occupant is seated on a rear seat of the motor vehicle.

[0029] In the figures, identical or functionally identical elements are provided with the same reference symbols.

[0030] A motor vehicle 1 is shown in a schematic representation in Fig. 1. A driver 2 is seated in the motor vehicle 1, wearing virtual reality glasses 3. The virtual reality glasses 3 are connected via signal or data transmission to a control device 4, which is configured to control the virtual reality glasses 3. The data connection is preferably wireless. The driver 2 sits in a driver's seat 5, with a passenger seat 6 and a rear seat 7 of the motor vehicle 1 unoccupied.

[0031] A method for operating the virtual reality glasses 3 in the motor vehicle 1 is described in more detail below. If the motor vehicle 1 is not operated in an autonomous driving mode, the control device 4 at least checks whether there are no other vehicle occupants in the motor vehicle 1 other than the driver 2. To verify this, a detection device 8 is installed or arranged in the motor vehicle 1.

[0032] The detection device 8 can, for example, detect seat occupancy using corresponding seat occupancy sensors to check whether there are no other vehicle occupants in the motor vehicle 1 besides the driver 2. The detection device 8 can also have corresponding sensors designed to check or monitor the opening and closing of vehicle doors of the motor vehicle 1, so that the control device 4 can check based on this whether there are no other vehicle occupants in the motor vehicle 1 besides the driver 2.

[0033] The detection device 8 can also have corresponding sensors that can monitor the opening and closing of seat belt buckles of the motor vehicle 1, wherein the control device 4 can check based on this whether there is no other vehicle occupant in the motor vehicle 1 other than the driver 2. In addition, the detection device 8 can also have one or more cameras, so that the detection device 8 or control device 4 can use cameras to check whether there is no other vehicle occupant in the motor vehicle 1 other than the driver 2. In general, the detection device 8 can supply the control device 4 with a wide variety of sensor data regarding the vehicle occupancy, so that the control device 4 can check the condition of whether there is no other vehicle occupant in the motor vehicle other than the driver 2 based on this sensor data.

[0034] If this condition is met, the control device 4 imposes the following restrictions on the virtual reality glasses 3: The display of any content using the virtual reality glasses 3 is prevented. Therefore, if the driver 2 is sitting alone in the motor vehicle 1, it is simply impossible for him to display any content using the virtual reality glasses 3 as long as the motor vehicle 1 is not operating in an autonomous driving mode. The driver 2 will then automatically remove the virtual reality glasses 3 so that he can perform his actual driving task without any distraction.

[0035] It is also possible that these are not virtual reality glasses 3, but augmented reality glasses. In this case, the display device only enables the display of traffic-related content. The augmented reality glasses can then only display content that supports the driver 2 in performing their driving task, for example, by displaying navigation instructions, speed warnings, and the like, but no entertainment content.

[0036] In Fig. 2, the motor vehicle 1 is shown in a further schematic representation, with another vehicle occupant 9 seated in the front passenger seat 6. In this case, it may be provided that the aforementioned restriction is not applied to the virtual reality glasses 3—or to the augmented reality glasses. It can be assumed that the other vehicle occupant 9 will urge the driver 2 to remove the virtual reality glasses 3. Or, contrary to the present representation, it may also be the case that the vehicle occupant 9 who is seated in the front passenger seat 6 has put on the virtual reality glasses 3, so that there is no need to restrict the functionality of the virtual reality glasses 3 in the manner mentioned above. The same then also applies to the augmented reality glasses already mentioned.

[0037] However, it is also possible that if the additional vehicle occupant 9 is located in the motor vehicle 1, a further condition is checked to determine whether this vehicle occupant 9 is located behind the front seats 5, 6 of the motor vehicle 1, in particular on the rear seat 7. Only if this additional condition is met is the display of any content using the virtual reality glasses 3 – or the augmented reality glasses – enabled. If this additional condition is not met, the aforementioned restrictions are applied. It can therefore also be provided that the virtual reality glasses 3 – or the augmented reality glasses – cannot be used at all by vehicle occupants who are seated in the front seats 5, 6.This makes it possible to prevent, in a particularly reliable manner, the driver 2 from being distracted from his driving task by wearing the virtual reality glasses 3 or by wearing the augmented reality glasses, provided that the motor vehicle 1 is not operated in an autonomous driving mode.

[0038] In Fig. 3 shows a further schematic representation of the motor vehicle 1, with the vehicle occupant 9 now seated in the rear seat 7. In this case, there are no restrictions regarding the functionality of the virtual reality glasses 3 or the augmented reality glasses. Contrary to the present illustration, it could also be the case that the virtual reality glasses 3 or the augmented reality glasses are worn by the vehicle occupant 9.

[0039] The display of any content in the case of virtual reality glasses 3 can be prevented, for example, by the control device 4 preventing data transmission of displayable content from the motor vehicle 1 and / or from other data sources, for example from a smartphone, a tablet computer - via an internet connection or the like - to the virtual reality glasses 3. It is also possible to prevent the display of any content in the case of virtual reality glasses 3 by deactivating the virtual reality glasses 3 via the control device 4. In the case of augmented reality glasses, for example, data transmission from the augmented reality glasses can be filtered so that only traffic-relevant content is transmitted to the augmented reality glasses.Alternatively, it is also possible to simply intervene in the control of the augmented reality glasses so that they can only display traffic-relevant content, if this restriction is to apply. The control device 4 can be part of the motor vehicle 1. Alternatively, it is also possible for the control device 4 to be part of the virtual reality glasses 3 or the augmented reality glasses.

[0040] By means of the method and the control device 4 explained, it is therefore possible in a particularly simple and reliable manner to prevent the driver 2 from being distracted from his actual driving task by wearing the virtual reality glasses 3 or by wearing the augmented reality glasses during non-autonomous driving operation of the motor vehicle 1.

Claims

[1] Method for operating a head-mounted display device in a motor vehicle (1), in which at least the condition is checked whether there is no other vehicle occupant (9) in the motor vehicle (1) other than a driver (2) when the motor vehicle (1) is not operated in an autonomous driving mode, and if this condition is met, the following restriction is made to the display device with regard to displaying contents: - the display of any content is prevented if the display device is a virtual reality headset (3) by preventing data transmission of displayable content from the motor vehicle (1) and / or from another data source to the virtual reality headset (3); - only the display of traffic-relevant content is enabled by means of the display device if the display device is augmented reality glasses, by enabling the transmission of traffic-relevant content on the glasses side and / or on the vehicle side, wherein, if there is another vehicle occupant (9) in the motor vehicle (1), a further condition is checked as to whether this vehicle occupant (9) is in the front passenger seat (6), and if this further condition is met, the display of any information is enabled by the display device. [2] Method according to claim 1, characterized bythat if there is another vehicle occupant (9) in the motor vehicle (1), a further condition is checked as to whether this vehicle occupant (9) is located behind respective front seats of the motor vehicle (1), in particular on a rear seat (7) of the motor vehicle (1), and only if this further condition is met, the display of any content by means of the display device is enabled, wherein if the further condition is not met, the restriction is made. [3] Method according to one of the preceding claims, characterized by that the display of any content in the case of virtual reality glasses (3) is prevented by deactivating the virtual reality glasses (3). [4] Method according to one of the preceding claims, characterized by that a seat occupancy detection system checks whether there is no other vehicle occupant (9) in the motor vehicle (1) apart from the driver (2). [5] Method according to one of the preceding claims, characterized by that the opening and closing of vehicle doors of the motor vehicle (1) is monitored and checked based on whether there is no other vehicle occupant (9) in the motor vehicle (1) apart from the driver (2). [6] Method according to one of the preceding claims, characterized by that the opening and closing of seat belt buckles of the motor vehicle (1) is monitored and checked based on whether there is no other vehicle occupant (9) in the motor vehicle (1) apart from the driver (2). [7] Method according to one of the preceding claims, characterized by that a camera-based check is carried out to determine whether there is no other passenger (9) in the motor vehicle (1) apart from the driver (2). [8] Control device (4) for operating a head-mounted display device in a motor vehicle (1), which is designed to carry out a method according to one of the preceding claims. [9] Motor vehicle (1) with a control device (4) according to claim 8.

Images