Traffic-related control of virtual reality devices

Abstract

The present invention relates to a method for controlling a VR device, in particular VR glasses, of a user in the vehicle interior of a vehicle via a communication connection. Vehicle data is received via the communication connection and evaluated, or the evaluated vehicle data is received. Based on the evaluated vehicle data, at least one traffic event is classified, and a control command for controlling the VR device is generated, which indicates to the user of the VR device a vehicle reaction to the traffic event that was unexpected for the user. The present invention also relates to a vehicle system.

Description

Technical Field

[0001] The present invention relates to a method for controlling a VR (virtual reality) device, in particular VR glasses, of a user in a vehicle interior via a communication connection, and a vehicle system. Background Art

[0002] Currently, various theoretical application possibilities for virtual reality headsets, or VR devices, are known, though these are currently under development. For example, the following library is known, which allows developers of VR media formats to coordinate their content with vehicle motion. By implementing vehicle motion as an output in a VR media format, users of VR devices can achieve a stronger sense of immersion. Furthermore, engaging with vehicle motion can help minimize the tendency toward motion sickness.

[0003] When using a VR device in a vehicle, the user is shielded from events in and around the vehicle. While this fosters an immersive experience, it also disconnects the user from significant events in the vehicle. For example, the user cannot directly perceive or adapt to an impending accident in the vehicle. This can lead to the user being startled or surprised. Summary of the Invention

[0004] The object of the present invention is to provide a method and a vehicle system by means of which a user of a virtual reality device is protected from being startled or surprised.

[0005] According to the invention, this object is achieved by a method having the features of claim 1 and a vehicle system having the features of claim 8. Advantageous embodiments and developments are disclosed in the dependent claims.

[0006] The method according to the present invention is used to control a VR device, in particular VR glasses, of a user in the interior of a vehicle via a communication connection. In the method according to the present invention, vehicle data is received via the communication connection and evaluated, or the evaluated vehicle data is received. Based on the evaluated vehicle data, at least one traffic event is classified, wherein a control command for controlling the VR device is generated, which indicates to the user of the VR device a vehicle reaction to the traffic event that was unexpected for the user.

[0007] This method enables the forwarding of specific information from the vehicle to the VR device and the display of that information. For example, a data channel between the vehicle and the VR device can be used, which can be used for motion compensation when the VR device is in use. For example, the data channel can be based on wireless or wired communication technologies such as WLAN, Bluetooth, NFC, radio, infrared, Ethernet, USB, etc.

[0008] For example, a VR device can be designed as a virtual reality headset. Here, VR media content can be output via at least one display integrated into the headset or via a portable device (e.g., a smartphone) installed within the headset. The VR device can also include an earpiece, camera sensors, microphones, etc. to optimize the user's immersion.

[0009] The communication connection can preferably be based on a wireless transmission standard, such as Bluetooth, and can be constructed between the vehicle-side control device and the VR device. By coupling the VR device with the data transmission of the vehicle, an immersive VR experience can still be provided to the user. By means of the method according to the present invention, the user can be notified of special events in and around the vehicle. Thus, the VR device can react to traffic events in order to prevent the user of the VR device from being startled. Therefore, the method can improve the comfort level when using VR media content while driving.

[0010] At least one traffic event can be defined or classified based on a vehicle signal or vehicle data. The vehicle data can also be indirectly provided to a user of the VR device in the vehicle to enable traffic-related control of the VR device.

[0011] Before the vehicle reacts, traffic events such as an impending rear-end collision or successful pedestrian detection are already recorded by the vehicle's control unit. The corresponding evaluation results can be transmitted to the VR device in the form of evaluated vehicle data or directly in the form of control commands. Therefore, vehicle-side mechanisms that access vehicle-side sensors can be used to control the VR device. Alternatively or additionally, the vehicle data can be evaluated by the VR device.

[0012] According to one embodiment, a control command is generated for outputting a message on a display of a VR device, activating transparency of the VR device, outputting an acoustic or graphical notification, pausing or terminating an application of the VR device, and / or disabling a noise suppression function. This can generate a notification to a user of the VR device that prevents the user from being startled.

[0013] The message in the display of the VR device can be implemented in the form of a warning or prompt. Here, the VR device can output an acoustic, tactile and / or visual message or notification as a reaction to the generated control command.

[0014] In another embodiment, the vehicle's reaction to a traffic event is indicated to the user of the VR device in the form of activation of an emergency brake assist device, an evasive maneuver of the vehicle, arrival at or approaching a destination location, triggered fatigue recognition, or an object recognized by a vehicle sensor. In such a traffic event, the vehicle may initiate a braking action or an evasive action as a reaction, which is perceived by the user of the VR device suddenly and without prior warning. Through this method, such a reaction of the vehicle can be taken into account in advance and the user of the VR device can be notified. Such control of the VR device enables the user to be warned in advance or to adapt to the impending vehicle reaction, thereby improving the user's driving comfort.

[0015] If the driver is identified as fatigued by the vehicle's fatigue detection, the user of the VR device can be directly or indirectly informed, which can, for example, help the vehicle driver stay awake and active. For example, the user of the VR device can be suggested a media product that requires collaboration with other non-VR users (such as the driver or other vehicle occupants). For example, such VR media content can be designed as a quiz.

[0016] The user of the VR device can preferably be a vehicle occupant sitting in the back seat or the front passenger seat of the vehicle. Here, the vehicle can be controlled automatically, semi-automatically or manually by the driver. The user of the VR device is preferably excluded from controlling the vehicle.

[0017] According to another embodiment, a control command for controlling the VR device is generated, which control command signals a communication desire of a vehicle occupant with a user of the VR device.

[0018] In another embodiment, the communication request of the vehicle occupant and the user of the VR device is implemented by a vehicle-side input of the vehicle occupant.

[0019] In the event of a communication request, at least one vehicle occupant can implement an input via the vehicle's media controls to discreetly communicate the communication request to the user of the VR device. This can be accomplished, for example, by activating a button in the graphical interface of the media controls, by pressing a key, or by using a voice-enabled vehicle assistant.

[0020] The user of the VR device can be notified via the communication connection. The notification or indication of a need to communicate can be displayed as a text message in the VR device, played as a voice message through the earpiece, or signaled by deactivating the earpiece's noise suppression. Indirect communication with the vehicle via the communication connection can prevent the user of the VR device from being startled, as opposed to direct contact with the user of the VR device (e.g., by touch).

[0021] According to another embodiment, the control commands for controlling the VR device are generated by a vehicle-side control device and / or by a processor unit of the VR device. This allows the calculation of the control commands and the classification of at least one traffic event to be performed on the vehicle side or by the VR device. In particular, the VR device can use calculations, warnings, and calculation results already performed by the vehicle to reduce the power requirements of the VR device and increase its battery life.

[0022] According to another embodiment, vehicle data and / or evaluated vehicle data are transmitted between the vehicle-side control unit and the VR device via a communication connection designed as a Bluetooth connection or a WLAN connection. This approach allows a communication connection to be established based on various wireless transmission standards, which allows the transmission of vehicle data for a VR experience relevant to driving dynamics.

[0023] The communication connection can preferably be implemented as a bidirectional data channel between the vehicle or a vehicle-side control device and the VR device. The corresponding communication connection can be based on one or more vehicle-internal or vehicle-external transmission technologies, such as Bluetooth, WLAN, LTE, 5G, etc.

[0024] The vehicle system according to the present invention comprises a vehicle having a vehicle-side control device and at least one VR device. The vehicle system is designed to carry out the method according to the present invention.

[0025] The vehicle system can use a VR device, such as VR glasses or a VR headset, in the back seat or passenger seat of the vehicle, regardless of whether the vehicle is under automatic or manual control. The user of the VR device can be informed of unexpected vehicle reactions to traffic events, thereby preventing the user from being startled or surprised and improving driving comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention is schematically illustrated in the accompanying drawings according to embodiments and will be further described with reference to the accompanying drawings, in which:

[0027] Figure 1 A vehicle system according to an embodiment according to the present invention is shown. DETAILED DESCRIPTION

[0028] Figure 1 A vehicle system 1 according to an embodiment of the present invention is shown. The vehicle system 1 comprises a vehicle 2 which can be designed as an automatically operable vehicle or a manually controlled vehicle. Two vehicle occupants 6 , 8 are exemplarily located in a vehicle interior 4 of the vehicle 2 .

[0029] The first vehicle occupant is the driver of vehicle 2. A second vehicle occupant 8 wears a virtual reality headset or VR device 10. The second vehicle occupant 8 is therefore a user 8 of the VR device 10. The first vehicle occupant 6 is a so-called non-VR user.

[0030] A data-carrying communication connection 14, for example based on the Bluetooth transmission standard, is established between the vehicle-side control unit 12 and the VR device 10. Vehicle data, or evaluated vehicle data, can be received and evaluated via the communication connection 14. Vehicle data can be determined or received, for example, by vehicle sensors 16. Route guidance information in the form of vehicle data can also be used.

[0031] The vehicle sensors 16 may include, for example, camera sensors, lidar sensors, radar sensors, and the like.

[0032] Based on the evaluated vehicle data, a traffic situation or traffic event can be classified by the vehicle-side control device 12 or by the processor of the VR device 10. Based on the traffic event, the reaction of the vehicle 2 to the traffic event can be estimated.

[0033] For example, a critical driving situation can be classified as a possible traffic incident. In this critical driving situation, if the safety distance of vehicle 2 is exceeded, an evasive maneuver or emergency braking action is initiated. This driving action can be automatically initiated by vehicle-side control unit 12 or by the emergency brake assist function, or by driver 6. The corresponding driving situation is usually pre-determined using vehicle sensors 16.

[0034] After the traffic event is classified, a control command is generated for controlling VR device 10 to indicate to user 8 of VR device 10 that the vehicle 2 or driver 6 reacted unexpectedly to the traffic event, thereby preventing user 8 from being startled. To this end, a warning message can be displayed in the VR display of VR device 10 via the generated control command.

[0035] Depending on the design of VR device 10, the see-through function can be activated via generated control commands. To position VR device 10 in space, some VR devices include camera systems that detect and evaluate the surrounding environment. Instead of a VR display, the see-through function displays the video stream from these camera systems, allowing user 8 to see behind VR device 10. Thus, the see-through function can be activated and deactivated in a traffic-driven manner to prepare user 8 for possible reactions of vehicle 2.

[0036] In another exemplary traffic event, vehicle 2 arrives at a destination. This can be recorded based on vehicle data in the form of navigation data. Upon approaching the destination, a corresponding indication can be displayed in VR device 10 so that user 8 can prepare to end VR use.

[0037] List of reference numerals:

[0038] 1Vehicle System

[0039] 2 vehicles

[0040] 4. Vehicle interior space

[0041] 6. First occupant / driver of the vehicle

[0042] 8. Second vehicle occupant / VR device user

[0043] 10VR devices / virtual reality headsets

[0044] 11Processor unit of VR device

[0045] 12 Vehicle-side control equipment

[0046] 14 Communication Connections

[0047] 16 vehicle sensors

Claims

1. A method for controlling a VR device (10) of a user (8) in a vehicle interior (4) of a vehicle (2) via a communication connection (14), wherein: Receiving vehicle data via a communication connection (14) and evaluating the vehicle data, or receiving evaluated vehicle data and classifying at least one traffic event based on the evaluated vehicle data, characterized in that a control command for controlling a VR device (10) is generated, by which a user (8) of the VR device (10) is indicated that the vehicle (2) reacted to the traffic event in an unexpected manner for the user (8), wherein the control command generated for controlling the VR device (10) signals a communication requirement of a vehicle occupant (6) with the user (8) of the VR device (10), wherein the vehicle occupant (6) is a non-VR user.

2. The method according to claim 1, wherein The generated control commands are used to output a message in the display of the VR device (10), to activate the transparency of the VR device (10), to output an acoustic notification or a graphic notification, to pause or terminate an application of the VR device (10) and / or to cancel a noise suppression function.

3. The method according to claim 1 or 2, wherein: The form of the vehicle's (2) reaction to the traffic event indicated to the user (8) of the VR device (10) is: activation of an emergency brake assist device, an evasive maneuver of the vehicle (2), arrival at or approaching a destination location, triggered fatigue recognition, or an object recognized by a vehicle sensor (16).

4. The method according to claim 1, wherein The communication needs of the vehicle occupant (6) and the user (8) of the VR device (10) are realized through the vehicle-side input of the vehicle occupant (6).

5. The method according to claim 1 or 2, wherein: Control commands for controlling the VR device (10) are generated by a vehicle-side control device (12) and / or by a processor unit (11) of the VR device (10).

6. The method according to claim 5, wherein: Vehicle data and / or evaluated vehicle data are transmitted between a vehicle-side control device (12) and a VR device (10) via a communication connection (14) designed as a Bluetooth connection or a WLAN connection.

7. A vehicle system (1) comprising a vehicle (2) having a vehicle-side control device (12) and at least one VR device (10), wherein: The vehicle system (1) is designed to carry out the method according to any of the preceding claims.

Images