METHOD FOR OPERATING A HEAD-WEARABLE ELECTRONIC DISPLAY DEVICE AND DISPLAY SYSTEM FOR DISPLAYING VIRTUAL CONTENT

Abstract

A method for operating a head-mounted electronic display device that allows a driver to easily perceive the physical limitations inside a vehicle, and a display system for displaying virtual content are provided. [Solution] In the display system 1, the method of operating the head-worn electronic display device 2 involves using a detection device 3 arranged on the electronic display device 2 to determine the relative movement of the electronic display device 2 with respect to the interior of the vehicle, and adapting the virtual viewpoint inside the vehicle interior where the wearer wearing the electronic display device 2 is located to the displayed virtual content according to the determined relative movement, at which time the detection device 3 detects the contours inside the vehicle interior and permanently displays a virtual contour corresponding to the contours inside the vehicle interior.

Description

[Technical Field]

[0001] The present invention relates to a method of operating a head-worn electronic display device and a display system for displaying virtual content. [Background technology]

[0002] In the future, many people will increasingly demand the use of immersive technology in their cars, both for driving support and for entertainment and relaxation while driving. Vehicle occupants will use virtual or augmented reality technologies to play media, consume media, or relax. It is important that each user knows the physical limits of the vehicle that actually surround them, especially when it comes to playful experiences inside the car and interactions with other virtual or real people. These limits can vary significantly depending on the type of vehicle, making it prohibitively expensive to provide data about the interior of such vehicles.

[0003] The use of virtual reality glasses in vehicles is shown, for example, in US Pat. No. 5,629,499. The focus of this publication is to prevent so-called motion sickness in the wearer of the virtual reality glasses. This is achieved, for example, in that the environment around the vehicle is recorded by a camera system and then incorporated using the virtual reality glasses.

[0004] Patent document 2 also focuses on preventing so-called motion sickness, proposing a method for displaying the vehicle surroundings using a display device arranged in the rear seat area, thereby preventing passengers in the rear seat area from feeling sick.

[0005] Finally, US Pat. No. 5,629,499 also shows a method for operating virtual reality glasses while driving a vehicle, which aims to present an image of the real vehicle's surroundings in addition to the virtual content itself. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] German Patent Application Publication No. 102015003882 [Patent Document 2] U.S. Patent Application Publication No. 2006 / 0015000 [Patent Document 3] German Patent Application Publication No. 102014003178 Summary of the Invention [Problem to be solved by the invention]

[0007] The object of the present invention is to provide a method for operating a head-worn electronic display device and a display system for displaying virtual content, which allows the wearer of such an electronic display device to perceive the physical limits inside a vehicle in a particularly simple way. [Means for solving the problem]

[0008] This object is achieved by a method for operating a head-worn electronic display device and is solved by a display system for displaying virtual content with the features of the independent claims. Advantageous embodiments with advantageous and important developments of the invention are specified in the remaining claims.

[0009] In a method for operating a head-worn electronic display device according to the present invention, a relative movement of the electronic display device with respect to a vehicle interior is determined by a detection device arranged on the electronic display device, and a virtual viewpoint of a wearer of the electronic display device in the vehicle interior is adapted to the displayed virtual content according to the determined relative movement, and a contour of the vehicle interior is detected by the detection device, and a virtual contour corresponding to the contour of the vehicle interior is displayed.

[0010] According to the present invention, so-called inside-out tracking is performed by a detection device located on the electronic display device. In inside-out tracking, the movement of the electronic display device is recorded by a detection device attached to the electronic display device itself. The detection device is designed to detect the environment of the head-worn electronic display device and thereby estimate the movement of the electronic display device relative to the environment, i.e., relative to the vehicle interior in a particular application.

[0011] According to the present invention, the detection device designed for inside-out tracking is further used to detect the contours of the vehicle interior, and a virtual contour corresponding to the contours of the vehicle interior is then displayed by the head-worn electronic display device. Thus, the detection device attached to the electronic display device can recognize the shape or structure of the vehicle interior, which can then be implemented in respective virtual or augmented reality applications by the electronic display device.

[0012] The virtual contour corresponding to the detected contour of the vehicle interior does not have to be exactly the same as the real one, the only important thing is that the virtual contour corresponds in terms of its shape to the detected contour of the vehicle interior. In other words, the method according to the invention makes it possible to use the electronic display device while the wearer of the electronic display device is sitting inside the vehicle to display his surroundings in the vehicle as a kind of structure or relief in addition to the virtual content already displayed.

[0013] Regardless of the type of vehicle the wearer of the electronic display device is currently sitting in, the method according to the present invention allows the wearer of the electronic display device to be aware of the physical limits of the vehicle that actually surrounds the display device at all times, even while the electronic display device is being worn. Whether the wearer of the electronic display device interacts with other virtual characters or similarly with people who are actually inside the vehicle, the wearer can be aware of the physical limits of the vehicle interior at all times, or at least as needed, even while wearing the electronic display device.

[0014] In an advantageous embodiment of the invention, the virtual contour is permanently displayed, so that the wearer of the electronic display device is aware at all times of how far he is from the actual physical limits of the vehicle interior, despite wearing the electronic display device, and, among other things, this ensures that the wearer of the electronic display device does not collide with the contours of the vehicle interior.

[0015] According to another advantageous embodiment of the present invention, the virtual contour is displayed only when a wearer of the electronic display device with a specific body part is located within a predetermined distance from the contour of the vehicle interior. The specific body part can be, for example, the head and / or hands of the wearer of the electronic display device. In this advantageous embodiment, the virtual contour is displayed only when a wearer of the electronic display device with a specific body part is located within a predetermined distance from the contour of the vehicle interior, supporting a particularly immersive presentation of virtual content. As long as the wearer of the electronic display device with a specific body part is sufficiently far from the contour of the vehicle interior, the virtual contour corresponding to the detected contour of the vehicle interior is not displayed. As soon as the wearer's specific body part gets too close to the actual contour of the vehicle interior, the electronic display device displays a virtual contour corresponding to the contour of the vehicle interior. This also ensures that the wearer of the electronic display device does not collide with the actual contour of the vehicle interior.

[0016] In a further advantageous embodiment of the invention, the detection device comprises a sensor device based on transit time measurement, by which the contour of the vehicle interior is detected. A time-of-flight camera, for example, can be used as the sensor device based on transit time measurement. The transit time measurement allows the actual contour of the vehicle interior to be recorded in a particularly accurate and reliable manner. As a result, a virtual contour can also be generated at least more or less one-to-one with the actual contour of the vehicle interior and can then be displayed by the electronic display device. This particularly reliably prevents the wearer of the electronic display device from colliding with the actual contour of the vehicle interior. In other words, the transit time measurement allows the actual contour of the vehicle interior to be determined in a particularly accurate manner. Thus, the wearer of the electronic display device can always determine how far away he or she is from the actual contour of the vehicle interior by means of the virtual contour displayed by the display device he or she is wearing, at least when necessary.

[0017] In a further advantageous embodiment of the invention, the detection device comprises a camera, which records the interior of the vehicle and detects the interior contour on the basis of the recording. The use of conventional cameras has the advantage that they are, for example, usually much cheaper than time-of-flight cameras. Using conventional cameras alone reduces costs considerably. Alternatively, it is possible to combine runtime measurements with conventional cameras. For example, if desired, the virtual contour can actually be displayed as in the real vehicle interior. This can be done by combining transit time measurements with camera images. Three-dimensional data is generated by transit time measurements. Data actually relating to the optical system is provided by the conventional camera.

[0018] In a further advantageous embodiment of the present invention, a game is displayed by the electronic display device, and a virtual contour corresponding to the contour of the vehicle interior is adapted to the current scene of the game. For example, if the wearer of the electronic display device is standing on a bridge in the virtual environment displayed by the electronic display device, a virtual balustrade is displayed that corresponds to a detected contour of the vehicle interior, such as the contour of the dashboard. Apart from the structure of the vehicle interior, the visual appearance of this virtual balustrade can be adapted to the currently displayed game scene. Similarly, other virtual content, such as media, can also be displayed by the display device, and a virtual contour corresponding to the contour of the vehicle interior is displayed in a manner adapted to the respective virtual content. As a result, the sense of immersion when displaying a wide variety of content by the electronic display device is supported or significantly improved.

[0019] A display system according to the present invention for displaying virtual content comprises a head-worn electronic display device, a detection device arranged on the electronic display device and designed to determine a relative movement of the electronic display device with respect to the interior of a vehicle, and a control device configured to adapt a virtual viewpoint of a wearer of the electronic display device, who is present in the interior of the vehicle, to the displayed virtual content according to the determined relative movement. The detection device is also designed to detect contours of the interior of the vehicle, and the control device is configured to control the electronic display device to display a virtual contour corresponding to the contours of the interior of the vehicle. Advantageous improvements according to the method of the present invention should be considered advantageous improvements according to the display device of the present invention, and vice versa. The display system in particular comprises means for carrying out the method steps. The electronic display device may be, for example, virtual reality glasses or augmented reality glasses. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a schematic diagram of a display system for displaying virtual content, the display system including a head-worn electronic display device, a detection device disposed on the electronic display device, and a control device. [Figure 2]1 is a schematic side view of a motor vehicle in which a wearer of an electronic display device is seated; [Figure 3] FIG. 1 is a schematic diagram of virtual content displayed by an electronic display device. DETAILED DESCRIPTION OF THE INVENTION

[0021] Further advantages, features and details of the invention arise from the following description of preferred exemplary embodiments and from the drawings, in which: the features and combinations of features described above, and / or described below with reference to the drawings, and / or depicted only in the drawings, can be used not only in the combinations shown in each case, but also in other combinations or alone, without departing from the scope of the invention.

[0022] In the drawings, identical or functionally identical elements are given the same reference symbols.

[0023] A display system 1 for displaying virtual content is shown in the schematic diagram of Fig. 1. The display system 1 comprises an electronic display device 2, which can be worn on the head and which can be, for example, virtual reality glasses or augmented reality glasses. The display system 1 also comprises a detection device 3 arranged on the electronic display device 2. The detection device 3 can, for example, comprise a time-of-flight camera, and alternatively or additionally, the detection device 3 can, for example, comprise a conventional camera. Furthermore, the display system 1 also comprises a control device 4 which is data-technically connected to the electronic display device 2 and to the detection device 3.

[0024] 2 shows a vehicle 5 in a schematic side view, with a wearer 6 wearing an electronic display device 2 while driving the vehicle 5. The relative movement of the electronic display device 2 with respect to the interior 7 of the vehicle 5 is determined by a detection device 3 arranged on the electronic display device 2 and not shown here. According to the determined relative movement, the virtual viewpoint of the wearer 6 of the electronic display device 2 arranged in the interior 7 of the vehicle is adapted to the virtual content 9 displayed by the electronic display device 2. In other words, the detection device 3 is used for so-called inside-out tracking of the electronic display device 2.

[0025] Said virtual content 9 is shown in a very schematic way in Fig. 3. The contour 8 of the vehicle interior 7, here represented diagrammatically by a steering wheel, is also detected by the detection device 3. Then, in addition to the actual virtual content 9, a virtual contour 10 corresponding to the contour 8 of the vehicle interior 7 is displayed.

[0026] The virtual contour 10 can be permanently displayed. Alternatively, however, the virtual contour 10 can be displayed only when the wearer 6 of the electronic display device 2, for example, brings his / her head or hand sufficiently close to the actual contour 8 of the vehicle interior 7. For example, a certain distance can be specified, and as soon as it is detected that the distance to the actual contour 8 of the vehicle interior 7 falls below this specified distance, the virtual contour 10, which corresponds geometrically to the actual contour 8 of the vehicle interior 7, is displayed. Regardless of whether the virtual contour 10 is permanently displayed or only displayed as needed, it is essential that the virtual contour 10 corresponds in terms of its structure or its shape and dimensions to the actual contour 8 of the vehicle interior 7. Otherwise, the superimposed virtual contour 10 would not look like the actual contour 8 of the vehicle interior 7 purely optically, but would look completely different.

[0027] For example, a game can be displayed as virtual content 9 using the electronic display device 2 while driving the vehicle 5. A virtual contour 10, which corresponds to the actual contour 8 of the vehicle interior 7, is adapted to the current scene in the game. This means that the superimposed virtual contour 10 does not look like the vehicle interior 7, but instead the virtual contour 10 only corresponds in terms of structure or shape to the actual contour 8 of the vehicle interior 7. The rest of the optical appearance of the virtual contour 10 is then adapted to the currently displayed or currently played game, as displayed by the electronic display device 2.

[0028] Thus, the wearer 6 can view any virtual content via the electronic display device 2. The actual outline 8 of the vehicle interior 7 is either permanently displayed or, if necessary, is identified to the wearer 6 by the electronic display device 2 fading in a virtual outline 10.

[0029] The detection device 3 of the display system 1 therefore has a dual function: firstly, it serves to perform the so-called position tracking of the electronic display device 2 by means of the inside-out tracking already mentioned; secondly, it also assumes the function of detecting the actual contour 8 of the vehicle interior 7, so that the electronic display device 2 can also display said virtual contour 10 corresponding to the contour 8 of the vehicle interior 7 in the manner described above.

[0030] Thus, the wearer 6 of the electronic display device 2 is always aware of the physical limits of the vehicle 5 surrounding the electronic display device 2, at least when necessary, i.e., when the electronic display device 2 is in close proximity. In addition to actually detecting the actual contour 8 of the vehicle interior 7, the detection device 3 is also preferably able to detect, for example, further occupants not shown here. In this regard, the electronic display device 2 can also display virtual contours corresponding to the detected occupants. In this case, the wearer 6 of the electronic display device 2, when wearing the electronic display device 2, can avoid colliding with not only the actual contour 8 of the vehicle interior 7 but also other occupants.

Claims

1. A method for operating a head-worn electronic display device (2), comprising: determining, by a detection device (3) arranged on the electronic display device (2), a relative movement of the electronic display device (2) with respect to a cabin interior (7) of a vehicle (5) being driven; and adapting a virtual viewpoint of the cabin interior (7) in which a wearer wearing the electronic display device (2) is present to the displayed virtual content (9) according to the determined relative movement of the electronic display device (2) with respect to the cabin interior (7) of the vehicle (5) being driven; and at the same time, detecting a contour (8) of the cabin interior (7) using the detection device (3), as well as detecting a further occupant, and permanently displaying the contour (8) of the cabin interior (7) and a virtual contour (10) corresponding to the detected occupant.

2. 2. A method for operating a head-mounted electronic display device according to claim 1, characterized in that the detection device (3) has a sensor device based on measuring the passage time, and this sensor device detects the contour (8) of the vehicle interior (7).

3. 3. The method of claim 2, wherein a time-of-flight camera is used as the sensor device based on measuring transit time.

4. A method for operating a head-worn electronic display device as claimed in any one of claims 1 to 3, characterized in that the detection device (3) has a camera, which records the interior of the vehicle (7) and detects the contour (8) of the interior of the vehicle (7) based on the recording of the interior of the vehicle (7).

5. 5. A method for operating a head-worn electronic display device according to claim 1, further comprising the step of displaying a game on the electronic display device (2) and adapting a virtual contour (10) corresponding to the contour (8) of the vehicle interior (7) to the current scene of the game.

6. A display system (1) for displaying virtual content (9), comprising: an electronic display device (2) that can be worn on the head; a detection device (3) arranged on the electronic display device (2) and designed to determine the relative movement of the electronic display device (2) with respect to the interior (7) of the vehicle (5) during operation; a control device (4) configured to adapt the field of view of a wearer (6) of the electronic display device (2) present in the vehicle interior (7) to the displayed virtual content (9) according to the determined relative movement of the electronic display device (2) with respect to the vehicle interior (7) of the vehicle (5) being driven; said detection device (3) being designed to detect the contour (8) of the interior of the vehicle (7) as well as to detect further occupants; and The display system for displaying virtual content is characterized in that the control device (4) is configured to control the electronic display device (2) to persistently display an outline (8) of the vehicle interior (7) and a virtual outline (10) corresponding to a detected occupant.

7. 7. A display system for displaying virtual content according to claim 6, characterized in that the electronic display device (2) is a virtual reality or augmented reality glass.

Images