Method for operating virtual reality glasses and system with virtual reality glasses

Abstract

Method for operating virtual reality glasses (12), comprising the steps: - displaying at least one virtual object (24) arranged in a virtual environment (22) by means of the virtual reality glasses (12) from a first observation position (A) predetermined within the virtual environment (22); - displaying at least one position symbol (28) at a position within the virtual environment (22) which corresponds to a second observation position (B) predetermined within the virtual environment (22); - selecting the displayed position symbol (28) as soon as a predetermined selection action has been detected; - displaying the virtual object (24) from the second observation position (B) as soon as a predetermined confirmation action with respect to the selected position symbol (28) has been detected, characterized in that - the position symbol (28) is only displayed within the virtual environment (22) when an actuating element (20) is touched, and / or that - a respective virtual viewing angle from which the virtual object (24) is displayed starting from the respective observation positions (A, B) is changed depending on a detected pivoting position of a head of a wearer (14) of the virtual reality glasses (12).

Description

[0001] The invention relates to a method for operating virtual reality glasses and a system with virtual reality glasses.

[0002] Virtual reality glasses are a specific type of head-mounted display, a visual output device worn on the head. They present images on a screen positioned close to the eyes or project them directly onto the retina. Virtual reality glasses also have sensors for detecting head movements. This allows the display of the calculated graphics to be adapted to the wearer's movements. Due to the physical proximity, the image areas displayed by head-mounted displays appear considerably larger than those of free-standing screens and, in extreme cases, can even cover the user's entire field of vision. Since the display follows all head movements based on the wearer's head position, they have the feeling of moving directly through a computer-generated image landscape.

[0003] Using such virtual reality glasses, a virtual reality can be presented, whereby virtual reality is usually referred to as the representation and simultaneous perception of reality in its physical properties in a real-time computer-generated, interactive virtual environment.

[0004] In a virtual reality environment, a particular challenge is providing a suitable interaction option for determining or changing the virtual position of the wearer of such glasses in relation to an object displayed by the virtual reality glasses. In particular, free movements within the displayed virtual reality, for example, induced via a controller or joystick, can quickly lead to nausea in the wearer of virtual reality glasses, as a contradiction arises during the virtual movement between the visual impressions and the lack of movement feedback from the vestibular system in the inner ear.

[0005] US 2011 / 0175932 A1 discloses a method for interacting with a graphical user interface, which is displayed, for example, using a head-mounted display. Objects of the graphical user interface are modified depending on the viewing direction and one or more detected inputs. For example, depending on a detected viewing direction and an input on an input device, a portion of the displayed graphical user interface is enlarged. As soon as another input is subsequently detected on the input device, a context-dependent action is performed.

[0006] US Pat. No. 7,928,926 B2 discloses a method for operating a head-mounted display. Depending on a detected swivel movement of the head of a wearer of the head-mounted display, a virtual viewing direction to the respective positions of objects displayed by the head-mounted display is changed. For example, if the user turns their head to the right, objects displayed by the head-mounted display are moved to the left, i.e., opposite to the detected swivel movement of the head of the wearer of the head-mounted display.

[0007] US 2007 / 0146390 A1 discloses a method for operating a head-mounted display. The position and orientation of a virtual object displayed by the head-mounted display are changed based on a detected movement of an input device.

[0008] The publication "Virtual and Augmented Reality" (ISBN 978-3-642-28903-3) describes the application of gaze tracking in virtual reality headsets. For example, a camera attached to a pair of virtual reality headsets captures the eyes of a wearer, and based on this, the wearer's gaze direction is determined.

[0009] US 2004 / 0027394 A1 describes a virtual reality device comprising a display element projecting a virtual environment and a plurality of waypoint elements, each defined by its own waypoint position. A user can automatically switch to one of the waypoint positions by simply selecting the corresponding waypoint element.

[0010] It is therefore the object of the present invention to provide a method for operating virtual reality glasses and a system with virtual reality glasses, by means of which an improved change of the virtual position of a wearer of the glasses within a virtual environment displayed by means of the virtual reality glasses is made possible.

[0011] This object is achieved by a method for operating virtual reality glasses and by a system comprising virtual reality glasses having the features of the independent patent claims. Advantageous embodiments with expedient and non-trivial refinements of the invention are specified in the dependent claims.

[0012] In the method according to the invention for operating virtual reality glasses, at least one virtual object arranged in a virtual environment is displayed by means of the virtual reality glasses from a first observation position predetermined within the virtual environment. Furthermore, at least one position symbol is displayed at a position within the virtual environment that corresponds to a second observation position predetermined within the virtual environment. The displayed position symbol is selected as soon as a predetermined selection action has been detected. The virtual object is displayed from the second observation position as soon as a predetermined confirmation action with respect to the selected position symbol has been detected.

[0013] In other words, the invention therefore provides that a wearer of virtual reality glasses can move, in particular jump, from a predetermined observation position to another observation position within the virtual environment by selecting a correspondingly displayed position symbol and confirming its selection. In doing so, one or more possible jump positions can be displayed to the wearer in the form of the displayed position symbol or further position symbols in their spatially correct position within the virtual environment. In other words, the predetermined observation positions are virtual viewpoints at which the wearer of the virtual reality glasses is virtually located within the displayed virtual environment, wherein the virtual reality glasses orwhose display device is controlled in such a way that the wearer of the virtual reality glasses has the impression that he is looking at the virtual object from the correspondingly specified observation position.

[0014] The problem mentioned above, which can occur during free movement within a virtual reality environment, is remedied by the inventive solution, since the wearer of the virtual reality glasses no longer moves freely within the displayed virtual environment according to their visual perception. Instead, the wearer of the virtual reality glasses can jump from one virtual observation position to the next using the inventive method, thereby abruptly changing their virtual viewpoint within the displayed virtual environment and thus also abruptly changing the perspective on the currently displayed virtual object. This prevents a contradiction from occurring between the visual impressions and the actual movement feedback from the vestibular system of the wearer of the virtual reality glasses, thus preventing the onset of nausea in the wearer of the virtual reality glasses.

[0015] Furthermore, the invention provides that the position symbol, preferably also the selection symbol, is only displayed within the virtual environment when an actuating element, in particular a button arranged on a remote control coupled to the virtual reality glasses, is touched. In other words, a type of movement mode is only activated when the actuating element is touched, which is characterized in that the position symbol, in particular also the selection symbol, is only displayed when a touch of the actuating element is detected. This makes it easy for a wearer of the virtual reality glasses to see whether or not they can currently perform a virtual position change within the displayed virtual environment.

[0016] Alternatively or additionally, the invention provides that the respective virtual viewing angle from which the virtual object is displayed based on the respective observation positions is changed depending on a detected pivoting position of the head of the wearer of the virtual reality glasses. The user can thus change their respective virtual viewing angle of the currently displayed virtual object by pivoting their head up or down or left or right. This results in particularly simple control of the virtual reality glasses, which changes the viewing angle of the currently displayed virtual object according to the pivoting movement of the wearer's head.

[0017] In an advantageous embodiment of the invention, the selection of the displayed position symbol occurs as soon as the predetermined selection action has been detected, namely that a selection symbol displayed within the virtual environment has been at least partially overlapped with the displayed position symbol. The wearer of the virtual reality glasses can control the movement of the selection symbol accordingly, thus effecting the selection of the displayed position symbol in a particularly simple and convenient manner. The displayed selection symbol, which can also be understood as a selection symbol, can be, for example, a crosshair, a mouse pointer, or the like.

[0018] In a further advantageous embodiment of the invention, it is provided that the selection symbol is moved within the virtual environment depending on a detected head movement of a wearer of the virtual reality glasses. For example, the virtual reality glasses can have a combination of a 3-axis gyrometer and acceleration sensors, by means of which the head movement of the wearer of the virtual reality glasses can be reliably detected. Because the selection symbol is moved within the virtual environment depending on the detected head movement of the wearer, the selection symbol can be moved within the virtual environment by the wearer of the virtual reality glasses in a particularly simple and convenient manner.

[0019] According to a further advantageous embodiment of the invention, it is provided that the selection symbol is moved within the virtual environment as a function of a detected eye movement of the wearer of the virtual reality glasses. For this purpose, the virtual reality glasses can, for example, have a detection device designed to detect the eye movements, in particular the viewing direction of the wearer of the virtual reality glasses. An eye movement-controlled movement of the selection symbol within the virtual environment has the particular advantage that the selection symbol can be moved particularly quickly and easily. The movement of the selection symbol can also occur as a function of both the detected head movement and the detected eye movement of the wearer of the virtual reality glasses.This provides a particularly convenient and intuitive way for the wearer of the virtual reality headset to control the selection symbol. The wearer of the virtual reality headset simply needs to look at the relevant position symbol displayed in the virtual environment to select the position symbol.

[0020] A further advantageous embodiment of the invention provides that the position symbol is hidden from the virtual environment as soon as the touch of the actuating element is released. Preferably, the selection symbol is also hidden from the virtual environment as soon as the touch of the actuating element is released. The wearer of the virtual reality glasses must therefore, for example, continuously keep a finger on the said actuating element in order to keep the movement mode activated. This results in a particularly conscious activation and deactivation of the movement mode. Therefore, as soon as the wearer of the virtual reality glasses does not wish to change their virtual position within the virtual environment and thus also their viewing angle of the displayed virtual object, they can simply, for example, remove their finger from the previously touched actuating element.The position symbol and preferably also the selection symbol are then hidden and the wearer of the virtual reality glasses can view the displayed virtual object without any disturbing influences.

[0021] In a further advantageous embodiment of the invention, the virtual object is only displayed from the second observation position when the predetermined confirmation action has been detected: that an actuating element, in particular a button on a remote control coupled to the virtual reality glasses, has been pressed. The actuating element is preferably the same actuating element by means of which the virtual position change is confirmed and the position symbol, in particular also the selection symbol, is displayed. For example, the remote control, which comprises the said button, can be designed such that it can be operated like a ballpoint pen. As soon as the wearer of the virtual reality glasses wishes to make a virtual position change, they press the virtual button like a ballpoint pen.If he only wants the position symbol, especially the selection symbol, to be displayed, he simply touches the button, as if he were only touching the top of a ballpoint pen and not pressing it.

[0022] Preferably, pressing the actuating element is distinguished from touching the actuating element by the fact that a greater stroke is performed on the actuating element when pressing and / or greater pressure is exerted on the actuating element than when touching. For example, the actuating element can comprise both capacitive sensors and pressure sensors. Alternatively or additionally, it is also possible for the actuating element to have one or more corresponding displacement sensors, by means of which the key stroke on the actuating element can be detected. The distinction between a mere touch and a pressing of the actuating element can thus be made reliably.

[0023] Preferably, a motor vehicle is displayed as the virtual object, with the position symbol being arranged and displayed outside or inside the motor vehicle. The wearer of the virtual reality glasses can thus jump from one virtual observation position to the next and view the virtual motor vehicle from a variety of perspectives. They can also jump to a virtual observation position inside the motor vehicle, for example, to view the interior from the driver's seat or from other seating positions within the motor vehicle. This makes it possible, for example, in sales outlets orIt is no longer necessary for motor vehicle showrooms to maintain a wide variety of different vehicle variants with a wide range of special equipment, as a potential customer can easily view differently equipped vehicle variants from the inside and outside as well as from a wide range of angles using virtual reality glasses.

[0024] The system according to the invention comprises virtual reality glasses which are designed to display at least one virtual object arranged in a virtual environment from a first observation position. Furthermore, the virtual reality glasses are designed to display at least one position symbol within the virtual environment at a position within the virtual environment which corresponds to a second observation position within the virtual environment. Furthermore, the system according to the invention has a control device which is designed to select the displayed position symbol as soon as a predetermined selection action has been detected. The control device is also designed to control the virtual reality glasses such that the virtual object is displayed from the second observation position as soon as a predetermined confirmation action with regard to the selected symbol has been detected.Advantageous embodiments of the method according to the invention are to be regarded as advantageous embodiments of the system according to the invention, wherein the system in particular comprises means for carrying out the method steps.

[0025] Further advantages, features, and details of the invention will become apparent from the following description of preferred embodiments and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combinations, but also in other combinations or on their own, without departing from the scope of the invention.

[0026] Embodiments of the invention are explained in more detail below with reference to schematic drawings. They show: Fig. 1 a schematic representation of a system for displaying a virtual environment, the system comprising virtual reality glasses worn by a user, a control device for controlling the virtual reality glasses and a remote control for controlling the image content displayed by means of the virtual reality glasses; Fig. 2 a schematic representation of a virtual environment within which a virtual motor vehicle is displayed to the wearer of the virtual reality glasses from a first virtual observation position; Fig. 3 a schematic representation in which the virtual observation position of the wearer of the virtual reality glasses on the virtual motor vehicle, as in Fig. 2 is shown, displayed; Fig. 4 again the virtual motor vehicle within the virtual environment, additionally showing an arrowhead-shaped selection symbol and a position symbol represented as a solid circle; Fig. 5 essentially the same representation as in Fig. 4, where the selection symbol has been brought into overlap with the circular position symbol; Fig. 6 a further representation of the virtual environment, wherein the virtual motor vehicle is now displayed from a second virtual observation position.

[0027] In the figures, identical or functionally identical elements are provided with the same reference symbols.

[0028] A system for displaying a virtual environment, designated 10 in total, is in Fig. 1. The system 10 comprises virtual reality glasses 12, which are configured to display a virtual environment and one or more virtual objects within the virtual environment. The virtual reality glasses 12 can, for example, have two individual displays arranged in front of the respective eyes of a wearer 14 of the virtual reality glasses 12, so that, with the aid of the two displays, a spatial effect can be induced when displaying the virtual environment for the wearer 14.

[0029] Furthermore, the virtual reality glasses 12 can comprise a combination of a 3-axis gyrometer and acceleration sensors, by means of which, in particular, pivoting movements of the head of the wearer 14 about respective spatial axes x1, y2, and z1 can be detected. As a result, a respective virtual viewing angle of the wearer 14, from which one or more virtual objects within the virtual environment are displayed, can be changed from respective observation positions within the virtual environment depending on a detected pivoting position of the head of the wearer 14 of the virtual reality glasses 12.

[0030] The system 10 further comprises a control device 16, which is informationally coupled to the virtual reality glasses 12. The control device 16 can, for example, be a computer with appropriate software installed thereon, so that the control device 16 can control the display of the virtual environment using the virtual reality glasses 12.

[0031] Furthermore, the system 10 also has a remote control 18, which is informationally coupled, preferably via a radio link, to both the virtual reality glasses 12 and the control device 16. Using the remote control 18, the wearer 14 can control the virtual reality glasses 12, for example, by pressing a button 20 on the remote control 18, in order to change the displayed virtual content accordingly.

[0032] In Fig. 2, a virtual environment 22 is shown, wherein a virtual motor vehicle 24 is displayed within the virtual environment 22 by means of the virtual reality glasses 12. X2 and z2, as well as y2, denote corresponding coordinate axes within the virtual environment 22. The letter A denotes a first observation position, i.e., a type of virtual viewpoint, within the virtual environment 22, from which the wearer 14 is currently looking at the motor vehicle 24.

[0033] The dashed circle 26 is a position symbol located at the first virtual observation position. Preferably, the position symbol 26 is not displayed at all within the virtual environment 22 in the view shown here. The position symbol 26 is shown here only for a better understanding of the following explanations.

[0034] In Fig. 3 shows a schematic representation of the virtual observation position A of the wearer 14 together with the position symbol 26 in a side view. The wearer 14 of the virtual reality glasses 12 is positioned according to the Fig. 2, the wearer 14 is thus virtually next to the motor vehicle 24 and is looking laterally at the motor vehicle 24. By pivoting his head, the wearer 14 can change a respective virtual viewing angle from which the motor vehicle 24 is displayed starting from the observation position 26. The virtual viewpoint of the wearer 14 remains unchanged; the wearer 14 can only change a respective viewing angle of the motor vehicle 24 by pivoting his head to the left and right or up and down.

[0035] If the wearer 14 of the virtual reality glasses 12 wishes to change their observation position A in order to view the motor vehicle 24 from a different virtual position, for example, from the front or from the rear, they simply need to touch the button 20 on the remote control 18. This activates a type of movement or jump mode.

[0036] As in Fig. 4, a position symbol 28 and a selection symbol 30 are then displayed within the virtual environment 22. The position symbol 28, displayed here in the form of a circular marking on the virtual floor of the virtual environment 22, is displayed at a position within the virtual environment 22 which corresponds to a second observation position B predefined within the virtual environment 22, i.e., another virtual viewpoint. The arrowhead-shaped selection symbol 30 is moved within the virtual environment 22 as a function of a detected head movement of the wearer 14 of the virtual reality glasses 12. Alternatively or additionally, it can also be provided that the selection symbol 30 is moved within the virtual environment 22 as a function of a detected eye movement of the wearer 14.

[0037] Should the wearer 14 of the virtual reality glasses 12 now wish to view the motor vehicle 24 from the further observation position B, he or she need only partially or completely overlap the selection symbol 30 with the position symbol 28 by means of a corresponding head and / or eye movement.

[0038] As soon as, as in Fig. 5, the selection symbol 30 has been brought into alignment with the position symbol 28, the control unit 16 registers that the position symbol 28 has been selected. In other words, it is registered that the wearer 14 may wish to change their virtual observation position because they have moved the selection symbol 30 over the position symbol 28. Preferably, appropriate visual feedback can be displayed as soon as the position symbol 28 has been selected. For example, the color of the position symbol 28 and / or also of the selection symbol 30 can be changed, so that the wearer 14 easily receives visual feedback about the selection of the position symbol 28.

[0039] Should the wearer 14 of the virtual reality glasses 12 now actually wish to change their virtual observation position from observation position A to observation position B, they simply need to press the button 20 on the remote control 18. For example, the button 20 can have a corresponding key travel with a stop, whereby confirmation of the position change from the virtual observation position 26 to the virtual observation position 28 only occurs when the button has been pressed down by the wearer 14, for example, to its stop.

[0040] In Fig. 6, the motor vehicle 24 is shown starting from the observation position B that has just been virtually assumed, after a virtual change of position has been carried out from the observation position A to the observation position B. The wearer 14 of the virtual reality glasses 12 therefore does not move continuously around the motor vehicle 24 in order to get from the observation position A to the observation position B. Instead, the wearer 14 virtually jumps from the observation position A to the Fig. 6, as soon as he has confirmed the change of position by pressing the button 20 on the remote control 18, after he has brought the selection symbol 30 into alignment with the position symbol 28.

[0041] Below, the position symbol 28 is shown only in dashed lines; preferably, it is no longer displayed after the position change and is shown here only for clarity. If, for example, the wearer 14 still keeps their thumb on button 20 after the position change, i.e., is still touching it, the selection symbol 30 and the position symbol 26 will continue to be displayed. As soon as the wearer 14 releases the button 20 on the remote control 18, i.e., no longer touches it, the position symbol 26 and the selection symbol 30 will disappear.

[0042] In the present case, the wearer 14 can view the motor vehicle 24 from the side diagonally behind from the observation position B, wherein here too the viewing angle of the motor vehicle 24 is changed depending on detected pivoting movements of the head of the wearer 14, wherein the observation position remains fixed at the observation position B.

[0043] Should the wearer 14 again wish to change position, they can activate the movement or jump mode by touching the button 20 of the remote control 18, whereby the selection symbol 30 and the position symbol 26 are again displayed. As soon as the wearer 14 has brought the selection symbol 30 into alignment with the position symbol 26 through a corresponding head or eye movement, the position symbol 26 is selected. As soon as the wearer 14 then presses the button 20, a virtual position change from the observation position B to the observation position A is again carried out, so that the wearer 14 of the virtual reality glasses 12 can again view the motor vehicle 24 purely from the side, as in the Fig. 2, Fig. 4 and Fig. 5, looks.

[0044] Within the virtual environment 22, in addition to the observation positions A and B, further position symbols (not shown here) can be displayed, so that the wearer 14 of the virtual reality glasses 12 can change their virtual observation position within the virtual environment 22 analogously to the procedure already described. Such position symbols can be arranged distributed around the motor vehicle 24 and also displayed distributed within the motor vehicle 24. The wearer 14 of the virtual reality glasses 12 can thus not only virtually jump from one observation position to the next, which lies outside the motor vehicle 24, but can also relocate their virtual observation position into the motor vehicle 24, for example, onto the driver's seat of the motor vehicle 24.

[0045] The wearer 14 of the virtual reality glasses 12 thus does not move continuously around the displayed motor vehicle 24; instead, he jumps from one virtual observation position to the next within the virtual environment 22 in order to change his viewing angle or perspective on the displayed motor vehicle 24.

Claims

[1] Method for operating a virtual reality headset (12), comprising the steps: - Displaying at least one virtual object (24) arranged in a virtual environment (22) using the virtual reality glasses (12) from a first observation position (A) specified within the virtual environment (22); - Displaying at least one position symbol (28) at a position within the virtual environment (22) which corresponds to a second observation position (B) specified within the virtual environment (22); - Selecting the displayed position symbol (28) once a predetermined selection action has been recorded; - Displaying the virtual object (24) from the second observation position (B) as soon as a predetermined confirmation action regarding the selected position symbol (28) has been recorded, characterized by , that - the position symbol (28) is only displayed within the virtual environment (22) when an actuating element (20) is touched, and / or that - a respective virtual viewpoint from which the virtual object (24) is displayed starting from the respective observation positions (A, B), depending on a detected swivel position of a wearer's head (14) of the virtual reality glasses (12). [2] Method according to claim 1, characterized by , that the selection of the displayed position symbol (28) takes place as soon as the predetermined selection action has been detected, that a selection symbol (30) displayed within the virtual environment (22) has been brought at least partially into overlap with the displayed position symbol (28). [3] Method according to claim 2, characterized by, that the selection symbol (30) is moved within the virtual environment (22) depending on a detected head movement of the wearer (14) of the virtual reality glasses (12). [4] Method according to claim 2 or 3, characterized by , that the selection symbol (30) is moved within the virtual environment (22) depending on a detected eye movement of the wearer (14) of the virtual reality glasses (12). [5] Method according to any one of the preceding claims, characterized by , that the actuating element (20) is a button arranged on a remote control (18) coupled with the virtual reality glasses (12). [6] Method according to any one of the preceding claims, characterized by , that the position symbol (28) is hidden from the virtual environment (22) as soon as the touch of the actuating element (20) is lifted. [7] Method according to any one of the preceding claims, characterized by, that the display of the virtual object (24) from the second observation position (B) only occurs when the predetermined confirmation action has been detected, namely that an actuating element (20), in particular a button arranged on a remote control (18) coupled with the virtual reality glasses (12), has been pressed. [8] The method of claim 7 in reference to claim 1 or 6, characterized by , that it is the same actuating element (20) by means of which the virtual position change is confirmed and the display of the position symbol (28) is effected. [9] Method according to any one of the preceding claims, characterized by , that pressing the actuating element (20) is distinguished from touching the actuating element (20) by the fact that when pressing, a larger stroke is executed on the actuating element (20) and / or a greater pressure is exerted on the actuating element (20) than when touching. [10] Method according to any one of the preceding claims, characterized by , that the virtual object (24) is displayed as a motor vehicle. [11] Method according to claim 10, characterized by , that the position symbol (28) is arranged and displayed outside or inside the motor vehicle (24). [12] System (10) with - a virtual reality headset (12) which is designed to ▪ to display at least one virtual object (24) arranged in a virtual environment (22) from a first observation position (A); ▪ to display at least one position symbol (28) within the virtual environment (22) at a position within the virtual environment (22) which corresponds to a second observation position (B) within the virtual environment (22); - a control device (16) which is designed to ▪ to select the displayed position symbol (28) once a predetermined selection action has been recorded; ▪ to control the virtual reality glasses (12) in such a way that the virtual object (24) is displayed from the second observation position (B) as soon as a predetermined confirmation action regarding the selected position symbol (28) has been recorded; characterized by , that the virtual reality glasses (12) are designed to, - to display the position symbol (28) within the virtual environment (22) only when an actuating element (20) is touched, and / or - to change a respective virtual viewpoint from which the virtual object (24) is displayed starting from the respective observation positions (A, B), depending on a detected swivel position of a wearer's head (14) of the virtual reality glasses (12).

Images