Method and device for guiding a user in a virtual environment

Abstract

Method and device for guiding a user in a virtual environment. The invention relates to a method for guiding a user (USR) in a virtual environment, this method comprising the generation (E15) of an indicator (IND) of a direction of movement that said user must make in order to perceive at least partially at least one virtual object (OVi) detected as entirely outside the field of vision (FOV) of the user (USR), said invisible virtual object, said indicator (IND) being capable of being reproduced for the user. Figure for the abstract: Fig 9

Description

Title of the invention: Method and device for guiding a user in a virtual environment

[0001] The present invention relates to the general field of augmented or virtual reality. In this disclosure, the term "virtual reality" covers both of these concepts.

[0002] The invention relates more particularly to the management of a user interface, this interface comprising application windows in a virtual environment.

[0003] For the purposes of this disclosure, these application windows are to be understood in a broad sense. They may, for example, be windows in which computer processes or applications are executed, such as office applications, calculation applications, simulation applications, Internet browsers, or, in another non-limiting example, windows for entering information or displaying results.

[0004] In a known manner, the opening, resizing, positioning and destruction of such windows are generally managed by primitives of the operating system of the computer on which these applications run.

[0005] It is common for a user to have several active applications and therefore many active windows in his environment.

[0006] When the user is operating in a real world, it is common for the user to use several screens to facilitate their interaction with these windows.

[0007] When the user is in a virtual environment, these windows are no longer displayed on a physical screen but presented in a virtual environment perceptible to the user through a suitable device. The user can manipulate these windows by detecting the movement of a pointer (for example, a dedicated object, such as a stylus, or a part of the user's body, such as their fingers, eyes, etc.).

[0008] In a scenario conceivable in the context of the invention, the user can, for example, open a virtual window above his real keyboard, and then manipulate it with one or more fingers to move or resize it.

[0009] These windows can then move out of the user's field of vision. This is especially true since the user may also move their head or move around. In such circumstances, it is common for the user to have difficulty finding an active window and to have to open a new window for the same application.

[0010] The multiplication of windows impacts the performance of the computer system on which they run and this is not desirable.

[0011] This problem is particularly sensitive for application windows but arises in the same way for any other type of virtual object.

[0012] The present disclosure proposes a method and device that aim to reduce these drawbacks. Description of the invention

[0013] To this end, the present disclosure relates to a method of guiding a user in a virtual environment, this method comprising the generation of an indicator of a direction of movement which the user must make in order to see at least partially at least one virtual object detected as entirely outside the user's field of vision, called the invisible virtual object, this indicator being capable of being reproduced for the user.

[0014] Thus, and in general, the present disclosure proposes to present the user with an indicator to guide him towards invisible virtual objects, in other words towards those which he can no longer perceive, even partially.

[0015] In a particular embodiment, the guidance method includes the detection of the invisible virtual object(s).

[0016] In a particular embodiment, the guidance method includes reproducing the indicator(s) for the user.

[0017] In one embodiment, the indicator is a graphic indicator. Alternatively, it could be an audible indicator or a sensory indicator.

[0018] In an embodiment in which the indicator is graphic, it is placed at the edge of the field of vision.

[0019] In one embodiment, this graphic indicator is aligned with a straight line passing through: (i) the center of the projection, on a screen of a virtual reality headset worn by the user, of a base delimiting the field of vision and by (ii) a center of the invisible object, said indicator pointing towards the invisible object.

[0020] In one embodiment, the invisible object is an application window. The guidance method thus prevents the user from creating new application windows when they already exist but are outside their field of vision.

[0021] Correspondingly, the present disclosure relates to a device for guiding a user in a virtual environment, this device comprising a module for generating an indicator of a direction of movement that the user must make in order to see at least partially at least one virtual object detected as entirely outside the field of vision, said invisible virtual object, said indicator being capable of being reproduced for the user.

[0022] This disclosure also relates to a virtual reality system comprising: - a system for tracking the position and orientation of a user's virtual reality headset; - a module for locating virtual objects within the user's virtual environment, and - a guidance device as mentioned above.

[0023] The invention also relates to a computer program on a data carrier and loadable into the memory of a computer, comprising program code instructions intended to control the execution of the steps of the guidance process as described above, when the program is executed on said computer.

[0024] The invention also relates to a data carrier in which the previous program is recorded. Brief description of the figures

[0025] Other features and advantages of the present invention will be better understood from the detailed description and accompanying figures, among which:

[0026] [Fig.1] represents a user in a virtual environment;

[0027] [Fig.2] represents a virtual reality system, according to an example of an embodiment;

[0028] [Fig.3] illustrates a spherical coordinate system;

[0029] [Fig.4] illustrates a field of vision;

[0030] [Fig.5] is a schematic representation of a virtual object guidance device, according to an example embodiment;

[0031] [Fig.6] represents visible and invisible objects;

[0032] [Fig.7] represents indicators in an embodiment;

[0033] [Fig.8] represents indicators in another embodiment;

[0034] [Fig.9] represents steps of a guiding process in an embodiment;

[0035] [Fig. 10] represents the hardware architecture of a guidance device in one embodiment. Detailed description

[0036] [Fig.1] represents a USR user experiencing an immersive environment in a virtual environment generated by a virtual reality system SYS described below with reference to [Fig.2].

[0037] This USR user has a virtual reality headset H equipped with an SCR screen. The position and orientation of the headset H are noted in this figure as POSHd and 0RHd.

[0038] In the embodiment described here, and as shown in [Fig.2], the SYS virtual reality system includes, in particular: - the H virtual reality headset; - an SSH system for tracking the position and orientation of the H virtual reality headset; - an MLO module for locating OVi virtual objects within the virtual environment; and - possibly one or more of the CRV virtual reality controllers; - an MSC module for tracking CRV virtual reality controllers; and -an MRO module for restoring OVi virtual objects.

[0039] The SSH system for tracking the position POSHd and the orientation ORHd of the virtual reality headset H makes it possible to determine, at any given moment, where the user USR is located and in which direction he is looking.

[0040] The SSH system includes, for example: - inertial sensors (accelerometers, gyroscopes, magnetometers, etc.) integrated into the helmet to measure the movements and orientation of the helmet; - cameras placed in the room to track markers placed on the helmet; and / or; - cameras integrated into the helmet to analyze the environment and determine the position and orientation of the helmet relative to landmarks in the room.

[0041] The MLO virtual object localization module is configured to continuously locate and synchronize OVi virtual objects with the position and orientation of the user USR.

[0042] The MLO module is specifically configured to determine an initial reference position of the virtual reality headset H in the real world, which corresponds to a starting position in the virtual environment ENV.

[0043] The OVi virtual objects are positioned in a coordinate system synchronized with the user's position and orientation. Thus, as the user moves, the SYS system updates the headset's position and orientation in real time, adjusting the positions of the OVi virtual objects accordingly.

[0044] In the embodiment described here, and as shown with reference to [Fig. 3], it will be assumed for the sake of simplicity that the position of a point Pi of a virtual object OVi is expressed by spherical coordinates (r, 6, ¢) in a frame (O, X, Y, Z) linked to the user's headset H, in which: - r represents the radial distance between point Pi and the origin O of the coordinate system, this origin corresponding for example to a fictitious point placed approximately between the user's eyes; - represents the azimuthal angle in the horizontal plane, for example measured with respect to the X axis. 0 represents the elevation angle, for example measured with respect to the horizontal plane (O, X, Y).

[0045] The CRV virtual reality controllers (e.g. gamepads, a pointer, a mouse, ...) and / or the hand gestures of the USR user are tracked by the MSC module to enable interaction with OVi virtual objects.

[0046] The MSC module uses, for example, the same tracking techniques as for the H helmet to locate the CRV controllers in space and computer vision algorithms to detect the movements of the hands and body and the body of the USR user.

[0047] In [Fig.3], the user's field of vision FOV has been represented in the form of a square-based pyramid whose apex (viewpoint) is, for example, centered on the origin O of the coordinate system.

[0048] For example, a user's FOV field of view is a square-based pyramid (of infinite height) whose apex angle a is equal to 110 degrees, providing a 55-degree field of view to the left, right, up and down (see [Fig.4]).

[0049] This user viewpoint defines a virtual camera that simulates the operation of a real camera in terms of capturing the image of a scene in which virtual objects would be present.

[0050] The OVi virtual object rendering MRO module is configured to project these virtual objects onto a screen of the H headset taking into account this field of vision.

[0051] To this end, the spherical coordinates of the OVi virtual objects are transformed into 2D coordinates on the headset screen by a projection mechanism that uses a projection matrix taking into account the field of view (FOV) and the focal length. The projected coordinates are converted into screen coordinates according to the headset resolution.

[0052] Thus, the virtual reality SYS system is configured to follow the movement of the USR user, including rotations of his head to adjust the position and orientation of the virtual camera, recalculate the projection of virtual objects according to this updated position and orientation, objects or parts of objects that go out of the user's FOV field of vision are no longer projected onto the screen of the headset H, giving the impression that they go out of his sight.

[0053] The SYS system is remarkable in that it includes a DG guidance device which will now be described with reference to [Fig.5].

[0054] In general, this device assists the USR user in finding virtual objects OVi that are no longer in his field of vision FOV, even partially, in other words, that are invisible.

[0055] In the description that follows, it is assumed that these virtual OVi objects are computer application windows.

[0056] Figure 6 shows schematic representations of:

[0057] - the user's FOV field;

[0058] - an OVi window entirely within this visual field;

[0059] - a window OV2 partially in this visual field;

[0060] - two windows OV3 and OV4 entirely outside this field of vision.

[0061] The OV1 and OV2 windows are visible and the OV3 and OV4 windows are invisible. They may, for example, be invisible either because the user decided to create them or move them outside their field of view (FOV) or because the user moved or turned their head.

[0062] The DG guidance device includes an MDET module to detect if a virtual object is invisible.

[0063] In the embodiment described here, each window is defined by these 4 corners and the spherical coordinates (r, 9, ¢) in the frame (O, X, Y, Z) linked to the user's headset H are continuously calculated by the MLO module for localizing virtual objects.

[0064] In the embodiment described here, a window is invisible when, for each of these corners: - its azimuthal angle 9 is greater than 55 degrees and - its elevation angle 0 is greater than 55 degrees.

[0065] The DG guidance device includes an MGEN module for generating an IND indicator and a MIND module for presenting this IND indicator in the user's virtual environment, this IND indicator indicating a direction of movement that the user must make to see at least partially an invisible virtual object.

[0066] As shown in [Fig.7], this IND indicator is for example represented by an arrow shown at the limit of the FOV field of vision, aligned on a straight line (not shown) passing through the center C of the projection on a screen of the helmet H of the square base delimiting the FOV field of vision and through the center of the invisible object, this indicator pointing towards the invisible object.

[0067] In this embodiment, the IND indicators move along the periphery of the field of vision according to the orientation of the virtual reality headset and the location of the invisible objects.

[0068] In a particular embodiment, the indicator IND has a property (size, opacity, ...) which depends on the distance between the parallel edges of the invisible object and the field of vision closest to each other.

[0069] The user therefore understands in which direction and with what amplitude he must move to see an invisible object.

[0070] In an embodiment illustrated in [Fig.8], if several invisible windows are located in the same direction or in a nearby direction, the indicator includes a number that indicates the number of these windows.

[0071] The different elements of the virtual reality system are linked together by means of communication which can be of any nature.

[0072] The steps of a guiding method will now be described in relation to [Fig.9],

[0073] In the embodiment described here, this process executes a loop which includes: - an E10 detection step;

[0074] -an El5 generation step; and - a presentation step E20, with steps E15 and E20 being implemented or not depending on the result of the detection step E10.

[0075] During step E10, it is detected whether, at the current time, at least one virtual object is invisible. In the embodiment described here, this step includes, in particular, determining the position of the virtual objects using a marker attached to the user's virtual reality headset and a step consisting of verifying whether at least one of these objects is completely outside the user's field of view (FOV).

[0076] When at least one virtual object is invisible, the method includes a step E15 for generating the indicator, and a step E20 for presenting this indicator in the user's virtual environment. This indicator IND indicates a direction of movement that the user must make to at least partially see said at least one invisible virtual object.

[0077] This step may consist of presenting an IND indicator as shown in Figures 7 and 8.

[0078] Figure 10 schematically represents the hardware architecture of a DG guidance device according to the invention.

[0079] In the embodiment described here, this guidance device has the hardware architecture of a computer. It includes in particular a processor 10, a read-only memory of type ROM 11, a read-only memory of type RAM 12. These means define the detection modules DET and indicator presentation modules IND described with reference to [Fig. 5].

[0080] The read-only memory 11 constitutes a recording medium. It includes a computer program P. This program P includes instructions which, when executed by the processor 10, implement the steps of the guidance process described previously with reference to [Fig.9].

Claims

Demands

1. Method of guiding a user (USR) in a virtual environment, this method comprising the generation (E15) of an indicator (IND) of a direction of movement that must be made by said user in order to see at least partially at least one virtual object (OVi) detected as entirely outside a field of view (FOV) of the user (USR), said invisible virtual object, said indicator (IND) being capable of being reproduced for the user.

2. A guidance method according to claim 1 comprising the detection (E10) of said at least one invisible virtual object (OV;).

3. A guidance method according to any one of claims 1 or 2 comprising the reproduction (E20) of said direction indicator (IND) for the user.

4. A guidance method according to any one of claims 1 to 3 wherein said indicator (IND) is a graphic indicator.

5. A guidance method according to claim 4 in which said graphic indicator is placed at the edge of the field of view (FOV).

6. A guidance method according to claim 4 or 5 wherein said graphic indicator is aligned with a straight line passing through the center of the projection, on a screen of a virtual reality headset worn by the user, of a base delimiting said field of view (FOV) and through a center of the invisible object, said indicator pointing towards the invisible object.

7. A guidance method according to any one of claims 1 to 6 wherein said invisible object is an application window.

8. A device for guiding a user in a virtual environment, this device comprising a module (MGEN) for generating an indicator (MIND) of a direction of movement that the user must make in order to see at least partially at least one virtual object (OVi) detected as entirely outside the field of view (FOV), referred to as the invisible virtual object, said indicator (IND) being capable of being reproduced for the user.

9. Virtual reality system (SYS) comprising - a system (SSH) for tracking the position and orientation of a user's virtual reality headset (H); - a module (MLO) for localizing virtual objects (OVi) in a user's virtual environment, and - a guidance device according to claim 8.

10. Computer program (P) on a data carrier (11) and loadable into the memory of a computer, comprising program code instructions for controlling the execution of the steps of the guidance process according to any one of claims 1 to 7, when the program is executed on said computer.

11. Data support (11) in which the program according to claim 10 is stored.

Images