Method for constructing fluid transitions of a hand in a virtual or augmented reality environment

Abstract

The invention relates to a method for generating a graphic representation of a virtual hand, performed as a function of a first set of parameters (10) representing a first state and a first position of a virtual hand in a virtual environment, and a second set of parameters (12) representing a geometry and a position of an object intended to be gripped by said hand. The method comprises determining (E6) at least a fourth set of parameters (60) representing a third intermediate state between the first state and a second state, and a third position between the first position and the second position. The hand grips the object in the first state or second state. The third state makes it possible to generate (E8) a graphic representation of the virtual hand from the first state in the first position, through the third state in the third position, to the second state in the second position.

Description

[0001] DESCRIPTION

[0002] TITLE: Method for constructing smooth transitions of a hand in a virtual or augmented reality environment

[0003] 1. Field of the invention

[0004] The present invention relates to the field of virtual reality. More specifically, the invention relates to the representation of a virtual hand within a virtual environment.

[0005] The user has visual access to this environment via video feedback (for example using a virtual reality headset).

[0006] For the purposes of the present application, the expression "state of a hand" refers to the different configurations that this hand can take, that is, to the relative position of the fingers, the phalanges and / or the palm of the hand. For example, such a hand can be in a closed state in which the hand forms a fist, an open state in which the fingers are stretched in line with the palm, a relaxed state in which the fingers are relaxed, or any intermediate state.

[0007] 2. Prior art and its drawbacks

[0008] Virtual reality (VR) is a field in which user immersion is essential to provide a satisfactory experience and ergonomics. In particular, in many situations where a user is immersed in a virtual reality environment, the user may be required to control a virtual hand to perform certain steps such as grasping and moving an object. This control of the virtual hand is, for example, achieved using a controller held by the user.

[0009] Joystick control is technically simple to implement, and allows the use of buttons and other haptic devices to allow the system to acquire instructions from the user. Joystick control also provides haptic feedback to the user when the virtual hand interacts with the environment in which it is immersed.

[0010] However, joystick control of the virtual hand is not without drawbacks. In particular, when a user holding a joystick wishes to grasp an object in the virtual environment using the virtual hand that he is controlling, the user first brings the virtual hand close to the object. Once the hand is close enough, the user sends an instruction, typically by pressing a button on the joystick, to grasp the object. The virtual hand must then make a transition from a first state, called rest, in which the virtual hand moves in its environment, to a second state, called grasping, in which the hand grasps the object.

[0011] The execution of this transition from a first neutral state to a second grasping state poses a problem, because either the transition is gradual and starts at the moment the grasping instruction is issued, which induces latency in its execution, or the transition is instantaneous and this greatly deteriorates the immersion for the user.

[0012] The present invention proposes to improve the situation.

[0013] 3. Statement of the invention

[0014] To this end, the invention proposes a method for generating a graphical representation of a virtual hand as a function of a first set of parameters representative of a first state of a virtual hand and a first position of said virtual hand in a virtual environment, and at least a second set of parameters representative of a geometry of at least one object intended to be gripped by said virtual hand and a position of said object in said virtual environment, the method comprising the following steps: a determination of at least a fourth set of parameters representative of at least a third state of the virtual hand intermediate between the first state and a second state of the virtual hand, as a function of the first set of parameters, of a third set of parameters representative of the second state of the virtual hand in a second position in the virtual environment,and at least one piece of information representative of a third position of the virtual hand located between the first position and the second position, the third set of parameters being a function of the first set of parameters and the second set of parameters, the virtual hand gripping the object in one of the states among the first state and the second state in a position among respectively the first position and the second position corresponding to the position of the object, the third state allowing generation of a graphic representation of the virtual hand from the first state at the first position to the second state at the second position via the third state at the third position.,

[0015] Thus, such a method for generating a graphical representation of a virtual hand offers a smooth transition of the virtual hand from an initial state to a state of grasping an object, and this in an anticipated manner, that is to say even before the user produces a grasping instruction by means of a control device.

[0016] More specifically, as the virtual hand approaches the object to be grasped, the hand gradually evolves from the first state, or initial state, to the second state, or grasping state, passing through the at least third state, or intermediate state, representative of a transition between the initial state and the grasping state. Thus, the virtual hand, whose movement and actions, such as grasping an object, can be controlled by a user by means of a control device, such as, for example, a controller or a joystick, in particular with six degrees of freedom (also noted "6DoF" for "six degrees of freedom" in English), in particular the control device is located in space, presents the user with more realistic behavior.In particular, the user does not need to give any specific instruction, via the control device, to gradually close the virtual hand when it is close to the object, thus simulating the behavior of a “real” hand.

[0017] The first state may for example be a state in which the virtual hand is slightly open (called the “neutral” or “relaxed” state), completely open (or “flat”), or in a current state of the virtual hand at the time when the method is implemented. The second state represents the state in which the virtual hand is found when it is as close as possible to the object, in a configuration in which the virtual hand is able to grip the object.

[0018] The third state of the virtual hand depends on the relative position of the virtual hand with respect to the object as well as the first state and the second state of the virtual hand. This intermediate state is all the closer to the grasping state the closer the virtual hand is to the object. As the virtual hand gets closer to the object, the intermediate state then becomes more and more similar to the grasping state. In fact, at the instant the control device issues an instruction to grasp the object, the virtual hand grasps the latter without abrupt transition or delay in the execution of the instruction because it is already in the grasping state or in a state similar to the grasping state. The quality of user experience is greatly enhanced, without sacrificing the responsiveness of the control of the virtual hand.According to a particular characteristic, the method comprises a determination, as a function of the first set of parameters and the second set of parameters, of a third set of parameters representative of the second state of the virtual hand in the second position.

[0019] According to a particular characteristic, the parameters of the third set of parameters are representative of a second state of the virtual hand when it grips the object.

[0020] According to a particular characteristic, the method comprises generating the graphic representation of the virtual hand from the first state at the first position to the second state at the position of the object, passing through the third state at the second position.

[0021] In one example, a state of the virtual hand is defined by a plurality of parameters including a list of angle values, an angle value describing an angle formed by two successive phalanges of a finger of the virtual hand or an angle formed by a phalanx and the palm of said virtual hand.

[0022] Treating the phalanges individually makes the graphic representation more realistic and smoother, because these values ​​can be finely determined according to the type of transition desired.

[0023] According to one example, said at least one fourth set of parameters further comprises a transition parameter whose value is between a minimum value corresponding to the first state and the first position of the virtual hand and a maximum value corresponding to the position of the object and the second state of the virtual hand.

[0024] In this example, the set of fourth parameter sets forms a continuum (or a sufficiently fine discretization) between the initial state and the grasping state. Each state and each position represented by one of the fourth parameter sets being associated with a value of the transition parameter, the graphical representation of the transition of the virtual hand is thus more realistic and promotes immersion. The continuum thus constructed makes it possible to adapt the graphical representation of the virtual hand according to the frequency at which frames are displayed by the video feedback giving access to the environment of the virtual hand, for example 90 Hz for virtual reality. In addition, this parameterization of the set of fourth parameter sets advantageously makes it possible to adapt the transition of the hand to the speed at which it approaches the object.

[0025] In one example, at least one angle value is a function of the transition parameter value.

[0026] Here, defining at least one angle value as a function of a value of the transition parameter allows the graphical representation of the virtual hand to be finely controlled, for example phalanx by phalanx, depending on the value of the transition parameter.

[0027] In one example, the function relating the value of the transition parameter to a given angle value is a linear interpolation passing through an angle value corresponding to the minimum value of the transition parameter and through an angle value corresponding to the maximum value of the transition parameter.

[0028] Here, the transition data are constructed by linear interpolation. The inventors observed that this choice of interpolation was a good compromise between realism and calculation speed.

[0029] In one example, a value of the transition parameter corresponds to: one minus a ratio between the relative position of the virtual hand with respect to the object and a transition distance when said ratio is positive, and zero otherwise.

[0030] In this example, the value of the transition parameter varies between the values ​​0 and 1. When the virtual hand is beyond a certain distance from the object, called the transition distance, the value of the transition parameter is zero, and the hand remains in its current state (e.g., a relaxed state). When the virtual hand gets closer to the object, the value of the transition parameter increases linearly with the proximity of the virtual hand to the object (defining proximity as the difference between the transition distance and the distance between the virtual hand and the object).

[0031] In one example, the first set of parameters is further representative of an orientation of the virtual hand in the virtual space, the method further comprising: determining, based on the position of the virtual hand, the orientation of the virtual hand and the position of the virtual object, an intention parameter whose value represents a probability of the user's intention to grasp said object with said virtual hand; determining whether said intention parameter satisfies a first condition, said satisfaction resulting in determining the third set of parameters and determining said at least one fourth set of parameters.

[0032] Here, the determination of the third set of parameters and said at least fourth set of parameters is conditioned on the satisfaction of a first condition, called the intention condition, by the intention parameter. This saves computation time, by only constructing the third set of parameters and said at least fourth set of parameters for an object that the user likely intends to grasp with the virtual hand.

[0033] In one example, the virtual environment comprises a plurality of objects, and the first condition is satisfied for the object associated with said intent parameter having the largest value.

[0034] This choice of first condition allows, in a multi-object environment, to calculate the transition of the object that the user most likely wishes to grasp.

[0035] In one example, each object is associated with at least one class, and the value of the second parameter associated with that object is a function of the at least one class associated with the object.

[0036] In this example, the objects in the environment are each labeled with a certain class (which may be common to several objects). These classes may, for example, correspond to different steps in a process, or to a type of object such as a tool. Thus, when the user has to manipulate several objects (for example, tools), these classes contribute to the calculation of the second parameter and therefore to the hand transition. This allows, on the one hand, to better discriminate the choice of the object serving as the basis for the transition by authorizing only objects relating to a given class, and on the other hand, plays a role of deception for the user, by encouraging the user to move the virtual hand towards the relevant object(s).

[0037] In one example, the method includes obtaining data representative of a user's gaze direction, wherein the intent parameter associated with an object takes on a value that is greater the more the gaze data indicates that the user is looking in the direction of said object.

[0038] In this example, determining a value for the intention parameter is, among other things, a function of the user's gaze direction. Obtaining the user's gaze direction can, for example, be done using a virtual reality headset. This solution offers the user faster, more ergonomic, and more intuitive control of the virtual hand, allowing them to control the choice of the object that serves as the basis for constructing the transition from the hand to grasping this object.

[0039] In one example, determining the intent parameter associated with the object includes determining the presence of the object within a cone extending from the palm of the virtual hand and including a vector orthogonal to the palm. In this example, determining the presence of the object using a cone allows the orientation of the virtual hand to be used to determine the value of the intent parameter. This allows a user to select the object they intend to grasp by orienting the virtual hand substantially in the direction of said object. This method eliminates the need to calculate values ​​for intent parameters associated with objects located outside the cone (or to set these values ​​to zero as long as the object is located outside the cone, which is equivalent).

[0040] In one example, the cone comprises rays forming a mesh of the cone, and determining a value of the intent parameter associated with a given object comprises determining at least one ray reaching the object, the value of said intent parameter being an increasing function of the collinearity between this ray and the vector orthogonal to the palm.

[0041] In this example, the value of the intent parameter associated with an object is greater the more the virtual hand is oriented towards the object. This allows a user to more precisely select the object on the basis of which the transition can be generated.

[0042] In one example, the value of the intent parameter associated with an object is a decreasing function of the distance between the palm and the object.

[0043] In this example, the further away an object is, the less likely the user is to intend to grasp it. This allows for various optimizations, including declaring the value of the intent parameter associated with an object as zero when the object is beyond a certain distance from the object. This makes it possible to calculate the value of the intent parameter for a large number of objects by discriminating between distant objects very quickly, for example by using techniques from calculating collisions in a physics engine.

[0044] The invention further relates to a computer device for implementing a graphical representation method as described above.

[0045] The invention further relates to a computer program product comprising program code instructions for implementing a method according to the invention as described previously, when executed by a processor.

[0046] Such a computer program may be recorded on a computer-readable recording medium. This recording medium may be any entity or device capable of storing the program. For example, the medium may include a storage medium, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording medium, for example a USB key or a hard disk.

[0047] On the other hand, such a recording medium may be a transmissible medium such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio or by other means, so that the computer program contained therein is remotely executable. The program according to the invention may in particular be downloaded over a network, for example the Internet.

[0048] Alternatively, the recording medium may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method which is the subject of the aforementioned invention.

[0049] The invention also relates to a virtual reality device comprising: a first generator of graphic representation of a virtual hand as a function of a set of parameters representative of a state of a virtual hand and of a position of said virtual hand in a virtual environment, a second generator of graphic representation of an object as a function of at least one set of parameters representative of a geometry of at least one object intended to be gripped by a virtual hand and of a position of said object in said virtual environment, a graphic reproducer configured to reproduce the virtual environment, the generated graphic representation of the virtual hand,and the generated graphical representation of the object; wherein the first generator of graphical representation of a virtual hand comprises a calculator determining at least a fourth set of parameters representative of at least a third state of the virtual hand intermediate between a first state and a second state of the virtual hand, as a function of a first set of parameters, a third set of parameters representative of a second state of the virtual hand in a second position in the virtual environment, and at least one item of information representative of a third position of the virtual hand situated between the first position and the second position, the third set of parameters being a function of the first set of parameters and the second set of parameters,the virtual hand gripping the object in one of the states among the first state and the second state in a position among respectively the first position and the second position corresponding to the position of the object, the third state allowing generation of a graphic representation of the virtual hand from the first state at the first position to the second state at the second position via the third state at the third position.,

[0050] 4. List of figures

[0051] Other aims, characteristics and advantages of the invention will appear more clearly on reading the following description, given as a simple illustrative, and non-limiting, example, in relation to the figures, among which:

[0052] [fig- 1] represents a simplified diagram of the steps of a control method according to the invention,

[0053] [fig. 2] represents a virtual hand controlled by the method of figure 1,

[0054] [fig. 3] represents a simplified diagram of a variant of the process of figure 1,

[0055] [fig- 4] represents step E4 of figure 3,

[0056] [fig. 5] represents an example of a device implementing the method of figure 1, and [fig. 6] represents a simplified diagram of a virtual reality device according to the invention.

[0057] 5. Detailed description of embodiments of the invention

[0058] 5.1. General principle of the invention

[0059] Reference is made to Figure 1, which represents a method of generating a graphical representation of a virtual hand in a virtual environment.

[0060] This method is carried out according to a first set of parameters 10 and at least one second set of parameters 12. The first set of parameters 10, also called a set of position parameters, is representative of a first state of a virtual hand and a first position of said virtual hand in the virtual space. The second set of parameters 12, also called an object parameter set, is representative of a geometry (in particular a shape) of at least one object intended to be gripped by said virtual hand and a position of said object in said virtual environment.

[0061] The method comprises a step E4 of determining a third set of parameters 40 representative of a second state of the virtual hand when it grips the virtual object, a step E6 of determining at least a fourth set of parameters 60 and a step E8 of generating a graphical representation of the virtual hand from the first state at the first position to the second state at the position of the object via the third state at the second position.

[0062] The determination step E4 is carried out as a function of the first set of parameters 10 and the second set of parameters 12. The determination step E6 is carried out as a function of the first set of parameters 10, the third set of parameters 12, and at least one item of information representative of a second position of the virtual hand in the virtual space located between the first position of the virtual hand and the position of said object. The fourth set of parameters 60 is representative of at least a third state of the virtual hand intermediate between the first state and the second state.

[0063] The first state of the virtual hand, called the initial state, may be the current state of the virtual hand, i.e. the state in which the virtual hand is at the time of the construction step E6. In another example, the initial state may be a so-called “neutral” state of the virtual hand, corresponding to a state of the virtual hand when it is at rest. The neutral state may be a state in which the virtual hand is completely open and flat, or alternatively a so-called relaxed state in which the fingers of the virtual hand are slightly curved. The first set of parameters 10 is referred to as the initial set, for similar reasons.

[0064] The second state, called the gripping state, corresponds to a state in which the virtual hand grips the object to be grasped. The third set of parameters 40, called the gripping parameter set, is representative of this state. This third set of parameters 40 is determined from the first set of parameters and the second set of parameters. Various methods for determining this set of gripping parameters will be described below.

[0065] The other sets of parameters 60 (i.e. the fourth set or sets of parameters) representative of the so-called intermediate states between the initial state and the gripping state are determined using the first set of parameters 10 and the third set of parameters 40. These states and these sets are described as intermediate because they are “between” the initial state and the gripping state (or “between” the initial set and the gripping set).

[0066] Step E8 of generating the graphical representation of the virtual hand is based on the different sets of parameters representative of the previously determined intermediate states, as well as on the initial set 10 and on the gripping set 40. The graphical representation thus obtained represents the transition of the hand to the different intermediate state(s). This makes it possible to obtain a smooth transition of the virtual hand between its initial state and the gripping state.

[0067] Reference is now made to Figure 2, which illustrates a virtual hand 100 and an object 120 both located in a virtual environment, the virtual hand 100 seeking to grasp the object 120. The virtual hand 100 is controlled via a control device (controller, joystick or any other device suitable for moving a virtual hand in a virtual environment). For the remainder of this description, the control device here is a controller.

[0068] The virtual hand 100 is controlled in position and orientation by movement instructions from the joystick manipulated by a user. The virtual hand 100 is located at a relative position 140 of the object 120, this relative position 140 being able to be derived from the first set of parameters 10 and the second set of parameters 12.

[0069] Here, the virtual hand 100 is represented in three states, labeled A, B and C in Figure 2. In state A, the virtual hand 100 is in an initial state (here, a relaxed state), at a first position. The virtual hand 100 in state A is therefore at a first distance from the object 100. In state C, the virtual hand 100 is in the grasping state. In this grasping state, the distance separating the virtual hand 100 from the object 120 is less than or equal to a grasping threshold. When the distance separating the virtual hand 100 from the object 120 is less than the grasping threshold, it is considered that the virtual hand 100 is in contact with the surface of the object 120 that it can grasp. In state B, the virtual hand 100 is in an intermediate state, and at a relative position 140 intermediate between the relative position of state A and the gripping threshold.

[0070] Whether the virtual hand 100 is in one or other of these states depends on the relative position 140 of the virtual hand 100 with respect to the object 120. An intermediate state of the artificial hand 100 is all the closer to the grasping state the closer the artificial hand 100 is to the object 120 (or the closer the distance between the virtual hand and the object is to the grasping threshold).

[0071] The graphical representation of the virtual hand 100 can be performed without any instruction from the user other than the movement of the virtual hand 100 controlled via the controller. In particular, the transition of the virtual hand 100 from a current state to the gripping state does not need to be triggered by an interaction of the user with the controller leading to the emission of an order to grip the object. In fact, the transition of the virtual hand 100 to the gripping state is a function of its movement relative to the object 120.

[0072] The method thus described makes it possible to provide a smooth transition from the virtual hand 100 to the grasping state C of the object 120 in an anticipated manner, even before a grasping instruction is generated by the controller. At the moment when the user wishes to grasp the object after having approached the virtual hand 100 via the controller, the virtual hand 100 is already in a state to be able to grasp the object, and an instruction to grasp the object 120 can be executed without latency or abrupt transition.

[0073] The virtual hand 100 thus adopts, thanks to this method, a behavior that is both reactive, because it is able to grasp the object 120 without any latency other than the movement of the virtual hand 100 towards the object 120, and fluid, because it is able to grasp the object without any abrupt transition.

[0074] Here we have described the general principle of a smooth transition from an initial state (for example the current state of the hand at its current position or a relaxed state at a position of the virtual hand without an object near the virtual hand) to a grasping state. This principle applies equally to implementing a smooth transition from the grasping state (at the grasping position) to another state of the hand more distant from the object (for example a relaxed state without an object near the virtual hand), always passing through an intermediate state at an intermediate position.

[0075] 5.2. Construction of the fluid transition

[0076] In an exemplary embodiment, the step E6 of determining the transition set(s) comprises the construction of one or more fourth sets of parameters 60 representative of intermediate states. A fourth set of parameters 60 representative of a given intermediate state comprises a transition parameter t. Such a transition parameter t can take a plurality of values ​​in a given interval.

[0077] Generally speaking, the transition parameter t can vary between a minimum value and a maximum value. To simplify the rest of the description, we consider for the following that the transition parameter t varies between 0 and 1 inclusive, of course other values ​​can be considered for the transition parameter t.

[0078] Among the set of possible intermediate states for the virtual hand 100, the intermediate state associated with the value 0 of the transition parameter t corresponds for example to the initial state. Similarly, the intermediate state associated with the value 1 of the transition parameter t corresponds for example to the grasping state.

[0079] The values ​​of the parameters constituting the sets of parameters representative of the different intermediate states of the virtual hand 100 evolve continuously, in the mathematical sense of the term, with respect to the value of the transmission parameter t. In a particular example, the values ​​of the parameters constituting the sets of parameters representative of the different intermediate states evolve monotonically from the initial state (when the transmission parameter t is equal to 0) to the grasping state (when the transmission parameter t is equal to 1).

[0080] In fact, these characteristics contribute to the fluidity of the transition. Indeed, the values ​​of the parameters constituting the sets of parameters representative of the different states vary continuously with the value of the transmission parameter t, provided that the latter itself varies continuously.

[0081] This choice is also flexible, as it allows for fine variation in the transition of the virtual hand between its different states. This makes it possible to adapt the speed of this transition to different situations and in particular to the display frequency of the virtual environment on a display device, such as the screen of a virtual reality headset. For example, the transition can be made at a display frequency of 90 Hz.

[0082] In one embodiment, a set of parameters representative of a state of the virtual hand comprises a list of angle values, each angle value describing the angle formed by two successive phalanges of a finger of the virtual hand or the angle formed by a phalanx and the palm of said virtual hand.

[0083] In this case, the angle value can be a function of the value of the transition parameter t. In a specific example, this function is a monotonic function going from a first value of the transition parameter t (that of the initial state) to a second value of the transition parameter t (that of the grasping state).

[0084] For example, it is possible to express that, for a hand with five fingers, each having at most three phalanges, for a phalanx p E [1, 3] of a finger n E [1, 5] (n and p being integers), the angle a p nof the phalanx p with the preceding phalanx p — 1, or the palm for the first phalanx, is expressed as a function of t, i.e. a p n (t). The state e(t) of the virtual hand for a value of the transition parameter t can then be expressed as a 3x5 matrix:

[0085] Defining the different states of the virtual hand 100 by parameters of angles between phalanges offers a lightweight format allowing to transcribe the richness of the states that a virtual hand can adopt as realistically as possible. This offers great finesse in the representation of the virtual hand 100.

[0086] In one example, the function associating a given angle value with a value of the transition parameter is a linear interpolation passing through the angle value of the initial state when the value of the transmission parameter is equal to 0, and through the angle value of the gripping state when the value of the transmission parameter is equal to 1. The interpolation can be linear, which allows for very fast calculation. The inventors have observed that linear interpolation is a good compromise between calculation speed and realism. Alternatively, the interpolation can be of a different type, for example a quadratic interpolation.

[0087] In one example, the virtual hand 100 remains in the neutral state (relaxed or completely flat) as long as it is located at a distance d greater than a transition distance d T . This transition distance d T can for example be worth 30 cm.

[0088] In this case, the value of the transition parameter t remains equal to 0 as long as the distance d between the virtual hand 100 and the object 120 is greater than the transition distance d T . When d < d T , t varies between 0 and 1 monotonically, and when d < transition threshold (and therefore the virtual hand 100 is closest to the object 120), t = 1. The transmission parameter t can vary continuously with respect to the ratio d / d T , which ensures a smooth transition. In one example, t = 1 — d / d T when d < d T . The d / d ratio T quantitatively represents the proximity of the virtual hand 100 to the object 120.

[0089] According to an implementation example, the determination of the parameters representative of a given state for a finger of the virtual hand 100 is carried out on the basis of the following algorithm: in which:

[0090] InterpolationClosure(value) is a linear interpolation function between the initial state (InterpolationClosure(O)) and the grasping state (InterpolationClosure(l)) as described above,

[0091] StabilizeRecursively determines the stopping of all anterior phalanges in a given finger,

[0092] & is the logical AND operator, and ! is the logical negation operator.

[0093] 5.3. Intention to seize

[0094] Reference is now made to Figures 3 and 4, which describe an alternative example of implementation of the method described in Figure 1. In this example, the method represented in Figure 3 comprises a step E2 of determining an intention of the user to grasp the object 120 with the virtual hand 100 that he controls.

[0095] When the virtual environment in which the virtual hand 100 operates comprises several objects, the method in this example makes it possible to determine for which object among the plurality of objects included in the virtual environment to construct the third set of parameters 40 and the fourth set of parameters 60. To do this, the method implements, prior to the step E4 of determining the third set of parameters 40, a step E2 of determining, for a given object, an intention parameter 20 representative of the user's intention to grasp said object. This step, like the transition of the virtual hand 100 described above, is carried out without needing specific instructions from the user. This improves the user's immersion in the virtual environment.

[0096] The determination step E2 comprises a step E20 of determining an intention parameter 20 associated with the object 120 and a step E22 of determining the satisfaction of an intention condition 22 on the basis of this intention parameter 20.

[0097] Step E20 is carried out as a function of the first set of parameters 10 and the second set of parameters 12. In fact, a value of the intention parameter 20 determined in step E20 is associated with the object 120 relating to the second set of parameters 12. The value of the intention parameter 20 is representative of a probability of intention of the user to grasp said object 120 with said virtual hand 100. Different methods for determining this value of the intention parameter 20 will be described below.

[0098] Step E22 determines the satisfaction, by the values ​​of the intention parameters 20 associated with the object 120 previously determined, of an intention condition 22. When the intention condition 22 is satisfied, this triggers step E6 of determining the third set of parameters 20 described above as a function of the first set of parameters 10 and the second set of parameters 12 whose intention data 20 satisfy the intention condition 22.

[0099] In addition to being able to discriminate between objects and determine one that can serve as a basis for the transition, this intention determination step E2 saves calculation time, for example by automatically eliminating a virtual object 120 whose intention parameter value would be too low (for example because it is too far from the virtual hand 100).

[0100] The value of the intention parameter represents a probability, as explained above. Thus, according to an exemplary embodiment, the value of the parameter is for example between 0 and 1 inclusive. When the value of the intention parameter of an object is 0, the user probably does not intend to grasp this object, and when the value of the intention parameter is 1, the user very probably intends to grasp this object.

[0101] According to an exemplary embodiment, when several objects are present, the intention condition 22 for an intention parameter associated with a given object 120 is the fact of having the highest intention value among the set of intention parameters 20.

[0102] According to another implementation example, at least one class is associated with at least one object (and for example with all objects). The classes that can be associated with an object form a set of classes. The value of the intention parameter is then a function of this at least one class. More precisely, at the time of the intention determination step E2, it is determined whether the class associated with the object belongs to a subset of classes of the set of classes. If this is the case, the determination of the intention parameter 20 continues, otherwise the intention condition 22 is considered not to be satisfied (or alternatively the intention parameter is declared to have a null value). In the case where several classes are associated with an object, the intention condition may be that at least one of these classes belongs to the subset of classes.

[0103] It should be noted that although at least one class is associated with the object, multiple objects can share the same class, even if the objects are of different natures.

[0104] This implementation example is particularly advantageous in an example situation where a user must carry out a certain process divided into tasks, each task involving one or more objects (for example, one or more tools). Here, each class is relative to a given task of the process. Each tool is used in one or more tasks of the process. Each tool is then associated with the class or classes relating to the tasks in which the tool can be used. When the user is at a given task, the method comprises presupposing that the user does not intend to grasp an object other than the one or those used in said given task. Objects that are not attached to this task therefore have their intention parameter value declared null until the current task (in which they are not used) is finished.This serves both to save computation time, but can also act as a deception for a user, by encouraging them not to go towards an object whose transition is not calculated (the hand therefore not making a transition towards grasping this object). User guidance is improved, without this deteriorating immersion compared, for example, to a display of highlighted objects. This example of implementation also makes it possible to prohibit the grasping of certain objects (for example if they are not intended to be grasped), and therefore not to build a smooth transition towards these objects.

[0105] According to an exemplary embodiment, the intention to grasp a given object may also depend on a direction of gaze of the user. More specifically, the method comprises acquiring data relating to the direction of gaze of the user. The value of the intention parameter 20 is then a function of this data relating to the direction of gaze. For example, if the data relating to the direction of gaze indicate that the user is looking in the direction of the object 120, then the value of the intention parameter 20 is close to 1. The step of determining the value of the intention parameter 20 may here comprise determining a distance between the direction of gaze of the user and the object, then determining a gaze factor which is a decreasing function of said distance between the gaze and the object.The gaze factor can then be incorporated into the calculation of the intent parameter value, for example by multiplying the intent parameter value by the gaze factor.

[0106] To avoid forcing the user to continuously look at the object 120, the gaze factor may be determined at regular intervals, for example every ten frames, or every second. These examples are given for illustrative purposes.

[0107] According to an exemplary embodiment, determining the value of the intention parameter associated with the object comprises determining the presence of the object in a cone whose vertex is located at the palm of the virtual hand and including a vector orthogonal to the palm. The cone may, for example, start from the center of the palm, although it may start from another location on the palm.

[0108] This determination of the intention by inclusion of the object in a cone makes it possible to play on the orientation of the virtual hand 100 for the determination of the value of the intention parameter. In particular, by judiciously orienting the virtual hand 100, the user is deemed not to have the intention of grasping objects located outside the cone. The vector orthogonal to the palm (also called normal to the palm) is preferably inside the cone, for example substantially at the center of the cone (but not necessarily).

[0109] The methods of determining the value of the intention parameter by class associated with an object, by means of the direction of gaze and by presence of the object in a cone are complementary and can be combined with each other.

[0110] In one example, rays forming a mesh of the cone are defined. Determining the value of the intention parameter associated with a given object then includes determining at least one ray reaching the surface of the object. The value of the intention parameter is determined as an increasing function of the collinearity between this ray and the vector orthogonal to the palm. Collinearity can for example be calculated as the scalar product of the unit vector defining the direction of the ray and the normal to the palm (also unitary, i.e. with norm equal to 1).

[0111] According to one example, the value of the intention parameter 20 associated with an object 120 is a decreasing function of the distance between the palm of the virtual hand 100 and the object 120.

[0112] This decreasing function can have a threshold beyond which the value of the intention parameter is zero. This makes it possible to very quickly exclude distant objects, for which it is reasonably certain that the user does not intend to grasp them in the short term.

[0113] According to an implementation example, the determination of membership in the cone previously described for the virtual hand 100 is carried out on the basis of the following algorithm: where minAngleX, maxAngleX, minAngleZ and maxAngleZ are the lower and upper bounds of the cone with respect to the X and Z directions (respectively); nbRaysX and nbRaysZ are the number of rays forming the mesh in the X and Z directions (respectively); palmForward is the normal to the palm and reach is the maximum distance a ray can reach. The bounds along X and Z and the normal to the palm together define the cone and the variables nbRaysX and nbRaysZ define the density of its mesh.

[0114] Each ray has an intersection distance with the object hitDist and an intersection point with the object hitPoint. hitDist and hitPoint are derived from the variable reach, among other things. This algorithm identifies all rays that encounter the object. From this set of rays, the algorithm extracts the ray with the smallest impact distance hitDist.

[0115] In order to favor rays closest to the palm normal (and limit object discrimination problems), the obtained hitDist distances can be weighted by the collinearity between the direction vector of each ray and the palm normal so as to obtain a weighted distance DistPond, for example as below (where DistPond is a linear combination of the hitDist distance and a distance weighting by the collinearity between the ray direction rayDirection and the palm normal palmForward). Here, the factors 1 / 5 and 4 / 5 are chosen arbitrarily, one could consider other pairs of weighting factors.

[0116] DistPond = h.itDist / 5 + hitDist x 4 / 5 x rayDirection ■ palmForward

[0117] The ray that has the smallest weighted distance is then retained as the “intentional” point of contact with the object. This intentional point of contact can serve as a basis for determining the grasping state in step E4 of determining the third set of parameters 40. When the determination of the values ​​of the intention parameters is also based on the direction of gaze and / or a class associated with the object, data relating to the direction of gaze or the correspondence between a class associated with the object can be used to automatically eliminate objects that do not satisfy conditions relating to the direction of gaze and / or the object class. This makes it possible to not have to take these objects into account for the calculations of intersection of the rays, reducing the calculation time required.

[0118] The various examples of step E2 for determining intention can be implemented in the case where only one virtual object 120 is present in the virtual environment. This makes it possible to determine the gripping clearance 40 and the intermediate clearance 60 only when the value of the intention parameter 20 is sufficient, for example when the virtual hand 100 is close to the virtual object 120. This saves calculation time.

[0119] 5.4. Construction of the grasping state

[0120] In step E4 of determining the third set of parameters 40, the gripping state is constructed. This gripping state can be constructed from the determination of a ray starting from the palm and intersecting the surface of the object. This ray can for example be the ray determined in step E4 of determining the values ​​of the intention parameters, and be the ray having the smallest weighted distance. This ray can alternatively be calculated in step E6, in particular when the method does not include step E4 of determining the values ​​of the intention parameters.

[0121] From this radius thus determined, a projection of the virtual hand is simulated in the direction of said radius and with the palm of the projection of the virtual hand 100 in contact with the object 120. Once this projection has been carried out, and therefore the projection of the virtual hand 100 in contact with the object 120, the fingers of the projection are closed on the object. Once the phalanges of all the fingers are closed as much as possible, the angles of the phalanges with their neighbors and / or the palm together define the values ​​of the parameters representative of the grasping state. This construction of the values ​​of the parameters representative of the grasping state may include a step of verifying the absence of interpenetration between the object and the virtual hand. This verification of the absence of interpenetration may be carried out at each iteration of the closing of the phalanges.

[0122] 5.5 Frequency of implementation of steps

[0123] A method has been described for providing a graphical representation of a virtual hand as a function of a first set of parameters (relating to an initial state and an initial position of a virtual hand) and at least a second set of parameters (relating to an object). This method comprises, as described above:

[0124] E2 (optional) determining an intention to grasp the virtual object 120 by the virtual hand 100

[0125] E4 the determination of a third set of parameters 40,

[0126] E6 determining at least a fourth set of parameters 60 as a function of this third set of parameters, and

[0127] E8 the generation of the graphical representation of the virtual hand from the first state at the first position (of the first parameter set) to the second state at the object position (of the third parameter set) through the third state at the second position (of the fourth parameter set). These steps E2 to E8 can be performed during the generation of each image of the graphical representation of the virtual hand. For example, for a virtual hand of the virtual hand type in virtual reality, the frequency of the graphical representation can be 90 Hz.

[0128] In an exemplary embodiment, these steps are not implemented every time an image is generated. Some steps may be performed every n images (e.g., every 10 images).

[0129] According to an exemplary embodiment, steps E4 and E6 (and possibly step E2 when it is part of the method) are implemented every n images (with n > 2). In fact, during the n-1 images where steps E4 and E6 (and possibly step E2) are not implemented, the last determined gripping set 40 and the last intermediate set(s) 60 are used. This advantageously makes it possible not to recalculate the gripping set 40 and the intermediate set(s) at all images, which reduces the calculation time for carrying out this method, without sacrificing too much realism as long as the virtual hand 100 and / or the object 120 do not move too quickly. When step E2 is part of the method, its frequency may be identical to that of steps E4 and E6, or different.According to one example, the gripping set 40 is determined every ten images, and the intermediate set(s) 60 are also determined every ten images (based on the gripping set 40 and the first set 10).

[0130] 5.7. Device

[0131] [Fig. 5] represents a device capable of implementing the graphic representation method according to Figure 1.

[0132] Such a device may comprise at least one hardware processor 51, a storage unit 52, and at least one network interface 53 which are connected to each other through a bus 54. Of course, the constituent elements of the device may be connected by means of a connection other than a bus.

[0133] The processor 51 controls the steps of the method. The storage unit 52 stores at least one program for implementing the method according to an embodiment of the invention to be executed by the processor 51, and various data, such as parameter sets used for calculations performed by the processor 51, intermediate data of calculations performed by the processor 51, etc. The processor 51 may be formed by any known and suitable hardware or software, or by a combination of hardware and software. For example, the processor 51 may be formed by dedicated hardware such as a processing circuit, or by a programmable processing unit such as a central processing unit (Central Processing Unit) which executes a program stored in a memory thereof.

[0134] The storage unit 52 may be formed by any suitable means capable of storing the program(s) and data in a computer-readable manner. Examples of the storage unit 52 include non-transitory computer-readable storage media such as semiconductor memory devices, and magnetic, optical, or magneto-optical recording media loaded into a read-write unit.

[0135] Reference is made to Figure 6, which represents an example of architecture of a virtual reality device according to one aspect of the invention. The virtual reality device 1 comprises a first graphic representation generator 2 of the virtual hand 100, a second graphic representation generator 3 of the object 120, and a graphic reproducer 4. These three elements can communicate together. The first generator 2 is capable of implementing a graphic representation of a virtual hand 100 as a function of a set of parameters (for example the first, third or fourth set of parameters described above) representative of the state of the virtual hand 100 and a position of said virtual hand 100 in its virtual environment.

[0136] The second generator 3 is capable of implementing a graphical representation of the object 120 as a function of at least one set of parameters (for example the second set of parameters 12 mentioned above) representative of a geometry of at least one object 120 intended to be gripped by the virtual hand 100 and of a position of said object 120 in said virtual environment.

[0137] The graphics reproducer 4 is configured to reproduce the virtual environment, the generated graphical representation of the virtual hand 100, and the generated graphical representation of the object 120.

[0138] The first graphic representation generator 2 comprises a calculator 5 determining at least a fourth set of parameters 60 representative of at least a third intermediate state of the virtual hand 100 between a first state and a second state of the virtual hand 100, as a function of a first set of parameters 10, of a third set of parameters 40 representative of a second state of the virtual hand 100 in a second position in the virtual environment, and of at least one item of information representative of a third position of the virtual hand 100 located between the first position and the second position, the third set of parameters being a function of the first set of parameters 10 and of the second set of parameters 12, the virtual hand gripping the object 120 in one of the states among the first state and the second state in a position among respectively the first position and the second position corresponding to the position of the object.

[0139] The third state makes it possible to implement a generation of a graphical representation of the virtual hand 100 from the first state at the first position to the second state at the second position via the third state at the third position (such as the generation implemented in the aforementioned step E8).

[0140] The virtual reality device 1 may comprise the hardware bricks shown in Figure 5 and described above in relation to this Figure 5.

Claims

DEMANDS 1. A method for generating a graphical representation of a virtual hand (100) as a function of a first set of parameters (10) representing a first state of a virtual hand (100) and a first position of said virtual hand (100) in a virtual environment, and of at least a second set of parameters (12) representing a geometry of at least one object (120) intended to be grasped by said virtual hand (100) and a position of said object (120) in said virtual environment, the method comprising the following steps: a determination (E6) of at least a fourth set of parameters (60) representing at least a third state of the virtual hand (100) intermediate between the first state and a second state of the virtual hand (100), as a function of the first set of parameters (10), of a third set of parameters (40) representing the second state of the virtual hand (100) in a second position in the virtual environment,and at least one piece of information representing a third position of the virtual hand (100) located between the first position and the second position, the third set of parameters being a function of the first set of parameters (10) and the second set of parameters (12), the virtual hand gripping the object (120) in one of the states among the first state and the second state in a position among respectively the first position and the second position corresponding to the position of the object, the third state allowing a generation (E8) of a graphical representation of the virtual hand (100) from the first state at the first position to the second state at the second position via the third state at the third position.

2. Method according to claim 1, wherein the method comprises a determination (E4), as a function of the first set of parameters (10) and the second set of parameters (12), of the third set of parameters (40) representing the second state of the virtual hand (100) in the second position.

3. Method according to claim 2, wherein the parameters of the third set of parameters (40) are representative of a second state of the virtual hand (100) when it grips the object (120).

4. A method according to any one of the preceding claims, wherein the method comprises the generation (E8) of the graphic representation of the virtual hand (100) from the first state at the first position to the second state at the position of the object via the third state at the second position.

5. A method according to any one of the preceding claims, wherein a state of the virtual hand is defined by a plurality of parameters comprising a list of angle values, an angle value describing an angle formed by two successive phalanges of a finger of the virtual hand or an angle formed by a phalanx and the palm of said virtual hand.

6. A method according to any one of the preceding claims, wherein said at least a fourth set of parameters further comprises a transition parameter whose value is between a minimum value corresponding to the first state and the first position of the virtual hand and a maximum value corresponding to the position of the object and the second state of the virtual hand.

7. A method according to claim 5 and claim 6, wherein at least one angle value is a function of the value of the transition parameter.

8. Method according to claim 7, wherein the function linking the value of the transition parameter to a given angle value is a linear interpolation passing through an angle value corresponding to the minimum value of the transition parameter and through an angle value corresponding to the maximum value of the transition parameter.

9. A method according to any one of claims 1 to 5, wherein the first set of parameters is further representative of an orientation of the virtual hand in the virtual space, the method further comprising: a determination (E20), as a function of the position of the virtual hand, the orientation of the virtual hand and the position of the virtual object, of an intention parameter whose value represents a probability of the user's intention to grasp said object with said virtual hand; a determination (E22) of the satisfaction, by said intention parameter, of a first condition, said satisfaction leading to the determination of the third set of parameters and the determination of said at least a fourth set of parameters.

10. Method according to claim 8, wherein the virtual environment comprises a plurality of objects, the first condition is satisfied for the object associated with said intention parameter having the highest value.

11. A method according to claim 10, wherein each object is associated with at least one class, and wherein the value of the second parameter associated with said object is a function of the at least one class associated with the object.

12. A method according to claim 8 or 9, further comprising obtaining data representative of a user's gaze direction, the intention parameter associated with an object taking a value that is all the greater as the gaze data indicate that the user is looking in the direction of said object.

13. Computer device for implementing a graphic representation method according to one of the preceding claims.

14. Product computer program comprising program code instructions for the implementation of a control method according to any one of claims 1 to 12, when executed by a processor.

15. A virtual reality device comprising: a first generator for a graphic representation (2) of a virtual hand (100) based on a set of parameters (10, 40, 60) representing a state of a virtual hand (100) and a position of said virtual hand (100) in a virtual environment; a second generator for a graphic representation (3) of an object (120) based on at least one set of parameters (12) representing the geometry of at least one object (120) intended to be grasped by a virtual hand (100) and a position of said object (120) in said environment virtual, a graphical reproducor (4) configured to reproduce the virtual environment, the generated graphical representation of the virtual hand (100), and the generated graphical representation of the object (120); in which the first graphical representation generator (2) of a virtual hand (100) comprises a computer determining at least a fourth set of parameters (60) representative of at least a third state of the virtual hand (100) intermediate between a first state and a second state of the virtual hand (100), as a function of a first set of parameters (10), a third set of parameters (40) representative of a second state of the virtual hand (100) in a second position in the virtual environment, and at least one piece of information representative of a third position of the virtual hand (100) located between the first position and the second position,the third set of parameters being a function of the first set of parameters (10) and the second set of parameters (12), the virtual hand gripping the object (120) in one of the states between the first state and the second state in a position between respectively the first position and the second position corresponding to the position of the object, the third state allowing a generation (E8) of a graphical representation of the virtual hand (100) from the first state at the first position to the second state at the second position via the third state at the third position.

Images