Immersive reality data processing based on object proximity

Abstract

The description concerns a method for generating, using an immersive reality system, an immersive environment comprising a representation of at least one second virtual element (AV2). The method includes modifying the transparency value of a transparency parameter of the representation of the second virtual element (AV2) in the immersive environment, based on its distance from a first element (AV1). Abstract figure: Figure 1

Description

Title of the invention: Processing of immersive reality data based on proximity between objects technical field

[0001] This description relates to data processing in the context of virtual reality or augmented reality, or more generally immersive reality. Prior art

[0002] Immersive realities result from the reproduction, often three-dimensional, of an environment in such a way that the user has the impression of moving within that environment. In particular, the concept of environment reproduction was primarily introduced with the emergence of virtual reality, made possible by immersive reality systems that often include so-called "virtual reality" headsets. Such headsets have screens that allow a user to view a virtual environment in 3D, or even a mixed environment: virtual and real (typically from sensors of a real environment). These immersive reality systems now include systems ranging from virtual reality to mixed reality, notably including augmented reality and augmented virtuality. The virtual environments reproduced by these immersive reality systems are: - universes generated ab nihilo, also called "virtually generated environment", or - existing places, also called "real environments", possibly remote in which the user has the impression of evolving.

[0003] Each reproduced virtual environment consists of one or more virtual elements, including virtual objects and / or avatars (hereinafter referred to as "virtual elements"), i.e., virtual people (a virtual representation of a real person, replicating the gestures, postures, and other characteristics of the real person). The virtual element is then reproduced in its position and form from the real environment at a given moment t of its evolution within the virtual environment. In particular, in the case of a shared virtual environment, several users of the immersive reality system can be represented in the virtual environment in the form of their respective avatars.

[0004] In particular, when a virtual environment corresponds to the reproduction of a real environment in real time, the virtual element reproduced at time t corresponds to the real object captured at time t, that is to say, in particular, that the position and shape of the virtual element reproduced in the virtual environment at that time t correspond to the position and shape of the real element captured in the real environment at time t. t. The reproduction time t, during real-time reproduction, corresponds to the capture time t plus possibly a processing / transmission delay between the sensor and the reproduction system, knowing that the delays in particular of transmission and / or processing of the captured data are negligible in practice.

[0005] Reproducing several elements (real or virtual) in the same position within the immersive environment can generate various problems. For example, two objects from two distinct environments (for example, one real and the other virtual) can end up "superimposed" in the same position within the immersive environment, which poses a problem of defining and representing one element in relation to the other.

[0006] For example, when one of the elements is a user's avatar, that user may experience difficulty perceiving the virtual element positioned at the same point as the avatar in the immersive environment. The user may also experience difficulty maintaining this position when at the same observation point due to a common human instinct, known as the "avoidance instinct," linked to respecting a so-called sphere of proxemics (such as a sphere of privacy, typically).

[0007] One solution that could be implemented in the case of an avatar overlapping with the virtual element associated with an object in the immersive environment is to simply make the virtual element transparent and leave the avatar in a full ("normal") representation. However, this solution does not resolve the previous problem when the virtual element is in the immediate vicinity of the avatar, without overlapping it. Typically, when the avatar interacts with the virtual element associated with the object, no solution currently exists to illustrate such an interaction in the immersive environment. Furthermore, the aforementioned solution does not allow the user to perceive all the virtual elements present in the environment, which does not facilitate interactions in a subsequent phase with virtual elements that would be rendered completely transparent.

[0008] The present description improves the situation. Summary of the presentation

[0009] To this end, it proposes a method for generating, by an immersive reality system, an immersive environment comprising a representation of at least one second virtual element, the method comprising: modifying a transparency value of a transparency parameter of the representation of the second virtual element in the immersive environment, as a function of a distance with a first element.

[0010] Such an implementation makes it possible, in particular, to improve a user's visibility. By adjusting the transparency of virtual elements according to distance, the Users can typically see and interact with important elements of the immersive environment more effectively. This allows users to focus on essential tasks without being distracted by irrelevant virtual elements, thanks to optimized interaction.

[0011] In one embodiment, said distance function is decreasing.

[0012] In one embodiment, the process involves the application of a transparency inversely proportional to a distance between the first element and the second virtual element.

[0013] The aforementioned transparency can be applied in real time as a function of said distance, determined between a current position of the first element and a current position of the second virtual element.

[0014] For example, the first element may correspond to a user of an immersive reality system. The person is then real and sees themselves directly in mixed reality and also sees the second virtual element in the immersive environment. They see, for example, their hands, their torso, etc., interacting with the second element, which is virtual. In this case, typically, the aforementioned current position of the first element is that of the system user and corresponds, for example, to their viewpoint of the immersive environment.

[0015] In one embodiment, said distance is determined according to a scale of reproduction of the immersive environment.

[0016] Alternatively to the embodiment where the first element is a real element (such as the user of the immersive reality system for example), the first element can be virtual.

[0017] For example, the first element may be an avatar of the user of the immersive reality system.

[0018] In such an embodiment, the transparency value is increased if said distance becomes less than a threshold function of a radius of sphere around the avatar.

[0019] Typically, in the case where the second virtual element is an avatar of another user, said sphere can correspond to a sphere of proxemics.

[0020] Such an implementation reduces proxemic conflicts because variable transparency avoids social awkwardness related to the interpenetration of spheres of intimacy, while maintaining an appropriate social presence. Furthermore, the immersion rendering system can be adapted to the different proxemic distances desired by a system user, offering a personalized experience based on user preferences and the scale of the environment, for example.

[0021] It has been observed, in fact, during a projection in an immersive environment, that users tend to adopt the same rules of interaction as in the real world. Indeed, the spheres of intimacy, or more broadly, the distances with Virtual elements representing objects or avatars of other users are generally adopted by users to be the same as in the real world.

[0022] In practice, being in the same location as a virtual element or another remote user embodied by a different avatar has no physical impact because no physical contact takes place. However, if two avatars are in the same location, the intrusion into the personal space of each user embodied by their respective avatar can be socially awkward and hinder a collaborative task. Furthermore, the avatar embodied by another user can block the view of an element for which a task is in progress in the virtual environment, thus hindering interaction.

[0023] In an embodiment, transparency can then be applied from a distance threshold between the two avatars of between 1 and 2 meters.

[0024] Alternatively or in addition, the transparency can be 50% for a distance between the two avatars, between 0.3 and 1 meter.

[0025] Alternatively or in addition, the transparency is maximal (for example 100%) for a distance between the two avatars, between 0 and 0.15 meters.

[0026] According to another aspect, a computer program is proposed comprising instructions to execute the steps of the process of the type above, when said instructions are executed by a processor.

[0027] According to another aspect, a computer program is proposed comprising instructions for implementing all or part of a process as defined herein when this program is executed by a processor. According to another aspect, a non-transient, computer-readable recording medium is proposed on which such a program is recorded.

[0028] According to another aspect, a device comprising a processing circuit is proposed for the implementation of the process. Brief description of the drawings

[0029] Other features, details and advantages will become apparent from reading the detailed description below and from analyzing the accompanying drawings, in which: Fig. 1

[0030] [Fig. 1] shows an example of an embodiment illustrating the principle of transparency applied as a (decreasing) function of the distance between two elements AVI and AV2. Fig. 2

[0031] [Fig.2] shows a possible embodiment of steps of a process of the type defined above. Fig. 3

[0032] [Fig.3] shows a possible embodiment of a device of the type defined above. Description of the implementation methods

[0033] Reference is now made to [Fig. 1], which illustrates the principle of transparency of at least one virtual element (here a second avatar AV2) as a function of its distance from another given element (for example, here another avatar, referred to as the "first avatar AVI" hereafter). The first avatar AVI could, for example, be that of a user of the virtual reality system, allowing the user to immerse themselves in an immersive environment.

[0034] On the left of [Fig.1], the first avatar AVI is represented in an ideal position relative to another virtual element, such as the second avatar AV2 in the illustrated example.

[0035] In the center of [Fig. 1], the second avatar AV2 becomes more transparent as the first avatar AV1 approaches it, illustrating a transparency formula that is a decreasing function of the distance D between the two avatars AVI and AV2. This transparency can begin to be applied to the pixels of the second avatar AV2 when the distance D becomes less than a first threshold THR1, which can correspond, for example, to a social sphere, examples of which are given later.

[0036] For example, the aforementioned decreasing function can be of the type: Transparency=k / (D-THR2), where k is a constant, D the distance between the first and second avatars and THR2 a second only, lower than the first threshold THR1, for example of the start of superposition of the two avatars.

[0037] It is thus proposed to use the distance D as a variable to define the level of transparency to be applied to the pixels representing a virtual element in the environment of the user's AVI avatar, this virtual element being the AV2 avatar in the example illustrated in [Fig.1].

[0038] To the right of this [Fig.1], the virtual element becomes completely transparent when the first avatar is very close.

[0039] Thus, this [Fig. 1] shows a dynamic transparency mechanism applied to virtual elements as a function of their distance from the user's avatar, in the example shown. It demonstrates how transparency gradually increases as the user or their avatar approaches a virtual element, allowing the user better visibility and, consequently, improved interaction with elements of the immersive environment. This mechanism effectively helps to reduce visual interference and maintain a smooth immersive experience, as well as respecting the defined proxemic distances to avoid social awkwardness.

[0040] Here, transparency applies to virtual elements based on their distance from a given element, such as a user's avatar. However, other variations are possible. For example, the user's avatar can be replaced by another element, such as a robot or a tool (possibly controlled by commands from equipment operated by the user). Again, Applying transparencies to virtual elements near such a tool allows the user manipulating the tool to have better visibility and interaction with the virtual elements in the environment. Thus, the element from which the distances and associated transparencies are defined is not necessarily an avatar, but could be, for example, a virtual element associated with another object.

[0041] However, in the case of a virtual element such as a user's AVI avatar, such an implementation allows for maintaining a social presence when the distance remains greater than a sphere of intimacy, and then for avoiding automatic user behaviors related to social rules when this distance D decreases. Such an implementation allows the user to have a better perspective, to limit extraneous information, and to focus on what is important within the immersive environment. For example, during collaboration within a virtual space shared between several users with their respective avatars, such an implementation allows a collaborative task to be the central focus of the interactions.

[0042] Figure 2 shows a processing scheme for adjusting the transparency of at least a virtual element OBJ2 in an immersive environment depending on its position relative to another element OBJ1, for example an avatar.

[0043] In step S1, one or more objects of interest, present in the space El (which can be real or virtual), are listed, and their respective positions are determined first in the space El in step S3.

[0044] In an example of an embodiment, symmetrically, at step S2, one or more objects of interest, present for example in another space E2 (which can also be real or virtual), are listed, and their respective positions are first determined in space E2 at step S4.

[0045] Of course, objects of a third or even a fourth space, or others, can be considered, without limitation to simply taking into account two spaces El, E2.

[0046] Then, in step S5, from the positions in the spaces El and E2, the positions of each of these objects are determined together in the global immersive environment El. It is from there that distances between objects can be calculated in the immersive environment, on the scale of the immersive environment.

[0047] In step S6, a test is carried out to determine whether there are at least two elements OBJ1 and OBJ2 separated from each other by a distance less than a THR threshold in the global immersive environment El, with typically a risk of overlap when the two elements OBJ1, OBJ2 come from two different respective spaces El, E2.

[0048] If this is the case at the output of test S6, then in step S7, the pixels representing the virtual element OBJ2 are made transparent, typically by decreasing the opacity of its representation. In augmented or virtual reality, this opacity parameter for representing virtual elements can be adjusted as easily as adjusting the intensity of a color (red, blue, or green). For illustrative purposes only, this opacity (or conversely, the transparency) can be seen as a mixture: - the pixel colors of the background of the immersive environment El, on the one hand, and - the initial pixel colors of the virtual element OBJ2, on the other hand, then the aforementioned transparency is applied with a degree of transparency designed to give more or less weight to the background colors of environment El in the aforementioned blend. This degree of transparency is calculated according to a decreasing function of the distance D between the two elements OBJ1, OBJ2. The closer the elements are, the higher the degree of transparency (and the greater the weight of the background colors of environment El in the aforementioned "blend").

[0049] This section describes the application of transparency to the pixels of the virtual element OBJ2, while the other element OBJ1 retains its original pixel colors. This is particularly relevant when the other element OBJ1 is virtual and corresponds to the avatar of a user assumed to be navigating the immersive environment El. This user is equipped with a virtual or augmented reality headset featuring a screen displaying the pixels to which the aforementioned transparency (or conversely, opacity) can be applied. Modulating the transparency of virtual elements in the immersive environment, which are close to the avatar (or even "overlaid" on it), according to their distance from the avatar, improves the user's interaction with the elements of the immersive environment via their avatar and, more generally, enhances the user's visibility within the immersive environment (which is no longer cluttered with irrelevant virtual elements).

[0050] Of course, such an implementation is presented here by way of example. The elements OBJ1 and OBJ2 may also originate from one and the same El space. In this case, typically the interaction of the avatar OBJ1 with a virtual element OBJ2 from the same space can be improved by applying transparency based on the distance separating OBJ1 from OBJ2.

[0051] Furthermore, the OBJ1 element can be virtual (such as an avatar) or real. Indeed, the OBJ1 element may not necessarily be represented in the immersive environment. Typically, for an immersive application, the first space El may simply correspond to the viewpoint of the real user (without possessing an avatar in the immersive environment El) in an augmented reality application where the user already sees their physical body, as well as representations of the elements surroundings of a space via a virtual reality headset. This can be the case, for example, in a video game where the user plays alone, in first person. Thus, the distances of other elements are measured from the user's point of view (the real object) in this case.

[0052] It will thus be understood that the term "immersive environment" generally refers to an environment which can be virtual or mixed: real and virtual, and which can be either 3D or 2D, for example in the case of a video game which is played on a 2D screen.

[0053] In all cases, whether the user is represented by an avatar or remains a real element (its current position defining a viewpoint in the immersive environment), all or part of the elements surrounding it can be represented in the immersive space with a level of transparency (or conversely of opacity) which increases (decreases, respectively) when the distance between one of these elements and the user decreases.

[0054] To avoid social discomfort when at least one of the elements around the user is an avatar, so-called "proxemic" distances between the user and this avatar can be taken into account, such as: - the so-called "intimate" distance: less than 40 cm (close mode: less than 15 cm, far mode: from 15 cm to 40 cm): in this case, the THR1 threshold can be for example 40cm and the THR2 threshold can be 15 cm; - the so-called "personal" distance: from 45 cm to 125 cm (close mode: from 45 cm to 75 cm, far mode: from 75 cm to 125 cm): in this case for example, the THR1 threshold can be 125cm, the transparency function can be set so that it is equal to 50% at 45 cm and the THR2 threshold (100% transparency) can remain at 15 cm as before; - the so-called "social" distance: from 120 cm to 360 cm (close mode: from 120 cm to 210 cm, far mode: from 210 cm to 360 cm) can also be used to refine the shape of the aforementioned transparency function, in particular for the first threshold THR1 for example; - the so-called "public" distance: beyond 360 cm (close mode: from 360 cm to 750 cm, far mode: beyond 750 cm), can also be used to refine the shape of the aforementioned transparency function, or to modify another parameter such as the color temperature depending on the distance: blue if the distance is greater than 750 cm and more "red" if the distance becomes less than 360 cm, signifying a possible start of social interaction.

[0055] However, the distances used to define the transparency function may be based on user preferences or may be evaluated individually by the immersive reality system for each user, in particular prior to first use, based on tests on a virtual object or another avatar.

[0056] Similarly, by way of illustration, a transparency function of the type Transparency=k / (D-THR2) has been presented above. However, other implementations besides the simple inverse of the distance D are possible, notably the inverse of a logarithmic function or a polynomial (of order greater than or equal to 2, for example). Typically, it is possible to adjust the coefficients of this function according to desired key values, for example, 100% at 15 cm, 50% at 45 cm, and 0% at 125 cm and above.

[0057] Figure 3 illustrates an example of the embodiment of a device for implementing such a process, comprising, in this example: - a CAS immersive reality headset worn by a UT user (from which the distances of surrounding elements can be calculated, from their point of view and / or from their avatar), - possibly one or more CAP sensors from a real-world environment to feed the immersive environment (for example, relative to the user's current position to define the position of their avatar and / or their point of view), - a CT processing circuit connected to or integrated into the CAS headset and comprising: - an IN input to receive, in particular, user preferences regarding distance thresholds from which to apply transparency, and specifically data from the CAP sensor, - a MEM memory for storing at least instructions from a computer program for implementing the above process, - a PROC processor capable of cooperating with the MEM memory to read and execute the aforementioned instructions in order to apply transparency to the pixels of the video signals played on the CAS headsets, depending on the aforementioned distance D, and - at least one OUT output interface to deliver such signals, suitable for being played on the CAS headset.

[0058] The object of this description is particularly applicable to collaborations (especially between users) in mixed realities, video games, metaverses, virtual spaces, etc., and thus makes it possible to facilitate user understanding in such immersive environments, especially when they are loaded with objects or avatars.

[0059] Typically, [Fig.3] only illustrates a helmet worn by a user as an example, and the user may, for example, use one or more screens, particularly in the context of a video game played, for example, with a controller by the user, where the above process may also be applicable.

Claims

Demands

1. A method for generating, by an immersive reality system, an immersive environment comprising a representation of at least one second virtual element, the method comprising: modifying a transparency value of a transparency parameter of the representation of the second virtual element in the immersive environment, as a function of a distance from a first element.

2. A method according to claim 1, wherein said distance function is decreasing.

3. A method according to any one of the preceding claims, comprising the application of a transparency inversely proportional to a distance between the first element and the second virtual element.

4. A method according to any one of the preceding claims, wherein said transparency is applied in real time as a function of said distance, determined between a current position of the first element and a current position of the second virtual element.

5. A method according to any one of the preceding claims, wherein said distance is determined as a function of a reproduction scale of the immersive environment.

6. A method according to any one of the preceding claims, wherein the first element is virtual.

7. A method according to claim 6, wherein the first element is an avatar of a user of the immersive reality system.

8. A method according to claim 7, wherein the transparency value is increased if said distance becomes less than a threshold (THR1) that is a function of a sphere radius around the avatar.

9. A method according to claim 8, wherein the second virtual element is an avatar of another user.

10. Method according to claim 9, wherein transparency is applied from a threshold (THR1) of distance between the two avatars (AVI, AV2) of between 1 and 2 meters.

11. A method according to any one of claims 9 and 10, wherein the transparency is 50% for a distance between the two avatars (AVI, AV2) of between 0.3 and 1 meter.

12. A method according to any one of claims 9 to 11, wherein the transparency is maximized for a distance between the two avatars (AVI, AV2) of between 0 and 0.15 meters.

13. A computer program comprising instructions for executing the steps of the process according to any one of the preceding claims, when said instructions are executed by a processor.

14. Device comprising a treatment circuit (TC) for implementing the process according to any one of claims 1 to 12.

Images