Processing of immersive reality data as a function of proximity between objects

Abstract

The description relates to a method, performed by an immersive reality system, for generating an immersive environment comprising a representation of at least one second virtual element (AV2), the method comprising: modifying a transparency value of a transparency parameter of the representation of the second virtual element (AV2) in the immersive environment, as a function of a distance from a first element (AV1).

Description

Description Title: Processing immersive reality data as a function of object proximity technical field

[0001] This description relates to data processing in the context of virtual reality or augmented reality, or more generally immersive reality. Previous technique

[0002] Immersive realities result from the reproduction, often three-dimensional, of an environment in such a way that the user has the impression of moving within it. In particular, the concept of environmental reproduction was introduced primarily with the emergence of virtual reality, made possible by immersive reality systems that often include so-called "virtual reality" headsets. Such headsets have screens that allow a user to view a virtual environment in 3D, or even a mixed environment: virtual and real (typically using sensors from a real-world environment). These immersive reality systems now encompass systems ranging from virtual reality to mixed reality, including augmented reality and augmented virtuality. The virtual environments reproduced by these immersive reality systems are: - universes generated ab nihilo, also called "virtually generated environment", or - existing places, also called "real environments", possibly remote, in which the user has the impression of moving.

[0003] Each virtual environment is composed of one or more virtual elements, including virtual objects and / or avatars (commonly referred to as "virtual elements" hereafter), that is, virtual people (a virtual representation of a real person, replicating the gestures, postures, and other characteristics of the real person). The virtual element is then reproduced in its position and form from the real environment at a given moment in time within the virtual environment. In particular, in the case of a shared virtual environment, several users of the immersive reality system can be represented in the virtual environment as their respective avatars.

[0004] In particular, when a virtual environment corresponds to the reproduction of a real environment in real time, the virtual element reproduced at time t corresponds to the real object captured at time t; that is to say, the position and shape of the virtual element reproduced in the virtual environment at that time t correspond to the position and shape of the real element captured in the real environment at time t. The time t of reproduction, during real-time reproduction, corresponds to the time t of capture plus possibly a processing / transmission delay between the sensor and the reproduction system, knowing that the delays, in particular those of transmission and / or processing of the captured data, are negligible in practice.

[0005] Reproducing multiple elements (real or virtual) in the same position within the immersive environment can lead to various problems. For example, two objects from two distinct environments (one real and the other virtual) might end up together. "superimposed" in the same position of the immersive environment, which poses a problem of definition and representation of one element in relation to the other.

[0006] For example, when one of the elements is a user's avatar, that user may have difficulty perceiving the virtual element positioned at the same point as the avatar in the immersive environment. Furthermore, the user may have difficulty maintaining this position when at the same observation point due to a common human instinct, known as the "avoidance instinct," linked to respecting a sphere of proxemics (such as a sphere of privacy).

[0007] One possible solution for when an avatar overlaps with a virtual element associated with an object in the immersive environment is to simply make the virtual element transparent and leave the avatar in its full ("normal" representation). However, this solution does not resolve the previous problem when the virtual element is in close proximity to the avatar, without overlapping it. Typically, when the avatar interacts with the virtual element associated with the object, no solution currently exists to illustrate such an interaction in the immersive environment. Furthermore, the aforementioned solution does not allow the user to perceive all the virtual elements present in the environment, which hinders interactions in a later phase with virtual elements that would be rendered completely transparent.

[0008] This description improves the situation. Summary of the presentation

[0009] To this end, it proposes a method for generating, by an immersive reality system, an immersive environment comprising a representation of at least one second virtual element, the method comprising: modifying a transparency value of a transparency parameter of the representation of the second virtual element in the immersive environment, as a function of a distance with a first element, the method comprising the application of a transparency inversely proportional to a distance between the first element and the second virtual element.

[0010] This implementation notably improves user visibility. By adjusting the transparency of virtual elements based on distance, users can typically see and interact with important elements of the immersive environment more effectively. This allows users to focus on essential tasks without being distracted by irrelevant virtual elements, thanks to optimized interaction.

[0011] In a realization, the said function of distance is decreasing.

[0012] The aforementioned transparency can be applied in real time according to said distance, determined between a current position of the first element and a current position of the second virtual element.

[0013] For example, the first element could correspond to a user of an immersive reality system. This person is real and sees themselves directly in mixed reality, while also seeing the second virtual element within the immersive environment. They see, for example, their hands, torso, etc., interacting with the second, virtual element. In this case, the aforementioned current position of the first element is that of the system user and corresponds, for example, to their viewpoint within the immersive environment.

[0014] In a production, said distance is determined according to a scale of reproduction of the immersive environment.

[0015] As an alternative to the realization where the first element is a real element (such as the user of the immersive reality system for example), the first element can be virtual.

[0016] For example, the first element could be an avatar of the user of the immersive reality system.

[0017] In such a design, the transparency value is increased if said distance becomes less than a threshold based on a radius of a sphere around the avatar.

[0018] Typically, in the case where the second virtual element is an avatar of another user, said sphere can correspond to a sphere of proxemics.

[0019] This approach reduces proxemic conflicts because variable transparency avoids social awkwardness related to the blurring of boundaries between personal spaces, while still maintaining an appropriate level of social presence. Furthermore, the immersive rendering system can be adapted to different proxemic distances desired by a user, offering a personalized experience based on user preferences and environmental scale, for example.

[0020] It has been observed, during a projection in an immersive environment, that users tend to adopt the same interaction rules as in the real world. Indeed, the spheres of intimacy, or more broadly, the distances from virtual elements representing objects or avatars of other users, are generally adopted by users to be the same as in the real world.

[0021] In practice, being in the same location as a virtual element or another remote user represented by a different avatar has no physical impact, as no physical contact occurs. However, if two avatars are in the same location, the intrusion into each user's personal space by their respective avatars can be socially awkward and hinder collaborative work. Furthermore, another user's avatar can block the view of an element for which a task is in progress within the virtual environment, thus impeding interaction.

[0022] In a design, transparency can then be applied from a distance threshold between the two avatars of between 1 and 2 meters.

[0023] Alternatively or in addition, the transparency can be 50% for a distance between the two avatars, between 0.3 and 1 meter.

[0024] Alternatively or in addition, maximum transparency (for example 100%) is achieved for a distance between the two avatars of between 0 and 0.15 meters.

[0025] According to another aspect, a computer program is proposed comprising instructions to execute the steps of the process of the type described above, when said instructions are executed by a processor.

[0026] In another aspect, a computer program is proposed that includes instructions for implementing all or part of a process as defined herein when executed by a processor. In another aspect, a non-transient, computer-readable recording medium is proposed on which such a program is recorded.

[0027] In another aspect, a device is proposed that includes a processing circuit for implementing the process. Brief description of the drawings

[0028] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: Fig. 1

[0029] [Fig. 1] shows an example of an implementation illustrating the principle of transparency applied as a (decreasing) function of the distance between two elements AV1 and AV2. Fig. 2

[0030] [Fig. 2] shows a possible embodiment of steps in a process of the type defined above. Fig. 3

[0031] [Fig. 3] shows one possible embodiment of a device of the type defined above. Description of the implementation methods

[0032] Reference is now made to Figure 1, which illustrates the principle of transparency of at least one virtual element (here, a second avatar AV2) as a function of its distance from another given element (for example, another avatar, referred to hereafter as the "first avatar AV1"). The first avatar AV1 could, for example, be that of a user of the virtual reality system, allowing the user to immerse themselves in a virtual environment.

[0033] On the left of Figure 1, the first avatar AV1 is shown in an ideal position relative to another virtual element, such as the second avatar AV2 in the illustrated example.

[0034] In the center of Figure 1, the second avatar AV2 becomes more transparent as the first avatar AV1 approaches it, illustrating a transparency formula that is a decreasing function of the distance D between the two avatars AV1 and AV2. This transparency can begin to be applied to the pixels of the second avatar AV2 when the distance D falls below a first threshold THR1, which could correspond, for example, to a social sphere, examples of which are given later.

[0035] For example, the aforementioned decreasing function could be of the type: Transparency=k / (D-THR2), where k is a constant, D the distance between the first and second avatars and THR2 a second only, less than the first threshold THR1, for example of the beginning of superposition of the two avatars.

[0036] It is therefore proposed to use the distance D as a variable to define the level of transparency to be applied to the pixels representing a virtual element in the environment of the user's avatar AV1, this virtual element being the avatar AV2 in the example illustrated in Figure 1.

[0037] To the right of this figure 1, the virtual element becomes completely transparent when the first avatar is very close.

[0038] Figure 1 illustrates a dynamic transparency mechanism applied to virtual elements based on their distance from the user's avatar, as shown in the example. It demonstrates how transparency gradually increases as the user or their avatar approaches a virtual element, granting the user improved visibility and, consequently, enhanced interaction with elements of the immersive environment. This mechanism effectively reduces visual interference and maintains a smooth immersive experience, while also respecting defined proxemic distances to prevent social awkwardness.

[0039] Here, transparency is applied to virtual elements based on their distance from a given element, such as a user's avatar. However, other variations are possible. For example, the user's avatar could be replaced by another element, such as a robot or a tool (possibly controlled by commands from a device operated by the user). Again, applying transparency to virtual elements near such a tool allows the user operating the tool to have better visibility and interaction with the virtual elements in the environment. Thus, the element from which distances and associated transparencies are defined is not necessarily an avatar, but could, for example, be a virtual element associated with another object.

[0040] However, in the case of a virtual element such as a user avatar (AV1), this implementation allows for maintaining a social presence when the distance remains greater than a sphere of intimacy, and then for avoiding automatic user behaviors linked to social rules when this distance (D) decreases. This implementation allows the user to have a better perspective, to limit extraneous information, and to focus on what is important within the immersive environment. For example, during collaboration within a virtual space. shared between several users with several respective avatars, such a creation makes it possible to focus a collaborative task as the central subject of interactions.

[0041] Figure 2 presents a processing scheme to adjust the transparency of at least one virtual element OBJ2 in an immersive environment based on its position relative to another element OBJ1, for example an avatar.

[0042] In step S1, one or more objects of interest, present in space E1 (which can be real or virtual), are listed, and their respective positions are determined first in space E1 in step S3.

[0043] In an example of implementation, symmetrically, at step S2, one or more objects of interest, present for example in another space E2 (which can also be real or virtual), are listed, and their respective positions are first determined in space E2 at step S4.

[0044] Of course, objects from a third or even a fourth space, or others, can be considered, without limitation to simply taking into account two spaces E1, E2.

[0045] Next, in step S5, from the positions in spaces E1 and E2, the positions of each of these objects are determined together in the global immersive environment El. It is from there that distances between objects can be calculated in the immersive environment, at the scale of the immersive environment.

[0046] At step S6, a test is carried out to determine if there are at least two elements OBJ1 and OBJ2 separated from each other by a distance less than a THR threshold in the global immersive environment El, with typically a risk of overlap when the two elements OBJ1, OBJ2 come from two different respective spaces E1, E2.

[0047] If this is the case at the output of test S6, then in step S7, the pixels representing the virtual element OBJ2 are made transparent, typically by decreasing the opacity of its representation. In augmented or virtual reality, this opacity parameter for representing virtual elements can be adjusted as easily as adjusting the intensity of a color (red, blue, or green). For purely illustrative purposes, this opacity (or conversely, transparency) can be seen as a mixture: - the pixel colors of the background of the immersive environment El, on the one hand, and - initial pixel colors of the virtual element OBJ2, on the other hand, then the aforementioned transparency is applied with a degree of transparency intended to assign more or less weight to the background colors of the environment El in the aforementioned blend. This degree of transparency is calculated according to a decreasing function of the distance D between the two elements OBJ1, OBJ2. The closer the elements are, the higher the degree of transparency (and the greater the weight of the background colors of the environment El in the aforementioned "blend").

[0048] This section describes the application of transparency to the pixels of the virtual element OBJ2, while the other element OBJ1 retains its original pixel colors. This is particularly relevant when the other element OBJ1 is virtual and represents the avatar of a user who is supposed to be interacting within the system. The immersive environment El. This user is equipped with a virtual or augmented reality headset featuring a screen displaying pixels to which the aforementioned transparency (or conversely, opacity) can be applied. Modulating the transparency of virtual elements in the immersive environment, which are close to the avatar (or even "overlaid" on it), according to their distance from the avatar, improves the user's interaction with the elements of the immersive environment via their avatar and, more generally, improves the user's visibility within the immersive environment (which is no longer cluttered with irrelevant virtual elements).

[0049] Of course, such a implementation is presented here as an example. The elements OBJ1 and OBJ2 can also originate from a single space E1. In this case, the interaction of the avatar OBJ1 with a virtual element OBJ2 from the same space can typically be enhanced by applying transparency based on the distance separating OBJ1 from OBJ2.

[0050] Furthermore, the OBJ1 element can be virtual (such as an avatar) or real. Indeed, the OBJ1 element may not necessarily be represented in the immersive environment. Typically, for an immersive application, the first space E1 may simply correspond to the viewpoint of the real user (without possessing an avatar in the immersive environment E1) in an augmented reality application where the user already sees their physical body, as well as representations of the surrounding elements of a space via their virtual reality headset. This can be the case, for example, in a video game where the user plays alone, in first-person view. Thus, the distances of other elements are measured from the user's viewpoint (real object) in this case.

[0051] It will thus be understood that the term "immersive environment" generally refers to an environment that can be virtual or mixed: real and virtual, and that can be either 3D or 2D, for example in the case of a video game that is played on a 2D screen.

[0052] In all cases, whether the user is represented by an avatar or remains a real element (its current position defining a viewpoint in the immersive environment), all or part of the elements surrounding it can be represented in the immersive space with a level of transparency (or conversely opacity) that increases (decreases, respectively) when the distance between one of these elements and the user decreases.

[0053] To avoid social awkwardness when at least one of the elements surrounding the user is an avatar, so-called "proxemic" distances between the user and that avatar can be taken into account, such as: - the so-called "intimate" distance: less than 40 cm (close mode: less than 15 cm, far mode: from 15 cm to 40 cm): in this case, the THR1 threshold can be for example 40cm and the THR2 threshold can be 15 cm; - the so-called "personal" distance: from 45 cm to 125 cm (close mode: from 45 cm to 75 cm, far mode: from 75 cm to 125 cm): in this case for example, the THR1 threshold can be 125cm, the transparency function can be set so that it is equal to 50% at 45 cm and the THR2 threshold (100% transparency) can remain at 15 cm as before; - The so-called "social" distance: from 120 cm to 360 cm (close mode: from 120 cm to 210 cm, far mode: from 210 cm to 360 cm) can also be used to refine the shape of the function of transparency mentioned above, particularly for the first threshold THR1 for example; - the so-called "public" distance: beyond 360 cm (close mode: from 360 cm to 750 cm, far mode: beyond 750 cm), can also be used to refine the shape of the aforementioned transparency function, or to modify another parameter such as the color temperature depending on the distance: blue if the distance is greater than 750 cm and more "red" if the distance becomes less than 360 cm, signifying a possible start of social interaction.

[0054] However, the distances used to define the transparency function may be based on user preferences or may be evaluated by the immersive reality system individually for each user, particularly prior to first use, based on tests on a virtual object or another avatar.

[0055] Similarly, as an illustration, a transparency function of the form Transparency=k / (D-THR2) was presented above. However, other implementations besides the simple inverse of the distance D are possible, notably the inverse of a logarithmic function or a polynomial (of order greater than or equal to 2, for example). Typically, it is possible to adjust the coefficients of this function according to desired key values, for example, 100% at 15 cm, 50% at 45 cm, and 0% at 125 cm and above.

[0056] Figure 3 illustrates an example of a device for implementing such a process, which in this example includes: - a CAS immersive reality headset worn by a UT user (from which the distances of surrounding elements can be calculated, from their point of view and / or from their avatar), - possibly one or more CAP sensors from a real environment to feed the immersive environment (for example, relative to a current position of the user in order to define a position of their avatar and / or their point of view), - a CT processing circuit connected to or integrated into the CAS headset and comprising: - an IN input to receive, in particular, user preferences regarding distance thresholds from which to apply transparency, and specifically data from the CAP sensor, - a MEM memory for storing at least instructions from a computer program for implementing the above process, - a PROC processor capable of cooperating with the MEM memory to read and execute the aforementioned instructions in order to apply transparency to the pixels of the video signals played on the CAS headsets, depending on the aforementioned distance D, and - at least one OUT output interface to deliver such signals, suitable for being played on the CAS headset.

[0057] The purpose of this description is particularly relevant to collaborations (especially between users) in mixed realities, video games, metaverses, virtual spaces, etc., and This makes it easier for users to understand such immersive environments, especially when they are loaded with objects or avatars.

[0058] Typically, Figure 3 only illustrates a headset worn by a user as an example, and the user may, for example, use one or more screens, particularly in the context of a video game played, for example, with a controller by the user, where the above process may also be applicable.

Claims

Demands

1. A method for generating, by an immersive reality system, an immersive environment comprising a representation of at least one second virtual element, the method comprising: modifying a transparency value of a transparency parameter of the representation of the second virtual element in the immersive environment, as a function of a distance with a first element, the method comprising applying a transparency inversely proportional to a distance between the first element and the second virtual element.

2. A method according to claim 1, wherein said distance function is decreasing.

3. A method according to any one of the preceding claims, wherein said transparency is applied in real time as a function of said distance, determined between a current position of the first element and a current position of the second virtual element.

4. A method according to any one of the preceding claims, wherein said distance is determined as a function of a reproduction scale of the immersive environment.

5. A method according to any one of the preceding claims, wherein the first element is virtual.

6. A method according to claim 5, wherein the first element is an avatar of a user of the immersive reality system.

7. A method according to claim 6, wherein the transparency value is increased if said distance becomes less than a threshold (THR1) as a function of a sphere radius around the avatar.

8. A method according to claim 7, wherein the second virtual element is an avatar of another user.

9. Method according to claim 8, wherein the transparency is applied from a threshold (THR1) of distance between the two avatars (AV1, AV2) of between 1 and 2 meters.

10. A method according to any one of claims 8 and 9, wherein the transparency is 50% for a distance between the two avatars (AV1, AV2) of between 0.3 and 1 meter.

11. A method according to any one of claims 8 to 10, wherein the transparency is maximal for a distance between the two avatars (AV1, AV2) of between 0 and 0.15 meters.

12. A computer program comprising instructions for executing the steps of the process according to any one of the preceding claims, when said instructions are executed by a processor.

13. Device comprising a treatment circuit (TC) for implementing the process according to any one of claims 1 to 11.

Images