METHOD FOR CREATING A VIRTUAL REPRESENTATION OF A REAL ENVIRONMENT, DEVICES AND CORRESPONDING SYSTEM

Description

1. Scope of the invention

[0001] The invention relates to the virtual representation of a real environment, such as a 2D plan or a 3D representation, and more particularly to the generation of a virtual representation of the real environment from a mixed virtual reality headset. 2. Prior Art

[0002] To generate a 3D representation of a real space, professionals generally use a 3D scanner to scan the room or space to be modeled. However, using such a device requires calibration and precise positioning within the space to be mapped for accurate measurements. Furthermore, the model obtained from data measured by a 3D scanner is not always optimal, especially when the space to be modeled contains obstacles, such as in a furnished room. The model then requires manual adjustments, which can be tedious and demand a certain level of expertise with CAD (Computer-Aided Design) tools. In addition, the cost of a 3D scanner is quite high.

[0003] Another method for generating a 3D representation of a real space is to use an existing 2D plan, either manually generated or obtained from a file. However, this method requires specialized skills in using modeling software. Furthermore, architectural 2D plans, when available, are sometimes slightly inaccurate compared to the actual construction.

[0004] Therefore, there is a need to improve the state of the art.

[0005] US 2019043259 A1 discloses the estimation of a safety zone in a VR environment; the modification of vertices of this zone via a controller and the display of warnings / collisions in the VR experience. 3. Description of the invention

[0006] It relates to a method of generating a virtual representation in at least 2 dimensions of a real environment, the generation being implemented by a mixed virtual reality headset intended to be worn by a user, the mixed virtual reality headset being associated with at least one interface device.

[0007] Advantageously, according to the invention, the generation process comprises: an acquisition of relative coordinates in the real environment, corresponding to a position of said interface device in the real environment, following an interaction of the user on said interface device, a determination as a function of said relative coordinates of the position of said interface device in the real environment of a corresponding point in the virtual representation, a generation of said virtual representation from at least the point associated with the relative coordinates of the acquired position of said interface device.

[0008] The invention thus makes it possible to generate a virtual representation of a real-world environment to scale, based on measurements taken in that real-world environment. The measurement process is facilitated by the advantageous use of a mixed-reality virtual reality headset combined with an interface device that the user manipulates and positions at the locations where they wish to take measurements for modeling the real-world environment.

[0009] By mixed virtual reality headset, we mean a virtual reality headset adapted to view both the real environment and the generated virtual representation.

[0010] Using such a helmet for the generation process according to the invention offers the advantage that, during measurements, the user can see where they are placing the interface device since they can see through the helmet. Furthermore, this type of helmet operates autonomously in that it does not require the installation of sensors to determine the position of the user wearing the helmet in real space or the position of the interface device. Thus, putting on the helmet for measurements is simple.

[0011] The invention simplifies the acquisition of real-world data for modeling the real environment by using readily available, user-friendly equipment at a reasonable price, thus making it accessible to a wider range of users. In particular, the generation method according to the invention does not require expertise in modeling software or measurement techniques. It also reduces the risk of errors in surveying the real-world environment.

[0012] According to a particular embodiment of the invention, the relative coordinates are defined with respect to a reference position of the real environment, the reference position corresponding to an initial position of the virtual reality headset associated with an origin point of a reference frame of the virtual representation.

[0013] According to another particular embodiment of the invention, when at least two points of the virtual representation are successively associated with at least two positions acquired by said interface device in the real environment, a virtual element is generated in the virtual representation. Advantageously, the size of the virtual element generated in the virtual representation is scaled with respect to the distance between the two positions acquired successively in the real environment.

[0014] According to this particular embodiment of the invention, adding a virtual element to scale relative to the real environment within the virtual representation is facilitated. Such a virtual element may be an existing element of the real environment, or a purely virtual element added by the user to the virtual representation but without an equivalent in the real environment. A virtual element is understood, for example, to be a planar element such as a wall, ceiling, door, window, trees, hedges, paths, etc., or a virtual object such as a light fixture, electrical outlet, radiators, furniture, etc., or even a geometric element such as a line, a flat, concave, or convex surface, etc.

[0015] According to another particular embodiment of the invention, a type of the virtual element is pre-selected from a library of virtual element types.

[0016] According to this particular embodiment of the invention, a list of types of virtual elements is offered to the user for selection before the user defines their size in the virtual representation.

[0017] According to another particular embodiment of the invention, when the virtual element is a plan element, the library of virtual element types includes at least one of the following types: wall, window, door, floor, ceiling, light fixture, electrical outlet, radiator, path, lawn, hedge, tree.

[0018] According to another particular embodiment of the invention, the mixed virtual reality headset displays the virtual representation to the user.

[0019] According to another particular embodiment of the invention, when the virtual representation is a 3-dimensional representation, the display of said virtual representation is carried out in superposition with the real environment viewed by the user via the mixed virtual reality headset.

[0020] According to this particular embodiment of the invention, the user can simultaneously visualize the real and virtual environments in superposition. For example, when considering modifications to the real environment (room or green space layout, construction work, etc.), the user can better understand the consequences of the proposed modifications for the real space.

[0021] The invention also relates to a mixed virtual reality headset, suitable for connection to an interface device. The headset comprises: a position detector of the interface device in a real environment in which said headset is placed, the detector acquiring relative coordinates of a position of said interface device in the real environment, a processor configured to determine, following a reception of said user interaction signal from said interface device, based on the relative coordinates a corresponding point in a virtual representation of at least 2 dimensions of said real environment, a generator of said virtual representation from at least the point associated with the relative coordinates of the acquired position of said interface device.

[0022] The invention also relates to an interface device suitable for connection to a mixed virtual reality headset and comprising: a transmitter exchanging with a locator of the interface device in a real environment in which said headset is placed, the locator acquiring relative coordinates of a position of said interface device in the real environment, a user interaction detector on said interface device, and a transmitter of a user interaction signal following said detected user interaction, the interaction signal being configured to trigger the generation of a virtual representation of at least 2 dimensions of a real environment from at least the point associated with the relative coordinates of the acquired position of said interface device.

[0023] Correspondingly, the invention also relates to a system for generating a virtual representation in at least 2 dimensions of a real environment, comprising a mixed virtual reality headset as described above and at least one interface device as described above.

[0024] The invention also relates to a computer program comprising instructions for implementing a method of generating a virtual representation of a real environment according to any of the particular embodiments described above, when said program is executed by a processor. Such a program may use any programming language. It may be downloaded from a communication network and / or stored on a computer-readable medium. This program may use any programming language and may be in the form of source code, object code, or code intermediate between source and object code, such as in a partially compiled form, or in any other desirable form.

[0025] In yet another respect, a computer-readable storage medium or information carrier is proposed, which includes instructions for a computer program as mentioned above. The storage media mentioned above can be any entity or device capable of storing the program. For example, the medium may include a storage means, such as Read-Only Memory (ROM), for example a CD-ROM or a microelectronic circuit ROM, flash memory mounted on a removable storage medium, such as a USB flash drive, or magnetic mass storage such as a Hard-Disk Drive (HDD) or Solid-State Drive (SSD), or a combination of memory operating according to one or more data recording technologies.On the other hand, recording media can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. In particular, the proposed computer program can be downloaded from a network such as the Internet.

[0026] Alternatively, the recording media may correspond to an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the process in question.

[0027] The process of generating a virtual representation of a real environment mentioned above can therefore be implemented in various ways, including in hardware or software form, or be implemented as a combination of hardware and software elements. 4. List of figures

[0028] Other features and advantages of the invention will become more apparent upon reading the following description of a particular embodiment, given by way of simple illustrative and non-limiting example, and the accompanying drawings, among which: [ FIG. 1 ] There figure 1 illustrates an implementation environment for the invention according to a particular embodiment of the invention, [ FIG. 2 ] There figure 2 illustrates steps in a process for generating a virtual representation of at least 2 dimensions of a real environment, according to a particular embodiment of the invention, [ FIG. 3 ] There figure 3 illustrates an example of taking measurements in the real environment, [ FIG. 4A ] There figure 4A illustrates an example of a 3D virtual representation of the real environment shown in figure 1 during the execution of the process of generating a virtual representation, [ FIG. 4B ] There figure 4B illustrates an example of a 3D virtual representation of the real environment shown in figure 1 during the execution of the process of generating a virtual representation, [ FIG. 4C ] There figure 4C illustrates an example of a 3D virtual representation of the real environment shown in figure 1 during the execution of the process of generating a virtual representation, [ FIG. 4D ] There figure 4D illustrates an example of a 3D virtual representation of the real environment shown in figure 1 during the execution of the process of generating a virtual representation, [ FIG. 5 ] There figure 5 illustrates an example of a mixed virtual reality headset and associated interface devices, [ FIG. 6 ] There figure 6 illustrates in a simplified way an example of the architecture of a mixed virtual reality headset according to a particular embodiment of the invention, [ FIG. 7 ] There figure 7 illustrates in a simplified manner an example of the architecture of an interface device according to a particular embodiment of the invention, [ FIG. 8A ] There figure 8A illustrates another example of a real-world environment to model, [ FIG. 8B ] There figure 8B illustrates an example of a virtual representation being generated of the real environment shown in figure 8A the virtual representation being displayed superimposed on the real environment. 5. Description of an embodiment of the invention

[0029] There figure 1 illustrates an example of implementing the invention according to a particular embodiment of the invention. figure 1 represents a real environment (10), here a room, to be modeled. The real environment is simplified here and includes, for example, a room comprising 4 walls (14, 15, 16, 17) and a door (18).

[0030] A user (11) is placed in the room and equipped with a mixed virtual reality headset (12) adapted to implement the method of generating a virtual representation according to a particular embodiment of the invention which will be described later in relation to the figure 2 .

[0031] An example of a mixed virtual reality headset is illustrated in figure 5 This is, for example, an Oculus Quest headset. The mixed virtual reality headset (12) is associated with two interface devices (13), also called controllers, intended to be held by a headset user and which allow the user to interact in the virtual environment projected by the headset (12).

[0032] Such a virtual reality headset is a standalone headset in that it does not require an additional terminal to operate. This type of headset is equipped with sensors, such as cameras, to continuously detect the movements of the two controllers attached to it.

[0033] We are now describing in reference to the figure 2 steps in the process of generating a virtual representation of the environment illustrated in figure 1 , according to a particular embodiment of the invention. These steps are described in relation to the figure 3 which illustrates the different measurement points in the real environment and the figures 4A-4D which illustrate the virtual representation of the real environment at different stages of modeling.

[0034] During step E20, using the virtual reality headset (12) and a controller (13), the user launches a real-world environment modeling application previously installed in the headset's memory. Such an application includes, but is not limited to: menus allowing the user to enter and manipulate objects in a virtual environment, menus for creating virtual objects such as a library of virtual elements to add to the virtual environment, these virtual elements may have default parameters, including size, which can be modified manually or by taking measurements in the real environment as will be described later, menus for modifying virtual elements (color, shape, texture).

[0035] When the application starts, a virtual representation is initialized by associating an original coordinate system position (point O on the figure 4A ) in the virtual representation at the user's initial place in the room. All coordinates determined by the virtual reality headset (headset coordinates when the user moves, controller coordinates when the user moves their hands and / or moves) will be defined relative to this initial reference position.

[0036] The virtual reality headset (12) continuously determines, or at regular time intervals, the position coordinates of the headset itself and of the controllers (13).

[0037] The user will then proceed to take measurements of the room in order to model it.

[0038] Regarding the room height, it can be preset by default and adjusted by the user, or entered manually. The user can also take a measurement from the floor and another from the ceiling to define the ceiling height.

[0039] The application then proceeds to generate the virtual representation by modeling the elements specified by the user. To do this, during step E21, the user selects an element to be modeled in the virtual representation from a library of plan elements. For example, in this case, it is a wall.

[0040] Depending on the application, the library of virtual element types to add to the representation may include elements such as wall, window, door, floor, ceiling, light fixture, electrical outlet, radiator, path, lawn, hedge, tree....

[0041] During step E22, the user moves to the wall (14) they wish to model and takes a measurement. To do this, they place one of the controllers on an edge of the wall (14) to be modeled, for example at point A on the figure 3 and interacts with the controller (13). During step E22, the headset then receives a control signal from the controller (13).

[0042] During step E23, following the reception of the control signal from the controller (13), the virtual reality headset determines, based on the relative coordinates of the controller's position in the real environment at the time the user interacted, a corresponding point in the virtual representation (here point A' on the figure 4A Optionally, the virtual reality headset associates the point thus determined with the coordinates according to which this point was determined. In other words, during step E23, the headset memorizes the relative coordinates of the controller, and possibly its orientation in space, which it continuously determines at the moment the headset receives the command signal.

[0043] During step E24, it is checked whether additional measurements are needed to model the selected plan element. The number of measurements depends on the type of plan element being modeled. For example, for a wall with a known height, two measurements (one at each end of the wall) are sufficient. For a door, it might be useful to take measurements at all four corners of the door, or to take only two measurements and define a default height, or to take a single measurement defining the door's location and define a default height and width.

[0044] For example, if it is a door to be modeled in the virtual representation, and such a door does not exist in the real environment, taking a measurement to define the location may be sufficient.

[0045] If further measurements are needed for the selected plan element, the process proceeds to step E22, and the user moves to another point in the part to take another measurement of the element to be modeled. For example, they go to point B, as shown in figure 3 Steps E22 and E23 are iterated and point B' of the virtual representation is illustrated in figure 4B is associated with position B of the real environment.

[0046] When the number of measurements is sufficient to model the plan element, the process proceeds to step E25, during which the selected plan element is generated in the virtual representation. For example, in the case of the wall (14) to be modeled, a virtual wall (43) is modeled, as illustrated in figure 4B .

[0047] Alternatively, the generation of the plan element in the virtual representation can be triggered at step E25 by the user, for example, by a command selected from a menu. In this alternative, it is no longer determined at step E24 whether the number of measurements is sufficient. The process determines the dimensions of the plan element to be generated based on the number of successive positions acquired and, if necessary, based on default dimensions for the element when the number of measurements is insufficient.

[0048] Advantageously, according to the invention, the size of the virtual element generated in the virtual representation is scaled with respect to the distance between the positions acquired in the real environment. Thus, the virtual representation is a faithful representation of the real environment and allows the generation of a 2D or 3D plan of the real environment in which the dimensions of each element relative to the others are respected.

[0049] During step E26, the headset updates the virtual representation of the real environment being modeled by memorizing the newly generated plan element. The application thus builds the virtual representation of the real environment as measurements are taken.

[0050] The virtual representation being modeled can be displayed to the user via the headset simultaneously with the visualization of the real environment through the headset.

[0051] In this way the user sees the progressive construction of the virtual representation as measurements are taken and elements are added to the representation.

[0052] Furthermore, the user can thus adapt the location or size of the virtual elements generated in the virtual representation. For example, they can make a virtual opening coincide with a real opening by selecting the object via an action on the controller and moving their arm to the desired location.

[0053] The virtual representation can be displayed at actual size to the user. In other words, the virtual representation is displayed superimposed on the visualization of the real environment. An example of such a visualization is illustrated in figure 8B .

[0054] There figure 8B illustrates an example of a virtual representation (81) being generated of a real environment (80) illustrated in figure 8A The virtual representation (81) is displayed superimposed on the real environment (80). The examples illustrated in figure 8A et 8B are shown from the point of view of an observer of the real scene and not from the point of view of the user wearing the helmet, for better understanding.

[0055] In this example, the virtual representation (81) is displayed in transparency so that the user (12) can see both the real environment and the virtual representation that he is building using the mixed virtual reality headset.

[0056] In another example, the user can zoom out on the virtual representation to display it in a smaller size, for example, to view the virtual representation in its entirety. An example of such a display is illustrated in figure 4C described later.

[0057] During step E27, it is checked whether the user has exited the virtual representation generation mode. If not, the headset waits for a new selection of an element to add to the virtual representation.

[0058] Otherwise, the generation process ends.

[0059] The virtual representation generated by the virtual reality headset can be stored in a file format for later use via playback on the VR headset or any other suitable display device. The process of generating a virtual representation is described here in the case of a 3D virtual representation. However, the process can be applied to the 2D modeling of a real-world environment to create an architectural plan, for example. Alternatively, a 2D representation can be obtained by converting the generated 3D representation.

[0060] According to other examples, the generated virtual representation can then be used in a virtual reality environment. For example, the generated virtual representation can allow a game to be adapted to the user's real environment, or it can be rendered to the user in virtual reality to allow the headset user to move around in the displayed virtual environment and possibly place virtual objects (furniture, etc.), modify the appearance of the elements of the virtual representation (change the colors, floors, ceilings, etc.).

[0061] There figure 4C This illustrates an example of the virtual representation generated by the headset when the user added the three other walls of the room (15, 16, and 17) and the door (18). figure 3 illustrates examples of positions (B, C, D, E) at which measurements could be taken using the headset and a controller and the figure 4C illustrates the points (B', C', D', E') of the virtual representation associated with these positions and the corresponding generated plane elements (15, 16, 17, 18) in the virtual representation.

[0062] There figure 4D illustrates an example of a virtual representation of the real environment shown in figure 3 in which the user has added a purely virtual plan element (48), i.e. not present in the real environment. For example, this is a window.

[0063] To do this, during the generation process, in step E21, the user selected a plan element of the window type, and in step E22, he positioned a controller at position F1 of the real environment and interacted with the controller.

[0064] Following receipt of the control signal, during step E23, the headset determined point F1' in the virtual representation based on the relative coordinates of position F1. And, in a particular embodiment of step E23, the headset further associated the relative coordinates of position F1 with point F1'.

[0065] Then, again, during step E22, the user positioned a controller at position F2 of the real environment and interacted with the controller and following the reception of the command signal, during step E23, the headset determined, based on the relative coordinates of position F2, the point F2' in the virtual representation, and then possibly associated them.

[0066] During step E25, a virtual window was added in the virtual representation between the two points F1' and F2'.

[0067] The process of generating a virtual representation described above allows the user to have a faithful 2D or 3D plan, quickly and easily produced without the need for special skills.

[0068] There figure 6 illustrates in a simplified way an example of the architecture of a mixed virtual reality DISP headset according to a particular embodiment of the invention.

[0069] Such a headset is configured to implement the process of generating a virtual representation according to any of the embodiments described above.

[0070] In the non-limiting example illustrated on the figure 6 The DISP headset includes a memory memory (MEM), a processing unit (TU), equipped for example with a processor (PROC), and controlled by the computer program (PG) stored in the MEM. The computer program (PG) includes instructions to implement the steps of the process of generating a virtual representation as described above, when the program is executed by the processor (PROC).

[0071] At initialization, the code instructions of the computer program PG are, for example, loaded into memory before being executed by the PROC processor. The PROC processor of the processing unit UT notably implements the process of generating a virtual representation according to any one of the embodiments described in relation to the figure 2 according to the instructions of the PG computer program.

[0072] To achieve this, the DISP headset includes a CAPT detection device configured to detect controllers associated with the DISP headset. This detection device includes cameras to capture the movements of the controllers, as well as any movement of the user wearing the headset.

[0073] The DISP headset also includes a PROJ projection device that allows a virtual reality environment to be visually reproduced for the user, including the virtual representation generated by the headset via the process described above.

[0074] The DISP headset also includes a COM communication interface allowing it to receive a control signal from a controller associated with the headset.

[0075] There figure 7 illustrates in a simplified manner an example of the architecture of an INT interface device according to a particular embodiment of the invention. The INT interface device is, for example, a controller associated with the headset described in relation to the figure 6 .

[0076] In the non-limiting example illustrated on the figure 7The interface device INT includes a processing unit UT', equipped, for example, with a processor PROC', and driven by the computer program PG' stored in memory MEM'. The processor PROC' and the memory MEM' are configured to detect user interaction on the interface device via a user interface (UI) of the interface device, and to send a user interaction signal to the associated mixed reality headset following the detected user interaction. To this end, the interface device INT includes a communication interface COM' to transmit the user interaction signal or control signal to the headset.

Claims

1. Method for generating a virtual representation in at least 2 dimensions, of a real environment, the generation being implemented by a mixed virtual reality headset intended to be worn by a user, the mixed virtual reality headset being associated with at least one interface device, the generating method comprising: - acquiring relative coordinates in the real environment, corresponding to a position of said interface device in the real environment, - following an interaction (E22) of the user with said interface device, determining (E23) depending on said relative coordinates of the position of said interface device in the real environment a corresponding point in the virtual representation, - generating (E25) said virtual representation based at least on the point associated with the relative coordinates of the acquired position of said interface device.

2. Generating method according to Claim 1, wherein said relative coordinates are defined with respect to a reference position of the real environment, said reference position corresponding to an initial position of the virtual reality headset, which initial position is associated with an origin point of a reference frame of said virtual representation.

3. Generating method according to either one of Claims 1 and 2, further comprising, when at least two points of the virtual representation are successively associated with at least two acquired positions of said interface device in the real environment, generating a virtual element in the virtual representation, the size of said virtual element generated in the virtual representation being scaled with respect to the distance between the at least two acquired positions in the real environment.

4. Generating method according to Claim 3, wherein a type of said virtual element is previously selected from a library of types of virtual element.

5. Generating method according to Claim 4, wherein, when the virtual element is a plane element, the library of types of virtual element contains at least any of the following types: wall, window, door, floor, ceiling, light, electrical socket, radiator, path, lawn, hedge, tree.

6. Generating method according to any one of Claims 1 to 5, further comprising displaying said virtual representation via the mixed virtual reality headset.

7. Generating method according to Claim 6, wherein, when the virtual representation is a 3-dimensional representation, said virtual representation is displayed in superposition with the real environment viewed by the user via the mixed virtual reality headset.

8. Mixed virtual reality headset capable of being connected to an interface device, said headset comprising: - a detector of the position of the interface device in a real environment in which said headset is placed, the detector acquiring relative coordinates of a position of said interface device in the real environment, - a processor configured to determine, following receipt of said user interaction signal from said interface device, depending on the relative coordinates, a corresponding point in a virtual representation in at least 2 dimensions of said real environment, - a generator of said virtual representation based at least on the point associated with the relative coordinates of the acquired position of said interface device.

9. Interface device capable of being connected to a mixed virtual reality headset and comprising: - a transmitter making exchanges with a locator of the interface device in a real environment in which said headset is placed, the locator acquiring relative coordinates of a position of said interface device in the real environment, - a detector of user interaction with said interface device, and - a transmitter of a user interaction signal following said detected user interaction, the interaction signal being configured to trigger generation of a virtual representation in at least 2 dimensions of a real environment based at least on the point associated with the relative coordinates of the acquired position of said interface device.

10. System for generating a virtual representation in at least 2 dimensions, of a real environment, comprising: - a mixed virtual reality headset according to Claim 8 and at least one interface device according to Claim 9.

11. Computer program comprising instructions for implementing the method for generating a virtual representation in at least 2 dimensions of a real environment according to any one of Claims 1 to 7, when said program is executed by the processor of a mixed virtual reality headset according to Claim 8.

12. Computer-readable medium comprising a computer program according to Claim 11.