Improved avatar creation

Abstract

The invention relates to a method for creating an avatar of a user, comprising: - obtaining (S1) at least image data of the user and processing (S2) said data for determining morphological parameters specific to the user, and - applying (S3) the user-specific morphological parameters to a parametric avatar model in order to obtain a personalized parametric avatar of the user.

Description

Advanced Avatar Creation

[0001] This description concerns the creation of an avatar of a natural person, according to attributes specific to this person.

[0002] An avatar can be used to virtually represent a physical person in computer applications, for example messaging, or in online games, or more generally in a virtual environment or "metaverse".

[0003] Current techniques for creating an avatar of a user of such applications use: - either a so-called "dense" reconstruction of the user's appearance via one or more cameras (for example, "RGB" type color cameras) and / or an infrared, depth camera, using a physical device (A), - or the creation of a parametric avatar, which allows the user to configure the appearance of their avatar using predefined models by clicking on dedicated icons (for example, "slider" type icons) to choose their hairstyle, hair color, height, skin color, eye color, etc., using a computer application (B).

[0004] However, dense reconstruction results in a non-customizable avatar and requires a very large volume of data. For example, the color of hair, eyes, skin, etc., cannot be modified. Furthermore, an overly realistic avatar is not always well received by users, who find their virtual representation shocking, because, although very close, it is not absolutely perfect (according to the theory of the "uncanny valley" initiated in particular by Masahiro Mori).

[0005] The creation of a parametric avatar, on the other hand, is not realistic because it does not correspond to the user's actual measurements and cannot faithfully reproduce their features, but it is light in resources and therefore much more suitable for virtual telepresence applications. Parametric avatars do not offer to take into account the user's real features. The user is offered predefined features, typical of cartoon characters, to which the user adds personalized eye colors, hair, glasses, beard, etc. The result remains "cartoonized" (in the sense of an appearance close to a cartoon character), despite some attributes chosen by the user.

[0006] This description improves this situation.

[0007] To this end, it proposes a method for creating an avatar of a user, comprising: - Obtaining at least image data of the user and processing said data to determine morphological parameters specific to the user, and - Applying the morphological parameters specific to the user to a parametric avatar model, to obtain a personalized parametric avatar of the user.

[0008] Thus, by implementing the above method, it is possible to obtain a parametric avatar, therefore modifiable by the user according to his own choices or needs, but nevertheless this initial avatar is realistic because personalized thanks to the user's image data. Indeed, the value of at least one parameter of the avatar is personalized according to at least one morphological parameter among the morphological parameters specific to the user determined according to the user's image data obtained by the avatar creation method.

[0009] The aforementioned morphological parameters of the user may typically be the shape and arrangement in the face of elements such as the nose, eyes, mouth, etc., or the dimensions (relative or absolute) of the segments of the limbs such as the forearms, legs, etc. or the color of skin, hair, possibly beard, etc.

[0010] They can be identified in image data by implementing artificial intelligence and / or colorimetry and photogrammetry applications.

[0011] Thus, the method according to the invention makes it possible to create a personalized avatar more quickly because it does not require the complex and fine parameterization implemented for the creation of an avatar by volumetric capture of the user. However, unlike the creation of a parametric avatar for which only aesthetic parameters can be selected from a catalog, the method according to the invention proposes to modify morphological parameters of the avatar not by selecting values ​​from a catalog but by using morphological parameters of the user, which makes it possible to create a personalized parametric avatar morphologically conforming to the user. The morphological conformity to the user of the parametric avatar created by the method according to the invention is at least partial (size and / or waist circumference and / or hand size etc.).

[0012] An advantage of such a parametric avatar personalized morphologically conforming to the user is that it allows the user to try products in the virtual world before a purchase, or even their manufacture (furniture, car, clothing, house, apartment, etc.). Another advantage of a parametric avatar personalized morphologically conforming to the user is that it allows interaction in augmented reality with real objects while limiting interaction errors since the avatar conforms to the morphology of the user.

[0013] A technical effect of this personalized configuration of the avatar based on the user's morphological parameters is that the parametric nature of the avatar allows the avatar to be subsequently modified to create one or more different versions.

[0014] For example, image data may be acquired by a plurality of cameras to stereoscopically construct a 3D model of the user.

[0015] Additionally or alternatively, the image data may be further acquired by at least one infrared camera to acquire at least image depth data of the user and construct a 3D model of the user.

[0016] In one embodiment, the method may comprise:- Determining from said image data at least relative measurements of the user, by photogrammetry, and evaluating morphological parameters characteristic of the user.

[0017] These relative measurements can be, for example, the distance between the eyes relative to the width of the face, or the height of the nose relative to the total height of the face, or the width of the nose relative to the width of the face, or the length of the forearm relative to the total length of the arm, etc.

[0018] These measurements are relative, but it is possible to measure and configure absolute dimensions. For example, the image data may also include data from a target of known size (e.g., in the field of images taken of the user) to determine absolute measurements of the user.

[0019] The method may further comprise:- Determining by colorimetry and from said image data, at least the skin and / or hair colors of the user.

[0020] Thus, the realistic avatar, initially constructed, can present a faithful color of the hair and / or skin and / or a possible beard, etc., of the user.

[0021] In addition to or as a variant of the use of colorimetry and / or photogrammetry applications, the method may provide for implementing artificial intelligence to identify, from said image data, distinct parts of the user's body, and to configure said distinct parts of the body.

[0022] For example, the aforementioned artificial intelligence can determine a color, a length, etc. of these different parts of the body by associating them with an identifier (by "labeling" processing of the different parts of the body). This identifier is then recognized by an application for creating a parametric avatar which receives this data.

[0023] In one embodiment, an application for creating a parametric avatar can be programmed to accept as input parameters at least:- shapes and dimensions of elements of the user's face,- colors at least of the user's skin and / or hair,- dimensions of segments of the user's limbs.

[0024] Examples of such a parametric avatar creation application are given below. The application “Make Human Project” is cited here simply as a non-limiting example.

[0025] By using this parameterized data, specific to the user, the parametric avatar created is realistic. However, since it is parametric, it can be modified at will by the user, for example.

[0026] Thus, in one embodiment, the method may further comprise: - modifying, at the user's request, the user's personalized parametric avatar.

[0027] For example, the user may choose to change a hair color, eye color, skin color, clothing type, or others.

[0028] In addition or as a variant, the method may further comprise, during the execution of an application in which the avatar is involved: - modifying, upon receipt of a command from the application, the user's personalized parametric avatar.

[0029] In this embodiment, for example as the application (e.g., an online game) runs, the avatar may be automatically modified by the application (e.g., by making the avatar grow and thicken as the user wins rounds of the game).

[0030] In one embodiment, in addition to the image data, voice data of the user is acquired to accompany a physical appearance of the user's personalized avatar, a realistic voice expression of the user.

[0031] Thus, in such an implementation, the avatar created is faithful both in physical appearance and in the voice given to the avatar. However, since the avatar is parametric, the user can choose to give it a modified voice (deeper for example, or other).

[0032] According to another aspect, there is provided a computer program comprising instructions for implementing the method as defined above when these instructions are executed by a processor of a processing circuit. According to another aspect, there is provided a non-transitory recording medium, readable by a computer, on which such a program is recorded.

[0033] According to another aspect, there is provided a computing device comprising a processing circuit configured to implement the method presented above.

[0034] Other features, details and advantages will become apparent upon reading the detailed description below, and upon analyzing the attached drawings, in which:

[0035] Illustrates the estimation of a parametric avatar as realistic as possible from a dense photorealistic model of a user,

[0036] Illustrates a possible embodiment consisting of the creation of multiple personalized avatars, from the most realistic parametric avatar possible,

[0037] Illustrates an example of a device for implementing the above method.

[0038] The subject of this description allows creating a user avatar by combining a parametric avatar creation application with a dense capture of the user's actual appearance.

[0039] With reference to, in a first step S1, the user US positions himself in front of a device A for creating his dense avatar, for example one or more CAM cameras or even an infrared depth camera.

[0040] The aforementioned CAM cameras can form a so-called "rig" of cameras (with around fifty cameras for example) enveloping the user in a half-cylinder or a cylinder to acquire a 3D model of the user's complete body, by stereoscopy. The aforementioned infrared depth camera can provide, as an alternative or in addition, 3D information by projection for example of an infrared grid with detection of the round-trip time of the rays at each point of the user's body. An infrared camera model of this type is known under the name "Kinect" ®.

[0041] In a second step S2, a computer processing circuit (of the type illustrated in the example) is connected to the device A to then create a “photorealistic avatar”: of the same size and having the same proportions as the user. Here, “photorealistic avatar” means a particularly realistic 3D model of the user, which may consist at least of a set of data stored in memory, this data determining: - The color of the skin, eyes, hair, any beard, eyebrows, etc. - The shape of the face: shape and height of the nose relative to the height of the face, shape and distance of the eyes relative to the width of the face, arrangement and shape of the mouth, arrangement and shape of the ears, shape of the chin, etc. - The shape of the limbs: relative length of the forearms relative to the arms, of the trunk, of the legs, etc. - The posture: curvature of the back, advancement of the pelvis, etc.

[0042] Of course, the above data examples are not exhaustive. Furthermore, the dimensions of the body parts can be given relative to another part of the body or be given in absolute terms. In the second case, a target of known dimensions can be placed with the user in the field of the CAM camera(s) for the implementation of the first image acquisition step S1.

[0043] For example, at this step S2, an artificial intelligence AI can intervene to:- Identify in the acquired images the colors of skin, hair etc., as well as the dimensions of parts of the body, and others,- Parameterize this identified information in the form of said stored data,- And provide at the following step S3 this PARAM data to a parametric avatar model.

[0044] Indeed, step S3 consists of parameterizing in the most realistic way possible the creation of the user's avatar with a parametric application B according to the photorealistic avatar obtained in step S2. At this step S3, the data thus parameterized (for example by the aforementioned artificial intelligence) can then be provided to an application B for creating a parametric avatar. This application B has the advantage of admitting a large number of parameters (other than simple skin or hair color, or the simple existence of a beard), so that the parametric avatar created remains faithful to the morphology of the user.

[0045] For example, a B application for creating a generic avatar with a parametric approach could be "Make Human Project": http: / / www.makehumancommunity.org / (open access to the source code), or MB-Lab https: / / mblab.dev / (plugin for the "3D Blender" modeler). Such application examples offer the advantage of being flexible in programming and in particular of admitting a large number of morphological parameters of the user.

[0046] At step S3, application B receives the user's parametric morphological data (limb length, exact face shapes, skin and hair colors, etc.) and constructs a parametric avatar respecting all the provided morphological data.

[0047] Thus, the user obtains a parametric avatar as realistic as application B allows by relying on a first, dense avatar, obtained using the acquisition device A. This avatar is in particular the user's measurements, relative or absolute, which is appreciable in particular for mixed reality telepresence systems.

[0048] Referring now to the, the user can then, in step S4, modify his realistic parametric avatar according to his own wishes. He can dress it with virtual clothes, create different avatars according to his needs and choices of representation in a metaverse. He can modify his appearance, erase his physical defects, change the color of his eyes, his hair, his skin, or others. Several possible modifications of his realistic avatar can be recorded so that the user then has a set of avatars that he can choose to use according to his context in the metaverse (for example in costume for a metaverse with clients, in casual clothes with friends, or disguised and physically transformed to celebrate Halloween, or others).For example: - a first avatar AVAT1 may be suitable for a first application APPLI1, for example virtual videoconferencing in the context of a job interview, - a second avatar AVAT2 may be suitable for a second application APPLI2, for example online gaming, - a third avatar AVAT3 may be suitable for a third application APPLI3, for example discussion on an informal social network, - etc.

[0049] In addition to physical appearance, a change in the individual's posture may be expected, or a change in the way the user moves their body (e.g. hand movements, gait, etc.).

[0050] Furthermore, advantageously, the current application APPLIi (i=1, 2, or 3) can, as it progresses, automatically modify the user's avatar AVATi. For example, the appearance of the user's avatar can be increasingly realistic during a virtual video conference in the context of a job interview, if the application detects that the mood of the participants characterizes a climate of trust between the participants during the interview. In the context of a game application, the appearance of the user's avatar can be increasingly "muscular" as the user wins rounds of the game, for example. This step of modifying the avatar according to the progress of the application is illustrated by the arrow S5 on the.

[0051] Referring now to the, a device DIS for implementing the above method may comprise a processing circuit comprising for example:- An input interface IN for receiving for example image data acquired by one or more cameras A,- A memory MEM for storing the instructions of a computer program of the type introduced above, and possibly storing at least temporarily the acquired image data,- A processor PROC capable of cooperating with the memory MEM to read and execute the instructions of such a program in order to control a parametric avatar creation application by providing it with relevant parameters, derived from the image data, and- An output OUT for delivering the data of a realistic parametric avatar, in the example illustrated.

[0052] The DIS device may be hosted on a server, to provide, via a communication network, an avatar version to a remote terminal available to the user. Optionally, the user may choose online a version of the avatar according to his choice and needs. This choice may be received (for example "change eye color", etc.) for example via the input interface IN of the DIS device.

[0053] Alternatively to using an output interface, the avatar can be created locally, with simple storage of the avatar data in the MEM memory, for example in a terminal available to the user.

[0054] Thus, the subject of this description allows the creation of customizable parametric avatars from the user's actual appearance. The use of an avatar can be extremely broad, and increasingly corresponds to an essential element of non-verbal communication between remote users. The subject of this description considerably reduces the avatar creation time, makes it possible to obtain a parametric avatar with the user's measurements (or possibly transformed according to the user's choice), to create multiple avatars according to the user's needs / tastes / choices / preferences.

[0055] However, in addition to parameterizing the user's physical appearance data, it is possible to further parameterize the user's voice, as well as the user's expression and speech. For example, in addition to the image data acquired in step S1, it is possible to have a text read to the user and to record the user's voice using one or more microphones. Using tonal detection and formants in the user's voice, it is then possible to further provide parameterized data of the user's voice (average tonal frequency, respective average intensities of the formants, and / or others) to a parametric, sound avatar creation application in step S2. In step S4, the user can slightly transform his voice, for example by making it lower in tonal frequency, or the current application APPLIi can modify the voice by making it lower, for example, as the user wins the rounds of a game.More generally, in addition to transforming the voice, it is possible to make the avatar speak in another language, or to modify / correct its oral expression with words more appropriate to a context, for example.

[0056] More generally, the present description is not limited to the embodiments described above as examples. It extends to other variants.

[0057] For example, an artificial intelligence has been described above to determine the different parts of the body and configure them. However, a photogrammetry application, combined with a colorimetry application, can already determine the user's morphological parameters, in sufficient number to inform a parametric avatar creation application capable of generating a realistic avatar, and therefore without necessarily resorting to artificial intelligence. Of course, these applications can be programmed to be compatible with the parameters expected by the parametric avatar creation model.

Claims

Method for creating a user avatar, comprising: - Obtaining (S1) at least some image data of the user and processing (S2) said data to determine morphological parameters specific to the user, and - Applying (S3) the morphological parameters specific to the user to a parametric avatar model, to obtain a personalized parametric avatar of the user. The method of claim 1, wherein image data are acquired by a plurality of cameras (CAM-A) to construct by stereoscopy a 3D model of the user. A method according to any one of the preceding claims, wherein image data is acquired by at least one infrared camera to acquire at least image depth data of the user and construct a 3D model of the user. A method according to any one of the preceding claims, comprising: - Determining from said image data at least some relative measurements of the user, by photogrammetry, and evaluating morphological parameters characteristic of the user. A method according to claim 4, wherein the image data further comprise data from a target of known dimensions for determining absolute measurements of the user. A method according to any one of the preceding claims, comprising: - Determining by colorimetry and from said image data, at least some skin and / or hair colors of the user. A method according to any one of the preceding claims, comprising: - Implementing artificial intelligence (AI) to identify, from said image data, distinct parts of the user's body, and to parameterize said distinct parts of the body. A method according to any one of the preceding claims, wherein a parametric avatar creation application is programmed to admit as input parameters at least: - shapes and dimensions of elements of the user's face, - at least skin and / or hair colors of the user, - dimensions of segments of the user's limbs. A method according to any one of the preceding claims, further comprising: - modifying (S4), at the user's request, the user's personalized parametric avatar. A method according to any one of the preceding claims, further comprising, during the execution of an application in which the user's avatar is involved: - modifying (S5), upon receipt of a command from the application, the user's personalized parametric avatar. A method according to one of the preceding claims, wherein, in addition to image data, user voice data is acquired to accompany a physical appearance of the user's personalized avatar with a realistic voice expression of the user. Computer program comprising instructions for implementing the method according to one of the preceding claims, when said instructions are executed by a processor (PROC) of a processing circuit. Computer device (DIS) comprising a processing circuit configured to implement the process according to any one of claims 1 to 11.

Images