Virtual character with parametric textures in scene description

Abstract

Parametric textures are linear combinations of textures that allow for editing of the texture. In one implementation, we describe how to encode an avatar with parametric textures and how to use parametric textures. In one example, we add a new list "textures" to the MPEG_node_avatar extension that defines a set of texture definitions for the current avatar. Each entry in the "textures" list (AvatarTexture[i]) defines how a texture can be used for the avatar. Specifically, each AvatarTexture entry can indicate the texture type, an illustrative description of the texture, the primary purpose of the texture, and the material. When the texture type is "targets," "targets" also indicates the attributes and associated weights for each texture target. In addition, for each texture target, we can also provide an illustrative description, the primary purpose of the texture target, and the minimum and maximum values for the texture target weights.

Description

[0001] Cross-reference to related applications This application claims priority to European Application No. 24305389.9, filed on March 15, 2024, which is incorporated herein by reference in its entirety. Technical Field

[0002] This embodiment generally involves parametric texture encoding and decoding of 3D virtual character models in scene description. Background Technology

[0003] Extended Reality (XR) is a technology for enabling interactive experiences where real-world environments and / or video content are augmented by virtual content, which can be defined across multiple sensory modalities, including visual, auditory, and tactile. During application runtime, the virtual content (e.g., 3D content or audio / video files) is rendered in real-time in a manner consistent with the user's context (environment, viewpoint, device, etc.). Scene graphs (such as the / glTF (Graphics Language Transfer Format) scene graph proposed by Khronos, and extensions such as the MPEG scene description format or Apple / USDZ) are a viable way to represent the content to be rendered. Scene Figure 1 The scene description framework combines a declarative description of the scene structure that associates real-world objects with virtual objects, with a binary representation of the virtual content. This framework ensures that timed media and corresponding virtual content are available at any point during application rendering. The scene description can also carry data at the scene level describing how users can interact with scene objects at runtime to achieve an immersive XR experience. Summary of the Invention

[0004] According to one embodiment, a method is presented, comprising: obtaining a representation type for representing a virtual avatar from a description of a 3D scene; obtaining a texture of the virtual avatar from the description; and reconstructing the virtual avatar based on the representation type and the texture.

[0005] According to another embodiment, an apparatus is presented including one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to: obtain a representation type for representing a virtual avatar from a description of a 3D scene; obtain a texture of the virtual avatar from the description; and reconstruct the virtual avatar based on the representation type and the texture.

[0006] According to another embodiment, a method is presented, comprising: indicating a representation type for representing a virtual avatar in a description of a 3D scene; and indicating a texture of the virtual avatar in the description.

[0007] According to another embodiment, an apparatus is presented including one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to: indicate a representation type for representing a virtual avatar in a description of a 3D scene; and indicate a texture of the virtual avatar in the description.

[0008] One or more embodiments also provide a computer program including instructions that, when executed by one or more processors, cause one or more processors to perform the method described according to any embodiment described herein. One or more of these embodiments also provide a computer-readable storage medium storing instructions for processing a scenario description according to the method described herein.

[0009] One or more embodiments also provide a computer-readable storage medium storing a scene description generated according to the method described above. One or more embodiments also provide a method and apparatus for transmitting or receiving a scene description generated according to the method described herein. Attached Figure Description

[0010] Figure 1 An example architecture for an XR processing engine is shown.

[0011] Figure 2 A syntactic example of a data stream encoding an extended reality scene description is shown.

[0012] Figure 3 An example diagram illustrating an extended reality scene is shown.

[0013] Figure 4 An example of an extended real-world scene description including behavioral data is shown.

[0014] Figure 5 The illustration shows a method for parsing an MPEG node avatar according to one embodiment.

[0015] Figure 6 The illustration shows a method for parsing an AvatarTexture according to one embodiment.

[0016] Figure 7 The illustration shows a method for resolving an AvatarTextureTarget according to one embodiment. Detailed Implementation

[0017] Various XR applications can be applied to different contexts and in real or virtual environments. For example, in industrial XR applications, when a camera mounted on a head-mounted display detects a reference object (component B of an engine) in a real environment, a virtual 3D content item (e.g., component A of the engine) is displayed. This 3D content item is positioned in the real world with a position and scaling defined relative to the detected reference object.

[0018] For example, in XR applications for interior design, 3D models of furniture are displayed when a given image from a catalog is detected in the input camera view. This 3D content is positioned in the real world with a location and scale defined relative to the detected reference image. In another application, audio files might start playing when a user enters an area near a church (whether a real church or a virtually rendered church in an extended reality environment). In yet another example, an advertising jingle file could play when a user sees a given can of soda in a real environment. In outdoor gaming applications, various virtual characters can appear based on the semantics of the scenery observed by the user. For example, bird characters are suited to trees, so if the XR device's sensors detect a real object described by the semantic tag "tree," birds flying around trees can be added. In companion applications implemented with smart glasses, car noise can be played in the user's headphones when a car is detected within the user's camera's field of view to warn them of potential danger. Furthermore, sound can be spatialized so that it comes from the direction of the detected car.

[0019] XR applications can also enhance video content rather than the real environment. The video is displayed on a rendering device, and when a timed event is detected in the video, virtual objects described in a node tree are overlaid. In this context, the node tree only includes descriptions of the virtual objects.

[0020] Figure 1 An example architecture of an XR processing engine 130 is shown, which can be configured to implement the methods described herein. Figure 1 The devices in the architecture are linked to other devices via bus 131 and / or via I / O interface 136.

[0021] Device 130 includes the following components linked together via data and address bus 131: - Microprocessor 132 (or CPU), which is, for example, a DSP (or digital signal processor). -ROM (or read-only memory) 133; -RAM (or random access memory) 134; - Storage interface 135; -I / O interface 136, used to receive data to be transmitted from the application; and -Power supply (in) Figure 1 (not shown in the text), such as batteries.

[0022] According to one example, the power supply is located externally to the device. In each of the mentioned memories, the term "register" as used in the specification can correspond to a small area (a few bits) or a very large area (e.g., the entire program, or a large amount of received or decoded data). ROM 133 includes at least the program and parameters. ROM 133 can store algorithms and instructions for executing techniques according to these principles. When powered on, CPU 132 uploads the program to RAM and executes the corresponding instructions.

[0023] RAM 134 includes in the registers: the program executed by CPU 132 and uploaded after device 130 is turned on, input data in the registers, intermediate data in the registers at different states of the method, and other variables in the registers used to run the method.

[0024] Device 130 is linked, for example, via bus 131 to a set of sensors 137 and a set of rendering devices 138. Sensors 137 may be, for example, a camera, microphone, temperature sensor, inertial measurement unit, GPS, humidity sensor, IR or UV light sensor, or wind sensor. Rendering devices 138 may be, for example, a display, speaker, vibrator, heater, fan, etc.

[0025] According to the example, device 130 is configured to implement the method according to this principle and belongs to the group that includes: -mobile device; - Communication equipment; -Gaming devices; - Tablet (or tablet PC); - Laptop computers; - Still image camera; and -Camera.

[0026] In XR applications, scene descriptions are used to combine explicit and easily parsed descriptions of the scene structure with some binary representation of the media content. Figure 2 A syntactic example of a data stream encoding an extended reality scene description is shown. Figure 2An example structure 210 for an XR scene description is shown. This structure consists of a container that organizes the stream using independent syntax elements. The structure may include a header section 220, which is a set of data common to each syntax element in the stream. For example, the header section includes some metadata about the syntax element describing the properties and roles of each syntax element. The structure also includes a payload comprising syntax elements 230 and 240. Syntax element 230 includes data representing media content items described in nodes of the scene graph associated with the virtual element. Images, meshes, and other raw data may have been compressed according to a compression method. Syntax element 240 is part of the data stream payload and includes data encoded according to the principles described herein.

[0027] Figure 3 An example diagram 310 is shown, illustrating an extended reality scene description. In this example, the scene graph may include descriptions of real-world objects, such as a “horizontal plane” (which could be a table or a road), and descriptions of virtual objects 312, such as an animation of a car. The scene description is organized as an array of nodes. Nodes can be linked with child nodes to form a scene structure 311. Nodes may carry descriptions of real-world objects (e.g., semantic descriptions) or descriptions of virtual objects. Figure 3 In the example, node 301 describes a virtual camera located within the 3D volume of an XR application. Node 302 describes a virtual car and includes an index of the car's representation, such as an index in a 3D mesh array. Node 303 is a child of node 302 and includes a description of one of the car's wheels. Similarly, it includes an index of the 3D mesh for that wheel. Because the scale, position, and orientation of objects are described in the scene nodes, the same 3D mesh can be used for several objects in the 3D scene. Scene graph 310 also includes nodes describing the spatial relationships between real and virtual objects.

[0028] In time-based media streaming, the scene description itself can evolve over time to provide relevant virtual content for each sequence of the media stream. For example, for advertising purposes, a virtual bottle could be displayed on a table during a video sequence of people sitting around it. This behavior can be achieved by relying on the framework defined in the MPEG Media Scene Description document.

[0029] Currently, the MPEG-I scene description framework uses "behavioral" data to enhance the scene description that evolves over time and provides a description of how users can interact with scene objects at runtime to obtain an immersive XR experience. These behaviors are associated with predefined virtual objects on which runtime interactivity is allowed to achieve user-specific XR experiences. These behaviors also evolve over time and are updated through existing scene description update mechanisms.

[0030] Figure 4 An example of an extended reality scene description including behavioral data stored at the scene level describes how a user can interact with scene objects at runtime (described at the node level) to achieve an immersive XR experience. When an XR application is launched, media content items (such as a mesh of virtual objects visible from the camera) are loaded, rendered, and cached for display when triggered. For example, when a sensor detects a plane in the real environment, the application displays the cached media content item as described in the relevant scene node. Timing is managed by the application based on features detected in the real environment and according to the timing of the animation. Nodes in the scene graph may also not include descriptions and simply act as parent nodes to child nodes. Figure 4 The diagram illustrates the relationship between behaviors included in the scene description at the scene level and the nodes that make up the scene graph. Behavior 410 is associated with predefined virtual objects that allow runtime interactivity to enable user-specific XR experiences. Behavior 410 also evolves over time and is updated via a scene description update mechanism.

[0031] One type of behavior includes: - Trigger 420 defines the conditions that must be met to activate this behavior; - Trigger control parameters define the logical operations between the defined triggers; -Action 430, which must be performed when the trigger is activated; -Action control parameters define the execution order of related actions; - Priority numbering, which enables the selection of the highest priority behavior when several behaviors compete at the same time on the same virtual object; - Optional interrupt action, specifying how to terminate the behavior when it is no longer defined in a newly received scene update; for example, the behavior is no longer defined if the relevant object does not belong to the new scene, or if the behavior is no longer relevant to the current media (e.g., audio or video) sequence.

[0032] Behavior 410 occurs at the scene level. Triggers are linked to nodes and their child nodes. Figure 4In the example, trigger 1 is linked to nodes 1, 2, and 8. Since node 31 is a child of node 1, trigger 1 is linked to node 31. Trigger 1 is also linked to node 14 as a child of node 8. Trigger 2 is linked to node 14. In fact, the same node can be linked to several triggers. Trigger n is linked to nodes 5, 6, and 7. A behavior can include several triggers. For example, a first behavior can be activated by trigger 1 "AND" trigger 2, where "AND" is the trigger control parameter for the first behavior. A behavior can have several actions. For example, a first behavior can first execute action m and then execute action 1, where "first and then" is the action control parameter for the first behavior. For example, a second behavior can be activated by trigger n, and first execute action 1 and then execute action 2.

[0033] Parametric textures Parametric textures can be modeled as a weighted sum of texture images: (1) in and It is an image, and Is with The associated weights. Specifically, Represents the base texture, and This is typically used to slightly modify a base texture to a target texture. We also clamp the values ​​to ensure they are within the valid color range. Therefore, this parametric model allows for the creation of new materials based on this combination.

[0034] We also assume that the base texture and texture targets have the same layout (same width, height, and vector size). Using the proposed parametric textures, we can create unique combinations for objects or characters in a scene using a finite number of texture targets, with a new item costing only a few texture target weights.

[0035] If someone provides additional information, the application can use a avatar with parametric textures contained in the glTF file. For example, the application can load a glTF file with a avatar and parametric textures, and let the user define which parametric textures to use and for what purpose. For instance, if some parametric textures are created to alter the avatar's facial hair, the user must know or guess this.

[0036] One of the main purposes of avatar encoding in scene descriptions is to exchange avatar instances in heterogeneous environments. For example, people can bring avatars into a virtual store to try on clothes. Users might want to test the possibilities of using different makeup looks. Modifying an avatar's makeup with parametric textures is straightforward for any 3D rendering application. However, the application must know which texture targets are relevant to the makeup and what weights to use.

[0037] In this specification, the proposed format conforms to the glTF format and is compatible with recent MPEG-I SD work to extend glTF with MPEG-I SD extensions. However, the meaning and usage are general and can be encoded in other formats (e.g., but not limited to XML and USD).

[0038] MPEG_node_avatar extension We have added a new list “textures” to the MPEG_node_avatar extension: Table 1 - MPEG_node_avatar attributes with a new list of "textures" The "isAvatar" attribute indicates whether the node is a virtual avatar.

[0039] The "type" attribute defines the type of avatar representation. This is a URN that can be defined by a standard or application. Each URN defines a list of all expected attributes of the avatar (number of vertices, number of faces, number of joints, etc.). For example, "urn:mpeg:sd:2023:avatar" is the standard URN for the Morgan avatar version 2023.

[0040] The “mapping” attribute indicates the mapping between child nodes and semantics. Depending on the avatar type (e.g., the value of the “type” attribute), each child node can be associated with a string ID that identifies a body part. For example, for Morgan, the head ID is “full_body / upper_body / head”.

[0041] The “textures” list is a new attribute we are introducing in this document. It defines a set of texture definitions for the current avatar. Each item in the “textures” list (i.e., AvatarTexture[i]) defines how a texture can be used for the avatar (see Table 2 for more details).

[0042] AvatarTexture definition AvatarTexture The attribute is defined as follows: .

[0043] Table 2 - AvatarTexture Properties The "type" attribute defines the type of texture. In this document, we consider the "main" and "targets" types. Other types may be proposed in the future.

[0044] If "type" is "main", then AvatarTexture Define a static texture. A static texture is unchanging: for example, a texture with a tattoo. In this case, the application knows that the texture can be used on the avatar and for what purpose. In the case of a tattoo texture, the application can use the texture to display a tattoo on the avatar.

[0045] If "type" is "targets", then AvatarTexture Dynamic textures are defined based on texture targets. Dynamic textures can vary according to given parameter values. In this specific case, we consider a texture consisting of a linear combination of a base texture and a texture target. The weights of this linear combination are parameter values ​​that can be changed and impart dynamics to the texture. For example, a texture can be created based on a base texture with normal skin tone and a single texture target with a greenish tint. When the texture target weight is zero, the avatar has normal skin tone and appears healthy. As the texture target weight increases, the avatar's skin tone begins to turn greenish, and the avatar appears more poisoned. Applications can use this dynamic texture to give the avatar a poisoned appearance.

[0046] The "description" attribute describes the texture and its intended use. This is purely illustrative; for example, the "description" attribute can be used in a user interface to help users understand what the texture is intended for. There are no expected rules for the value of this attribute; for example, it can be in any language. Applications can use the "description" attribute to help users understand what the texture is used for in a more understandable way than the "purpose" attribute. If the "description" attribute does not exist, a default "description" attribute can be generated based on the "purpose" value.

[0047] The "purpose" attribute contains a string identifying the purpose of the texture. It must conform to a predefined value by the application or standard. This document specifies a string type, but an enumeration type can also be used. The value of this attribute must not change across applications: for a given value, each application must interpret it in the same way. For example, the standard might define the string "avatar:health:status" to indicate that the texture is used to represent the health status of an avatar. For instance, if an application wants to display a poisoned avatar, it knows it can use such a texture for this purpose.

[0048] "material" is the index of an existing material in the "materials" list at the root of the glTF file.

[0049] "path" is a marker in the material. TextureInfo The string representing the attribute. This is the JSON path starting from the material's JSON node. If "type" is "targets", then the target... TextureInfo The "MPEG_texture_targets" extension is required.

[0050] The “targets” list is defined by “material” and “path”. TextureInfo The attributes of each texture target. The size of "targets" is equal to the value indicated by "material" and "path". TextureInfo The number of texture targets in the list. Each item in “targets” (AvatarTextureTarget[i]) adds a semantic definition to the corresponding texture target (see Table 3). For example, the first item in “targets” is an attribute of the first texture target, the second item in “targets” is an attribute of the second texture target, and so on.

[0051] The “weights” attribute defines the default weights for each texture target of this avatar texture. The length of this array must be the same as the number of items in the “targets” list. If present, these weights must be used instead of the weights within the “weights” attribute of “MPEG_texture_targets” in the texture identified by “material” and “path”.

[0052] AvatarTextureTarget definition AvatarTextureTarget The attribute is defined as follows: name type describe usage description String Describe the texture target and its purpose (e.g., for use in the UI). O purpose String The main purpose of defining this texture target. M range Number [2] Minimum and maximum values ​​of texture target weights. O .

[0053] Table 3 - AvatarTextureTarget Properties The "description" attribute describes the texture target and its intended use. This is purely illustrative; for example, the "description" attribute can be used in the user interface to help users understand the intended purpose of the texture target. For instance, if the purpose of the texture target is to represent a poisoned virtual image, the description could be "Poisoned," "Gives a poisoned status," or "Empoisoned," etc. If the "description" attribute is not present, the application can generate a description based on the "purpose" attribute.

[0054] The "purpose" attribute contains a string that identifies the purpose of the texture target. It must conform to a predefined value by the application or standard. This document presents a string type, but an enumeration type can also be used. For example, "avatar:health:poison" could be defined by a standard to indicate that the texture target can be used to represent a poisoned avatar. With a texture target weight of 0, the avatar should not appear poisoned. Then, as the application increases the texture target weight, the avatar will appear more severely poisoned.

[0055] The "range" attribute defines the minimum and maximum values ​​that a texture target weight can take. Applications use the "range" attribute to select the corresponding texture target weight value. For example, if "range" is [0.5, 1.0], the application should use a weight value of at least 0.5 for that texture target. For instance, if the texture target's purpose is a poisoned state, the application would still use zero weight for an unpoisoned avatar, but if the application wants to display a poisoned avatar, it must use a weight between 0.5 and 1.0.

[0056] glTF mode example For clarity, these examples only show the parts of the glTF file that are related to materials and nodes: we assume there is a scene with one (or more) meshes representing the virtual image.

[0057] Example We consider the following glTF file for encoding the virtual avatar's header: .

[0058] The idea behind this example is to encode the avatar's head using a skin texture that can be altered using parametric textures. This parametric texture has three texture targets: the first target gives a yellowish tint, as if the avatar is sick; the second target gives a greenish tint, as if the avatar is poisoned; and the third target adds a tattoo.

[0059] The glTF parser first decodes all content except for content from the "MPEG_node_avatar" extension. For most content, the standard process is followed. For the "MPEG_texture_targets" extension, it parses the "MPEG_texture_targets" attribute in "materials / pbrMetallicRoughness / baseColorTexture / extensions". The "targets" list contains three texture targets: -Texture target 0 references accessor 3, with weight 0; -Texture target 1 references accessor 4 with weight 0; -Texture target 2 references accessor 5, with weight 0.

[0060] Then, it decodes the “MPEG_node_avatar” content found in the first node. It indicates that the node is a virtual avatar (“isAvatar” is true), of standard type (“type” is “urn:mpeg:sd:2023:avatar”), and has a part corresponding to the header (“mappings” attribute).

[0061] Next, the "textures" attribute of "MPEG_node_avatar" is parsed. A single... AvatarTexture This is an item. The type of this avatar texture is "targets" ("type" is "targets"), which means that this texture defines the properties of a parameterized texture. This texture can be described using the message "Health" ("description" is "Health"). The purpose of this texture is to change the health appearance of the avatar ("purpose" is "avatar:health:status"). The string "avatar:health:status" is an example of the value of the "purpose" property and must be defined by the application or standard.

[0062] The virtual avatar texture uses the base color texture of the PBR (pbrMetallicRoughness) in the first material ("material" is 0, and "path" is "pbrMetallicRoughness / baseColorTexture"). TextureInfo The attribute includes the "MPEG_texture_targets" extension, which is consistent with the virtual avatar texture of type "targets". AvatarTexture The “targets” list has three items, consistent with the number of targets in the parameterized texture (the “targets” in “MPEG_texture_targets” has three items). That is, each avatar texture target in the first avatar texture definition of the MPEG_node_avatar extension will use the texture of the corresponding target in the MPEG_texture_targets extension: target 0 (“Disease”) uses accessor 3, and target 1 (“Poison”) uses accessor 4.

[0063] The first item in the "targets" list for the avatar texture is about the first texture target, which can be described by the string "Disease" (the "description" is "Disease"). The purpose of this texture target is to change the avatar texture so that he or she looks sick (the "purpose" is "avatar:health:disease"). To achieve this effect, we must use texture target weights between 0.5 and 1.0, because the "range" is [0.5, 1.0].

[0064] The second item in the list of "targets" for the avatar texture is about the second texture target, which can be described by the string "Poison" (the "description" is "Poison"). The purpose of this texture target is to change the avatar texture so that he or she appears poisoned (the "purpose" is "avatar:health:poison"). To achieve this effect, we must use a texture target weight between 0.2 and 0.8, since the "range" is [0.2, 0.8].

[0065] The last item in the "targets" list for the virtual avatar texture is ignored because the value of "purpose" is "unused".

[0066] The choice of value for the “purpose” attribute depends on the application or criterion and may differ from the value in this example.

[0067] Processing Model coding We assume the application collects all avatar texture definition data and stores it in some internal format. For example, this data could be a content creation application with a UI where a user can create a new avatar texture definition of a given type for each avatar, input a description, select a purpose, and define descriptions and purposes for some texture targets.

[0068] The application can then construct an internal JSON representation of the glTF file, in which all these virtual avatar texture definitions are added.

[0069] For each avatar, an MPEG_node_avatar extension is added. For those avatars with texture definitions, a "textures" list is created, with one entry for each avatar texture definition stored in the application's internal data. Each entry in the "textures" list is populated with a "type" attribute, a "description" attribute (if any), and a "purpose" attribute. For dynamic textures with targets, "type" is "targets," and a "targets" list is added to the avatar texture definition (and ultimately, a "weights" attribute if default weights are defined). In each entry in the "targets" list, we add a "description" attribute (if any description of the avatar texture target exists), a "purpose" attribute corresponding to the purpose of that avatar texture target, and a "range" attribute (if any weight range exists).

[0070] MPEG_node_avatar parsing Figure 5 The illustration shows a method for parsing a node with support for the MPEG_node_avatar extension according to one embodiment. Specifically, in step 510, the glTF loader first decodes the standard attributes and possible extensions. In step 520, if MPEG_node_avatar exists, the loader parses MPEG_node_avatar; otherwise, parsing ends (599). In step 530, the "isAvatar" attribute is parsed (required; an error occurs if it does not exist). In step 540, the "type" attribute is parsed (required; an error occurs if it does not exist). In step 550, the "mappings" attribute is parsed (required; an error occurs if it does not exist). In step 570, if the "textures" list exists, the "textures" list is parsed (560). The parsing of each item in the "textures" list is, for example, in... Figure 6The results of data found in the MPEG_node_avatar extension (if any) are stored in a dedicated structure and should not alter the scene.

[0071] AvatarTexture Analysis Figure 6 The illustration depicts a method for parsing items in the "textures" list of an MPEG_node_avatar according to one embodiment. Specifically, in step 610, the "purpose" attribute is parsed (required, error occurs if it does not exist). In step 620, the "material" and "path" attributes are parsed (required, error occurs if they do not exist). "material" must index existing material in the glTF. "path" must indicate the path within that material. TextureInfo Attributes. In step 640, if the "description" attribute exists, the "description" attribute is parsed (630). In step 650, if "type" is "main", the avatar texture type is "main", and parsing ends (699). If "type" is "targets" (660), the avatar texture type is "targets", and it is indicated by "material" and "path". TextureInfo Texture targets must be present; otherwise, an error (680) occurs. Then, the "targets" list must exist, and each item in this list is parsed in step 670. If the "weights" list exists, each item in the "weights" list is parsed in step 690. Parsing ends in step 699.

[0072] AvatarTextureTarget Analysis Figure 7 The illustration shows a solution according to one embodiment. AvatarTexture Items in the "targets" list ( AvatarTextureTarget The method is as follows: Specifically, in step 710, the "purpose" attribute is parsed (required; an error occurs if it does not exist). In step 730, if the "description" attribute exists, it is parsed (720). In step 750, if the "range" attribute exists, it is parsed (740). The process ends in step 799.

[0073] use cases Virtual Avatar Customization in the Virtual World In this use case, we assume the existence of virtual worlds manipulated by different companies, governments, or users. Users can import their avatars into these virtual worlds, and these avatars will function correctly in each virtual world that follows the same standards.

[0074] These avatars are expected to have many characteristics, such as the ability to change their textures. The need for change can stem from various reasons, such as presenting a good image for an interview, something fun to do with friends, or being affected by environmental conditions (cold, illness, etc.). To correctly change the texture, an application can examine the user's avatar file and see if a corresponding texture is found. For example, if a standard defines a purpose called "avatar:beautiful" for obtaining a beautiful avatar texture, an application can use "avatar:beautiful" for an interview. Similarly, if a purpose called "avatar:health:disease" exists for displaying a sick avatar, an application can use "avatar:health:disease" to present a sick avatar, for example, in a game where the avatar might be sick, or when the user triggers a "sick" emote. Texture changes can be binary: we replace the default avatar texture with another texture (e.g., the "main" avatar texture type). Textures can also be weighted: we replace the default avatar texture with a parameterized texture that has relevant weights (such as the "targets" avatar texture type).

[0075] Virtual avatar customization in virtual stores In this use case, we assume the existence of virtual stores operated by different companies, governments, or users. Users can import their avatars into these virtual stores, and these avatars will function correctly in each virtual store that follows the same standards.

[0076] People can buy clothing in virtual stores. They first import their avatars into the store, and then, assuming their avatars have their body measurements, they can try on clothes. They can change the texture of their avatars to see how the clothing suits them depending on the situation. For example, avatar textures can model their tan (weighted, from low to high) to help them see how swimsuits can complement their skin tone. Avatar textures can model their everyday makeup to see how dresses can suit them. Avatar textures can also model the amount of facial hair and show the relevance of accessories such as sunglasses.

[0077] Processing workflow When creating 3D assets, a common approach involves dividing the process into several stages. Each stage focuses on a specific task, such as handling mesh modeling, texturing, animation, lighting, etc. These stages are often handled by different people or teams, or by the same person at different times. To manage this, a good solution is to store the results of each stage in a new file. Using the current glTF standard, one might have already followed this approach for several tasks; however, this is not necessarily the case for other tasks, such as virtual avatar modeling.

[0078] Using the current glTF standard, a avatar's texture must be baked into a single texture. All creation parameters are lost during this process. As a result, if an artist receives a avatar with a baked texture, he or she cannot modify it: he or she must ask someone in the previous stage to modify the avatar with the baked texture and resend it. Using the proposed encoding, the artist can send a avatar with parametric textures: artists in subsequent stages can adjust the parametric texture weights themselves without requiring assistance from those in previous stages. Furthermore, the artist can describe and define the purpose of each texture and its parameters: this helps subsequent artists understand which textures to use and how to use them.

[0079] Various numerical values ​​are used in this application. These specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0080] Various methods are described herein, and each of these methods includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, in various embodiments, terms such as "first," "second," etc., may be used to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply a sequence of modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or within a time period overlapping with the second decoding.

[0081] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementations of the discussed features can be implemented in other forms (e.g., apparatus or program). Apparatus can be implemented, for example, in suitable hardware, software, and firmware. Methods can be implemented, for example, in an apparatus, such as a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate the transfer of information between end users.

[0082] References to "an embodiment" or "an embodiment" or "an implementation" or "an implementation," as well as other variations thereof, mean that the specific features, structures, characteristics, etc., described in connection with that embodiment are included in at least one embodiment. Therefore, the phrases "in an embodiment" or "in a manifestation" or "in an implementation" or "in an implementation," and any other variations appearing throughout this application, do not necessarily refer to the same embodiment.

[0083] Additionally, this application may relate to "determining" fragments of various information. Determining information may include one or more of, for example, estimation information, calculation information, prediction information, or information retrieved from memory.

[0084] Furthermore, this application may relate to “accessing” fragments of various information. Accessing information may include one or more of the following: receiving information, retrieving information (e.g., retrieving information from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0085] Additionally, this application may relate to "receiving" fragments of various information. As with "access," receiving is intended to be a broad term. Receiving information may include one or more of, for example, accessing information or retrieving information (e.g., retrieving information from memory). Furthermore, "receiving" is generally referred to in one or more ways during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0086] To be understood, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one of…” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be clear to those skilled in the art and related fields, this can be extended to as many items as possible listed.

[0087] As will be apparent to those skilled in the art, implementations can generate various signals that are formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of a spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

Claims

1. A method comprising: Obtain the representation type used to represent the virtual avatar from the description of the 3D scene; The texture of the virtual avatar is obtained from the description; as well as The virtual image is reconstructed based on the representation type and the texture.

2. An apparatus comprising one or more processors and at least one memory coupled to said one or more processors, wherein said one or more processors are configured to: Obtain the representation type used to represent the virtual avatar from the description of the 3D scene; The texture of the virtual avatar is obtained from the description; and The virtual image is reconstructed based on the representation type and the texture.

3. The method of claim 1, or the apparatus of claim 2, wherein the texture is indicated by at least one of the following: - Texture type -The purpose of the texture, -A descriptive description of the texture, and - The material used for the texture.

4. The method of claim 1 or 3, or the apparatus of claim 2 or 3, wherein the texture is formed as a combination of a base texture and one or more texture targets, and wherein the texture is further indicated by attributes and weights of each texture target used to form the texture.

5. The method of claim 3 or 4, or the apparatus of claim 3 or 4, wherein the texture type indicates whether the texture is a static texture or a dynamic texture.

6. The method of claim 4 or 5, or the apparatus of claim 4 or 5, wherein the attribute of the texture target is indicated by at least one of a descriptive description of the texture target, the purpose of the texture target, and a weight range of the texture target.

7. The method of claim 6, further comprising, or the apparatus of claim 6, wherein the one or more processors are further configured to perform: Select a weight for the corresponding texture target of the virtual avatar, wherein the weight is within the range.

8. The method according to any one of claims 1 and 3-7, further comprising, or according to any one of claims 2-7, means, wherein the one or more processors are further configured to perform: Select the texture for use in the virtual avatar.

9. A method comprising: The description of the 3D scene indicates the representation type used to represent the virtual avatar; as well as The description indicates the texture of the virtual avatar.

10. An apparatus comprising one or more processors and at least one memory coupled to said one or more processors, wherein said one or more processors are configured to: The description of the 3D scene indicates the representation type used to represent the virtual avatar; and The description indicates the texture of the virtual avatar.

11. The method of claim 9, or the apparatus of claim 10, wherein the texture is indicated by at least one of the following: - Texture type -The purpose of the texture, -A descriptive description of the texture, and - The material used for the texture.

12. The method of any one of claims 9 or 11, or the apparatus of any one of claims 10 or 11, wherein the texture is formed as a combination of a base texture and one or more texture targets, and wherein the texture is further indicated by attributes and weights of each texture target used to form the texture.

13. The method of claim 11 or 12, or the apparatus of claim 11 or 12, wherein the texture type indicates whether the texture is a static texture or a dynamic texture.

14. The method of claim 12 or 13, or the apparatus of claim 12 or 13, wherein the attribute of the texture target is indicated by at least one of a descriptive description of the texture target, the purpose of the texture target, and a weight range of the texture target.

15. A non-transitory computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1, 3-9, and 11-14.

Images