Avatar format validation in scene description

Abstract

Some embodiments of a method can include obtaining candidate avatar format data for an avatar, obtaining reference avatar format data for the avatar, wherein the reference avatar format data comprises a first set of one or more definitions; performing a validation process on each of the one or more definitions of the first set, wherein the validation process can include retrieving a current definition from the first set of one or more definitions; generating a comparison result by determining whether the candidate avatar format data satisfies the current definition; and if the comparison result is false, exiting the validation process; and if the comparison result is true for each iteration through the validation process, indicating that the candidate avatar format is valid.

Description

[0001] Cross-references to related applications This application claims the benefit of European patent application No. EP24305073, filed on January 11, 2024, entitled "Verification of Avatar Format in Scene Description", which is incorporated herein by reference in its entirety. Background Technology

[0002] This application is suitable for user representations of interaction in 3D scenes and immersive environments. User representations such as avatars are commonly used in 3D immersive environments. MPEG-1 Scene Description (SD) (23090-14) provides an interactive framework that supports the use of user representations (e.g., avatars) in these environments and in extended reality (XR) such as virtual reality (VR), augmented reality (AR), and / or mixed reality (MR). Summary of the Invention

[0003] A first example method according to some embodiments may include: obtaining candidate avatar format data of an avatar, obtaining reference avatar format data of an avatar, wherein the reference avatar format data includes a first set of one or more definitions; performing a verification process for each of the one or more definitions in the first set, wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; generating a comparison result by determining whether the candidate avatar format data satisfies the current definition; and exiting the verification process if the comparison result is false; and indicating that the candidate avatar format is valid if the comparison result is true for each iteration of the verification process.

[0004] For some embodiments of the first example method, the first set includes at least one definition regarding the mesh topology of the avatar.

[0005] For some embodiments of the first example method, the first set includes at least one definition of the semantic topology of the avatar.

[0006] For some embodiments of the first example method, the first set includes at least one definition regarding the deformation of the avatar.

[0007] In some embodiments of the first example method, the first set includes at least one definition regarding the skeleton of the avatar.

[0008] In some embodiments of the first example method, the first set includes at least one definition regarding the material of the avatar.

[0009] For some embodiments of the first example method, the first set includes at least one definition regarding the interactivity of the avatar.

[0010] For some embodiments of the first example method, the first set includes at least one definition regarding a first sub-format and at least one definition regarding a second sub-format.

[0011] Some embodiments of the first example method may further include: obtaining second reference avatar format data for the avatar, wherein the second reference avatar format data includes one or more defined second sets; repeating the verification process using the second reference avatar format data; and indicating that the candidate avatar format is valid only when the candidate avatar format is indicated as valid using at least one of the reference avatar format data or the second reference avatar format data.

[0012] In some embodiments of the first example method, at least one of the reference avatar format data, the second reference avatar format data, and the candidate avatar format data satisfies the MPEG_node_avatar extension standard.

[0013] A first example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0014] Example methods for verifying candidate avatar formats according to a reference avatar format according to some embodiments may include: comparing one or more features of the candidate avatar format with one or more definitions of the reference avatar format; and verifying the candidate avatar format if one or more features of the candidate avatar format satisfy the one or more definitions.

[0015] A third example method according to some embodiments may include: performing a verification process on each of one or more definitions in a first set of reference avatar format data, wherein the reference avatar format data includes the first set of one or more definitions, and wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; determining a comparison result by comparing the current definition with a set of candidate avatar format data; exiting the verification process if the comparison result is false; and indicating that the candidate avatar format is valid if the comparison result is true for each iteration of the verification process.

[0016] A third example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0017] A fourth example method according to some embodiments may include: obtaining reference avatar format data of an avatar, wherein the reference avatar format data includes a first set of one or more definitions; obtaining candidate avatar format data of the avatar; performing a verification process for each of the one or more definitions in the first set, wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; determining a comparison result by comparing the current definition with the candidate avatar format data; and exiting the verification process if the comparison result is false; and indicating that the candidate avatar format is valid if the comparison result is true for each iteration of the verification process.

[0018] A fourth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0019] A fifth example method according to some embodiments may include: obtaining reference avatar format data of an avatar, wherein the reference avatar format data includes a first set of one or more definitions; obtaining candidate avatar format data of the avatar; performing a verification process for each of the one or more definitions in the first set, wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; determining a comparison result by comparing the current definition with the candidate avatar format data; indicating that the candidate avatar format is invalid if the comparison result is false; exiting the verification process if the comparison result is false; returning to the beginning of the verification process if the comparison result is true and more definitions from the first set are yet to be retrieved; and indicating that the candidate avatar format is valid if the comparison result is true and all definitions from the first set have been retrieved.

[0020] A fifth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0021] A sixth example apparatus according to some embodiments may include at least one processor configured to perform any of the methods listed above.

[0022] A seventh example apparatus according to some embodiments may include a computer-readable medium storing instructions for causing one or more processors to perform any of the methods listed above.

[0023] An eighth example apparatus according to some embodiments may include at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform any of the methods listed above.

[0024] Example bitstreams according to some embodiments may include bitstreams generated according to any of the methods listed above. Attached Figure Description

[0025] Figure 1A This is a schematic side view of an example waveguide display that can be used with extended reality (XR) applications according to some embodiments.

[0026] Figure 1B This is a schematic side view illustrating example alternative display types that can be used with extended reality applications according to some embodiments.

[0027] Figure 1C This is a schematic side view illustrating example alternative display types that can be used with extended reality applications according to some embodiments.

[0028] Figure 1D This is a system diagram illustrating a set of example interfaces for a system according to some embodiments.

[0029] Figure 2 This is a schematic diagram illustrating an example MORGAN skeletal model according to some embodiments.

[0030] Figure 3 This is a flowchart illustrating an example avatar format verification process according to some embodiments.

[0031] Figure 4 This is a flowchart illustrating an example avatar format verification process according to some embodiments.

[0032] Entities, connections, arrangements, etc., depicted and combined with descriptions in various figures are presented by way of example and not by way of limitation. Similarly, any and all statements or other indications concerning what a particular figure “depicts,” what a particular element or entity in a particular figure “is” or “has,” and any and all similar statements (which, in isolation and out of context, might be interpreted as absolute and therefore limiting) may only be correctly interpreted as having been constructively preceded by a clause such as “In at least one embodiment, …”. For the sake of brevity and clarity, this implicit introductory clause will not be repeated verbatim in the detailed description. Detailed Implementation

[0033] Figure 1AThis is a schematic side view illustrating an example waveguide display that can be used with extended reality (XR) applications according to some embodiments. The image is projected by an image generator 102. The image generator 102 can project the image using one or more of a variety of technologies. For example, the image generator 102 can be a laser beam scanning (LBS) projector, a liquid crystal display (LCD), a light-emitting diode (LED) display (including organic LED (OLED) or micro LED (µLED) displays), a digital light processor (DLP), a liquid crystal on silicon (LCoS) display, or other types of image generators or light engines.

[0034] Light representing image 112 generated by image generator 102 is coupled into waveguide 104 via in-coupler 106. In-coupler 106 diffracts the light representing image 112 into one or more diffraction orders. For example, ray 108 representing one of the rays from the bottom portion of the image is diffracted by in-coupler 106, and one of the diffraction orders 110 (e.g., the second order) is at an angle capable of propagating through waveguide 104 via total internal reflection. Image generator 102 displays the image as instructed by control module 124, which operates to render image data, video data, point cloud data, or other displayable data.

[0035] At least a portion of the light 110, already coupled into waveguide 104 by diffraction-in coupler 106, is coupled out of the waveguide by diffraction-out coupler 114. At least some of the light coupled out of waveguide 104 replicates the angle of incidence of the light coupled into the waveguide. For example, in the illustration, the coupled rays 116a, 116b, and 116c replicate the angle of the coupled ray 108. Because the light leaving the out coupler replicates the direction of the light entering the in coupler, the waveguide essentially replicates the original image 112. The user's eye 118 can focus on the replicated image.

[0036] exist Figure 1A In the example, the outgoing coupler 114 couples out only a portion of the light from each reflection, allowing a single input beam (such as beam 108) to generate multiple parallel output beams (such as beams 116a, 116b, and 116c). In this way, even if the user's eye is not perfectly aligned with the center of the outgoing coupler, at least some of the light originating from each part of the image may reach the eye. For example, if the eye 118 moves downwards, beam 116c may enter the eye even if beams 16a and 116b do not, so the user can still perceive the bottom of image 112 despite the positional shift. Therefore, the outgoing coupler 114 partially functions as an exit pupil dilator in the vertical direction. The waveguide may also include one or more additional exit pupil dilators (…). Figure 1A(not shown in the image), so that the exit pupil expands in the horizontal direction.

[0037] In some embodiments, waveguide 104 is at least partially transparent with respect to light originating from outside the waveguide display. For example, at least some of the light 120 from a real-world object (e.g., object 122) passes through waveguide 104, allowing the user to see the real-world object when using the waveguide display. Since the light 120 from the real-world object also passes through diffraction grating 114, there will be multiple diffraction orders, and therefore multiple images. To minimize the visibility of multiple images, it is desirable to have high diffraction efficiency for the zeroth order (not deflected by 114) and the zeroth order for light 120, while higher diffraction orders are lower in energy. Therefore, in addition to expanding and coupling out virtual images, the out-coupler 114 is preferably configured to allow the zeroth order of the real image to pass through. In such embodiments, the image displayed by the waveguide display can appear to be superimposed on the real world.

[0038] Figure 1B This is a schematic side view illustrating example alternative display types that can be used with extended reality applications according to some embodiments. In the XR head-mounted display device 130, a control module 132 controls a display 134 (which may be an LCD) to display images. The head-mounted display includes a partially reflective surface 136 that reflects (and in some embodiments, both reflects and focuses) the image displayed on the LCD to make the image visible to the user. The partially reflective surface 136 also allows at least some external light to pass through, thereby allowing the user to see their surroundings.

[0039] Figure 1C This is a schematic side view illustrating example alternative display types that can be used with extended reality applications according to some embodiments. In the XR head-mounted display device 140, a control module 142 controls a display 144 (which may be an LCD) to display an image. The image is focused by one or more lenses of a display optics 146 to make the image visible to the user. Figure 1C In this example, the external light does not reach the user's eyes directly. However, in some such embodiments, an external camera 148 can be used to capture images of the external environment and display such images on a display 144 along with any virtual content that may also be displayed.

[0040] The embodiments described herein are not limited to any particular type or structure of XR display device.

[0041] Figure 1D This is a system diagram illustrating a set of example interfaces for a system according to some embodiments. Interfaces such as... Figure 1DThe system 150 is a system for implementing extended reality display devices and their control electronics. System 150 can be embodied as a device including the various components described below and configured to perform one or more of the aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 150 can be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 150 are distributed across multiple ICs and / or discrete components. In various embodiments, system 150 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 150 is configured to implement one or more of the aspects described in this document.

[0042] System 150 includes at least one processor 152 configured to execute instructions loaded thereon for implementing, for example, the various aspects described herein. Processor 152 may include embedded memory, input / output interfaces, and various other circuitry as known in the art. System 150 includes at least one memory 154 (e.g., a volatile memory device and / or a non-volatile memory device). System 150 may include a storage device 158, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 158 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0043] System 150 includes an encoder / decoder module 156 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 156 may include its own processor and memory. The encoder / decoder module 156 represents one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, as is known to those skilled in the art, the encoder / decoder module 156 may be implemented as a separate element of system 150, or it may be incorporated within processor 152 as a combination of hardware and software.

[0044] Program code to be loaded onto processor 152 or encoder / decoder 156 to execute the various aspects described in this document may be stored in storage device 158 and subsequently loaded onto memory 154 for execution by processor 152. According to various embodiments, one or more of processor 152, memory 154, storage device 158, and encoder / decoder module 156 may store one or more items of various kinds during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0045] In some embodiments, the memory within processor 152 and / or encoder / decoder module 156 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be processor 152 or encoder / decoder module 156) is used for one or more of these functions. External memory may be memory 154 and / or storage device 158, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding; also known as H.265 and MPEG H Part 2), or VVC (Multi-Functional Video Coding, a new standard being developed by the Joint Video Experts Group JVET).

[0046] Inputs to the components of system 150 can be provided by various input devices, as indicated in block 172. Such input devices include, but are not limited to: (i) an RF section that receives radio frequency (RF) signals transmitted over the air, for example by a broadcasting device; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a universal serial bus (USB) input terminal; and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 1C Other examples not shown include composite videos.

[0047] In various embodiments, as known in the art, the input device of block 172 has associated respective input processing elements. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements to perform these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0048] Furthermore, the USB and / or HDMI terminals may include their respective interface processors for connecting system 150 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) can be implemented as needed, for example, within a separate input processing IC or within processor 152. Similarly, aspects of USB or HDMI interface processing can be implemented as needed, either within a separate interface IC or within processor 152. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 152 and encoder / decoder 156, to operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0049] Various components of system 150 can be housed within an integrated housing. Within the integrated housing, various components can be interconnected and transmit data between them using a suitable connection arrangement 174 (e.g., internal buses as known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0050] System 150 includes a communication interface 160 that enables communication with other devices via a communication channel 162. The communication interface 160 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 162. The communication interface 160 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 162 may be implemented, for example, in a wired and / or wireless medium.

[0051] In various embodiments, data is streamed to or otherwise provided to system 150 using a wireless network such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 162 and a communication interface 160 suitable for Wi-Fi communication. The communication channel 162 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box that delivers data via an HDMI connection of input block 172 to provide streaming data to system 150. Still other embodiments use an RF connection of input block 172 to provide streaming data to system 150. As indicated above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks instead of Wi-Fi, such as cellular networks or Bluetooth networks.

[0052] System 150 can provide output signals to various output devices, including display 176, speaker 178, and other peripheral devices 180. Display 176 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 176 can be used in televisions, tablets, laptops, mobile phones, or other devices. Display 176 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). In various examples of embodiments, other peripheral devices 180 include one or more of a standalone digital video disc (or digital multifunction disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 180 that provide functionality based on the output of system 150. For example, a disc player performs the function of playing the output of system 150.

[0053] In various embodiments, signaling such as AV links, Consumer Electronics Control (CEC), or other communication protocols enabling inter-device control with or without user intervention is used to communicate control signals between system 150 and display 176, speaker 178, or other peripheral devices 180. Output devices can be communicatively coupled to system 1000 via dedicated connections through their respective interfaces 164, 166, and 168. Alternatively, output devices can be connected to system 150 via communication interface 160 using communication channel 162. Display 176 and speaker 178 can be integrated into a single unit with other components of system 150 in electronic devices such as, for example, a television set. In various embodiments, display interface 164 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0054] For example, if the RF portion of input 172 is part of a standalone set-top box, then display 176 and speaker 178 may alternatively be separate from one or more other components. In various embodiments where display 176 and speaker 178 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0055] System 150 may include one or more sensor devices 168. Examples of sensor devices that may be used include one or more GPS sensors, gyroscope sensors, accelerometers, light sensors, cameras, depth cameras, microphones, and / or magnetometers. Such sensors can be used to determine information such as the user's position and orientation. Where system 150 is used as a control module (such as control modules 124, 132) for an extended reality display, the user's position and orientation can be used to determine how image data is rendered so that the user perceives the correct portion of a virtual object or scene from the correct viewpoint. In the case of a head-mounted display device, the device's own position and orientation can be used to determine the user's position and orientation for the purpose of rendering virtual content. In the case of other display devices such as telephones, tablets, computer monitors, or televisions, other inputs can be used to determine the user's position and orientation for the purpose of rendering content. For example, the user can use a touchscreen, keypad or keyboard, trackball, joystick, or other inputs to select and / or adjust the desired viewpoint and / or viewing direction. When the display device has sensors such as accelerometers and / or gyroscopes, the viewpoint and orientation used for rendering content can be selected and / or adjusted based on the movement of the display device.

[0056] The embodiments can be implemented by computer software implemented by processor 152, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. As a non-limiting example, memory 154 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 152 can be of any type suitable for the technical environment and can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0057] This article describes the encoding of avatars for 3D scene representation, and the procedures for validating avatar 3D models (mesh, materials, structure, and / or related items) relative to an avatar reference model.

[0058] Current scene description formats (such as glTF and USD) can be used to encode 3D models representing avatars. Users can manipulate these models to move or interact in a virtual world. Some implementations can indicate and / or define which part of the scene is the avatar, as in glTF with the MPEG_node_avatar extension.

[0059] As understood, the current standard does not define how to verify a model that can be considered an incarnation satisfying the set of attributes. There appears to be no efficient way to encode all these attributes. Therefore, applications must know all these attributes beforehand to correctly instantiate the incarnation model.

[0060] Additional background information can be found in the references P. Ekman and WV Friesen. Measuring facial movements , 1:1 Environmental Psychology and Nonverbal Behavior 56-75 (1976) and FACS Quick Reference MELINDAOZEL <dot>It can be found in COM, and can be found in melindaozel. <dot>Obtained from com / facs-cheat-sheet / (last accessed on January 6, 2024).

[0061] A validation process can be used to ensure that candidate avatar formats meet the same properties as the reference avatar. If a candidate avatar is validated, the application can use the validated avatar as the reference avatar. Because of this approach, applications can handle many different avatar models, as long as they are validated using one of the reference avatars supported by the application. As understood, other systems have prior knowledge and can only handle data perfectly suited to them, and may not be able to use avatar data from external systems.

[0062] The verification process can be based on a set of definitions that the avatar format must satisfy. Each definition is identified by a unique ID such as MT1, MT2, or ST1. Each definition relates to one or more parameters or concepts in the model. For example, MT1 is approximately the number of vertices.

[0063] The validation process can use a reference avatar format that lists the definitions to be considered, and may ultimately include one or more specific values ​​appended to the definitions. For example, a sample MORGAN avatar format includes the MT1 definition specifying that the avatar mesh must consist of 53,698 vertices, which can be written in the following syntax: "MT1(Morgan) = 53,698 vertices". The validation process does not need to include all the definitions listed herein. According to some embodiments, candidate avatar formats only need to satisfy a subset of the reference avatar format definitions. Generally, such a validation process only processes a subset of these definitions.

[0064] The validation of candidate avatar formats ensures that every definition of the reference avatar format is satisfied. For example, a candidate avatar format validated for MORGAN avatar formats must have 53,698 vertices (e.g., the MT1 (Morgan) definition). The candidate avatar format must satisfy all reference avatar definitions. For the MORGAN example, the candidate avatar format must validate MT1 (Morgan) as well as other MORGAN definitions such as MT2 (Morgan) and MT4 (Morgan).

[0065] Mesh topology definition.

[0066] The avatar format can be a mesh topology, which may include, for example, mesh topology (MT) identifiers MT1 through MT5. Identifier MT1 may indicate the number of vertices. Identifier MT2 may indicate the number of faces (polygons). Identifier MT3 may indicate the order of the vertices. Identifier MT4 may indicate the order of the faces. Identifier MT5 may indicate the value of the vertex index in the face buffer.

[0067] The term "vertex" refers to the "POSITION" attribute in a glTF mesh primitive, and the term "face" refers to the "index" property of a glTF mesh. These properties can be found in a single glTF mesh primitive, or, in some embodiments, distributed across several primitives, where they are found from components referenced via MPEG_node_avatar. For example, an avatar mesh can be divided into many glTF meshes, where each mesh is referenced by a glTF node, and MPEG_node_avatar treats all of these nodes as child nodes. For more details on glTF, see the specification. glTF 2.0 KHRONOS 3D Format Working Group (2021).

[0068] Semantic topology definition.

[0069] An avatar format can define how the avatar grid is organized at the semantic level. This can include, for example, semantic topology (ST) identifiers ST1 to ST4.

[0070] The identifier ST1 indicates the presence of a part, such as a "head" or "leg". This presence can be mandatory or optional. For example, the format might require at least one arm and at most four arms. The part might be limited to a single point on the grid (i.e., a landmark). In some embodiments, different types of parts are possible. For example, the format might have grid segments and landmarks.

[0071] The identifier ST2 indicates that each body part can be defined by a unique identifier (such as an integer or a string). For example, "head" can be the identifier for the head. Identifiers can handle hierarchical relationships, such as "full_body / upper_body / head" which represents the head within the upper body of the full body.

[0072] The identifier ST3 indicates that each part can be associated with a low-level property, such as the range of face index or vertex index.

[0073] The identifier ST4 indicates a semantic structure definition, such as an indication that the head must be connected to the neck. This structure can be partitioned or hierarchical. For example, the head part can be included in the "upper body" part.

[0074] Deformation definition.

[0075] An avatar format can specify a process that can change the shape of the avatar. For example, an avatar format can include deformation (DF) identifiers DF1 to DF3.

[0076] The identifier DF1 can indicate the presence of a mesh deformable object, such as an animated or deformable target. For example, the format can be based on a Facial Action Coding System (FACS) model to indicate the deformable target and specify which elements are being used. See references P. Ekman and WV Friesen. Measuring facial movements 1:1 Environmental Psychology and Nonverbal Behavior 56-75 (1976) and FACS Criterion, MELINDA OZEL, available at melindaozel <dot>Information about FACS can be found at com / faces-cheat-sheet / (last accessed: January 6, 2024).

[0077] The identifier DF2 can indicate the location of each deformable object in the glTF description by means of an index of an array or name. For example, the name of the deformable target might be a name found in the "targetNames" of the "extras" property of the glTF mesh, or a name provided by the glTF extension. For example, the first deformable target could be action element #1 (AU1), the FACS element that raises the inner side of the eyebrow.

[0078] The identifier DF3 can indicate the weight basis of the deformable target, such as the vector space across the deformable target. For example, the FLAME face model can be used as described by Tianye, Li et al. Models for learning facial shapes and expressions from 4D scans It is used as described in ACM Transactions on Graphics 194:1-17 (2017) (Tianye), 38:6.

[0079] Deformation targets can be attached to some or all of the glTF meshes of the avatar mesh. The semantics of deformation targets attached to a mesh can be the same or different in the glTF meshes. For example, deformation targets in all glTF meshes of the avatar mesh can be associated with the same semantics. For example, the first deformation target in the glTF mesh may be associated with AU1 from FACS, but may only exist in the glTF meshes that move the inner eyebrow.

[0080] Skeletal definition.

[0081] An avatar format can use one or more skeleton (SK) identifiers SK1 through SK7 to define one or more skeletons. Identifier SK1 identifies the family or category of the skeleton. Identifier SK2 identifies the number of joints. Identifier SK3 identifies the semantics of each joint. Identifier SK4 identifies the connections between joints. Identifier SK5 identifies the semantics of each connection between joints. Identifier SK6 identifies the degrees of freedom of each joint and their range of values. Identifier SK7 identifies the number of possible joints associated with a body part. For example, three or five joints can be indicated for the right arm.

[0082] Material definition.

[0083] An avatar format can use one or more of the material (M) identifiers M1 to M3 to define one or more material properties.

[0084] The identifier M1 identifies a specific glTF material, where some properties are static and others are dynamic. For example, a specific glTF material can be defined where all properties are static except for the "baseColorFactor" field in the "pbrMetallicRoughness" component of the glTF material. The avatar format can define the range of correct values ​​for the properties. For example, the "baseColorFactor" field can allow values ​​from [0.5, 0.5, 0.5, 0] to [1.0, 1.0, 1.0, 1.0].

[0085] The identifier M2 identifies a set of allowed glTF materials. In some embodiments, some glTF materials may have dynamic properties, and similarly, in some embodiments, some glTF materials may be static, or a combination of dynamic and static properties.

[0086] The identifier M3 identifies the parameter characteristics. For example, the texture image used by the avatar material can be the result of a linear combination of textures. For faces, the FLAME model can be used (see...). Tianye ).

[0087] Other identifiers.

[0088] Avatar formats can define components related to interactivity. For example, body parts can be dedicated to specific actions. For instance, hands can be used to lift and / or pick up objects in a virtual world.

[0089] In some embodiments, the identifiers previously discussed can be applied to a single avatar format.

[0090] In some embodiments, an avatar format may include several subformats. Each subformat may use the identifiers discussed in previous sections. A valid avatar can be an avatar that satisfies all specifications / definitions of one of the subformats.

[0091] Transform into formatted data.

[0092] The MORGAN avatar format (as defined in MPEG) will be discussed below. An example avatar format has data segments for parts, vertices, faces, and bones. The MORGAN format is currently part of the MPEG specification family.

[0093] Table 1 shows example body part data. This section can include the names of all body parts, the number of vertices, the number of faces, and the index range of the faces. The first column of Table 1 lists the names of all body parts. These body parts are not necessarily partitions, as some parts may contain other parts. The second column of Table 1 lists the number of vertices in each part. The third column of Table 1 lists the number of faces in each part. The fourth column of Table 1 lists the index range of the faces.

[0094] For this example, the vertex data segment indicates the presence of 53,698 vectors (x, y, z). This value is indicated in the first row of Table 1 as a count of all vertices in the body. Vertices are encoded as vectors here. In this example, the term "vector" is used to interpret axes located in the (x, y, z) system. In some embodiments, the term "vector" may be replaced by the term "vertex".

[0095] The face data section in this example lists 106,952 triples (index of vertex 1, index of vertex 2, index of vertex 3). This value is indicated as the total face count in the first row of Table 1.

[0096] Figure 2 An example skeleton is shown in the figure, and this example skeleton is referenced in Table 2. Table 2 shows the corresponding skeleton. Figure 2 The skeletal data is as follows. As indicated in Table 2, there are 52 hierarchical joints. Each joint has a unique name and associated transformation. In addition, most of the joints listed in Table 2 also list their corresponding sub-body parts.

[0097] Figure 2 This is a schematic diagram illustrating an example MORGAN skeletal model according to some embodiments. Figure 2 An illustrated version of the MORGAN skeletal model 200 is shown for the skeletal body parts and joints listed in Table 2.

[0098] Mesh topology definition.

[0099] Using the data listed in Table 1, set several mesh topology identifiers. The MT1 (Morgan) identifier indicates that there are 53,698 vertices.

[0100] The MT2 (Morgan) identifier indicates the existence of 106,952 triangles.

[0101] The MT4 (Morgan) identifier indicates the format specification as shown in the Morgan incarnation. Potential improvements to CDAM2 23090-14 [SD] Update of Appendix H of the document – Reference Avatar MPEG Conference #143, the order of faces defined in input file m63997 (July 2023) ("m63997").

[0102] The MT5 (Morgan) indicator field indicates the value of the vertex index in the face buffer as defined in the MORGAN Avatar Representation Format Specification (m63997).

[0103] Semantic topology definition.

[0104] Using the data listed in Table 1, several semantic topology identifiers were set. The ST1 (Morgan) identifier indicates the existence of 46 parts, as shown in the first column of Table 1. Excluding the "full_body" row, there are 46 rows in Table 1, corresponding to the 46 parts.

[0105] The ST2 (Morgan) identifier lists a unique string used to identify each part. See the first column of Table 1.

[0106] The ST3 (Morgan) identifier lists the predefined number of vertices for each region, which is shown in the second column of Table 1. The ST3 (Morgan) identifier also lists the predefined face index range, which is shown in the fourth column of Table 1.

[0107] The ST4 (Morgan) identifier lists the parts that form a hierarchy. The forward slash " / " in the part name encodes the hierarchy within the part. For example, all IDs that begin with "full_body / upper_body / head / ..." are parts within "full_body / upper_body / head".

[0108] Skeletal definition.

[0109] Using the example data listed in Table 2, several bone identifiers were set up. An avatar format can define one or more bones using one or more definitions (such as the definitions for identifiers SK2, SK3, and SK4).

[0110] The identifier SK2 (Morgan) indicates the presence of 52 joints.

[0111] The SK3 (Morgan) identifier indicates that each joint has a unique name, and each name corresponds to a body part.

[0112] The SK4 (Morgan) identifier indicates the hierarchy that defines the joints it connects to.

[0113] verify.

[0114] For this example, the candidate incarnation format is valid relative to the MORGAN reference incarnation format if the candidate satisfies all of the following definitions: • Mesh topology defines MT1 (Morgan), MT2 (Morgan), MT4 (Morgan), and MT5 (Morgan). • Semantic topology definitions: ST1 (Morgan), ST2 (Morgan), ST3 (Morgan), and ST4 (Morgan) • Skeletal definition: SK2 (Morgan), SK3 (Morgan), and SK4 (Morgan).

[0115] By selecting these definitions, applications may be able to make assumptions and provide specific characteristics. For example, verifying any avatar referenced by Morgan could assume that the hand is always in the same location in the grid (e.g., within the same vertex index range), even if the geometry of the hand differs (e.g., the (x, y, z) values ​​of the hand vertices may differ). For some embodiments, new definitions (such as MTx, or even new definition categories, such as YYx) can be used with this verification process. For reference avatar formats, many other combinations of definitions can be selected (even those not listed above).

[0116] For some embodiments, there is no definition of a minimum number that the reference avatar format must have. For some embodiments, the reference avatar format may be limited to a single definition, such as only the number of vertices (MT1 in the MORGAN format).

[0117] Figure 3 This is a flowchart illustrating an example avatar format verification process according to some embodiments. In processing blocks 301 and 302, candidate avatar formats (to be verified or not verified) and reference avatar formats (whose definition is used for verification) are received and / or retrieved as input. In processing block 303, the next definition of the reference avatar format is retrieved. D Each definition in the reference avatar format (e.g., "MT1 (<reference avatar format name>)") is used to analyze the format.

[0118] In decision box 304, relative to the current definition of the reference avatar format. D The candidate avatar data is analyzed and evaluated. Evaluation can be direct or indirect. For example, for the definition MT1, MT1 is valid if the candidate format explicitly defines the number of vertices. In another case, if a vertex buffer is provided with a format, the number of vertices can be calculated. Such calculations can be performed to satisfy / verify the MT1 definition. If the current definition is not satisfied / verified, the example procedure proceeds to box 305 to indicate that the candidate format is invalid. In decision box 306, evaluation is performed to determine if there are more definitions in the reference avatar format. If no definition exists, the procedure proceeds to procedure box 307 to indicate that the candidate format is valid. If more definitions exist, the procedure returns to retrieve the next definition from the reference format.

[0119] In some embodiments, the verification process can be completed offline before runtime. In other embodiments, the verification process can be completed at runtime when the candidate avatar format is potentially used.

[0120] In some embodiments, reference avatar format data blocks and / or candidate avatar format data blocks may conform to the MPEG_node_avatar extension specification. In some embodiments, each avatar URN (in the "Type" attribute of MPEG_node_avatar) may be "bound" to a reference avatar. If a standard URN is used, the avatar model can be used under the assumption that the avatar model meets all the requirements of the reference model. In some embodiments, the user of the avatar model may run all checks to ensure that the avatar model meets all the requirements of the reference model.

[0121] In some embodiments, the verification process can be used with MPEG_node_avatar to verify the contents of the glTF regarding the URN, which is defined in the "Type" attribute of the MPEG_node_avatar. For example, if the "Type" field is "urn:mpeg:sd:2023:avatar" (e.g., MORGAN), then the avatar bindings (rigs) within the glTF must satisfy all MORGAN definitions (MT1, ...). The verification process can also be used with other URNs or newer standardized avatars. For example, the URN "urn:mpeg:sd:2024:avatar" can be used with the MORGAN 2.0 format, which can be the current MORGAN format with updated and new / updated definitions, such as MT1 or other definition identifiers.

[0122] Figure 4 This is a flowchart illustrating an example avatar format verification process according to some embodiments. In some embodiments, example process 400 may include obtaining candidate avatar format data for avatars at step 402. In some embodiments, example process 400 may further include obtaining reference avatar format data for avatars at step 404, wherein the reference avatar format data includes a first set of one or more definitions. In some embodiments, example process 400 may further include performing a verification process at step 406 on each of the one or more definitions in the first set. In some embodiments, the verification process of example process 400 may include retrieving a current definition from the first set of one or more definitions at step 408. In some embodiments, the verification process of example process 400 may further include generating a comparison result at step 410 by determining whether the candidate avatar format data satisfies the current definition. In some embodiments, the verification process of example process 400 may further include exiting the verification process at step 412 if the comparison result is false. In some embodiments, example process 400 may further include determining at step 414 whether more definitions remain in the first set. If more definitions remain in the first set, example procedure 400 may return to the top of the loop to retrieve the current definition 408 from the first set of one or more definitions. Otherwise, example procedure 400 continues. In some embodiments, example procedure 400 may also include indicating that the candidate avatar format 416 is valid if the comparison result is true for each iteration of the verification procedure.

[0123] While the methods and systems according to some embodiments are generally discussed in the context of extended reality (XR), some embodiments can be applied to any XR context, such as, for example, virtual reality (VR) / mixed reality (MR) / augmented reality (AR) contexts. Furthermore, although the term "head-mounted display (HMD)" is used herein according to some embodiments, for some embodiments, some embodiments can be applied to wearable devices capable of, for example, XR, VR, AR, and / or MR (which may or may not be attached to the head).

[0124] A first example method according to some embodiments may include: obtaining candidate avatar format data of an avatar, obtaining reference avatar format data of an avatar, wherein the reference avatar format data includes a first set of one or more definitions; performing a verification process for each of the one or more definitions in the first set, wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; generating a comparison result by determining whether the candidate avatar format data satisfies the current definition; and exiting the verification process if the comparison result is false; and indicating that the candidate avatar format is valid if the comparison result is true for each iteration of the verification process.

[0125] For some embodiments of the first example method, the first set includes at least one definition regarding the mesh topology of the avatar.

[0126] For some embodiments of the first example method, the first set includes at least one definition of the semantic topology of the avatar.

[0127] For some embodiments of the first example method, the first set includes at least one definition regarding the deformation of the avatar.

[0128] In some embodiments of the first example method, the first set includes at least one definition regarding the skeleton of the avatar.

[0129] In some embodiments of the first example method, the first set includes at least one definition regarding the material of the avatar.

[0130] For some embodiments of the first example method, the first set includes at least one definition regarding the interactivity of the avatar.

[0131] For some embodiments of the first example method, the first set includes at least one definition regarding a first sub-format and at least one definition regarding a second sub-format.

[0132] Some embodiments of the first example method may further include: obtaining second reference avatar format data for the avatar, wherein the second reference avatar format data includes one or more defined second sets; repeating the verification process using the second reference avatar format data; and indicating that the candidate avatar format is valid only when the candidate avatar format is indicated as valid using at least one of the reference avatar format data or the second reference avatar format data.

[0133] In some embodiments of the first example method, at least one of the reference avatar format data, the second reference avatar format data, and the candidate avatar format data satisfies the MPEG_node_avatar extension standard.

[0134] A first example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0135] Example methods for verifying candidate avatar formats according to a reference avatar format according to some embodiments may include: comparing one or more features of the candidate avatar format with one or more definitions of the reference avatar format; and verifying the candidate avatar format if one or more features of the candidate avatar format satisfy the one or more definitions.

[0136] A third example method according to some embodiments may include: performing a verification process on each of one or more definitions in a first set of reference avatar format data, wherein the reference avatar format data includes the first set of one or more definitions, and wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; determining a comparison result by comparing the current definition with a set of candidate avatar format data; exiting the verification process if the comparison result is false; and indicating that the candidate avatar format is valid if the comparison result is true for each iteration of the verification process.

[0137] A third example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0138] A fourth example method according to some embodiments may include: obtaining reference avatar format data of an avatar, wherein the reference avatar format data includes a first set of one or more definitions; obtaining candidate avatar format data of the avatar; performing a verification process for each of the one or more definitions in the first set, wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; determining a comparison result by comparing the current definition with the candidate avatar format data; and exiting the verification process if the comparison result is false; and indicating that the candidate avatar format is valid if the comparison result is true for each iteration of the verification process.

[0139] A fourth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0140] A fifth example method according to some embodiments may include: obtaining reference avatar format data of an avatar, wherein the reference avatar format data includes a first set of one or more definitions; obtaining candidate avatar format data of the avatar; performing a verification process for each of the one or more definitions in the first set, wherein the verification process includes: retrieving a current definition from the first set of one or more definitions; determining a comparison result by comparing the current definition with the candidate avatar format data; indicating that the candidate avatar format is invalid if the comparison result is false; exiting the verification process if the comparison result is false; returning to the beginning of the verification process if the comparison result is true and more definitions from the first set are yet to be retrieved; and indicating that the candidate avatar format is valid if the comparison result is true and all definitions from the first set have been retrieved.

[0141] A fifth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0142] A sixth example apparatus according to some embodiments may include at least one processor configured to perform any of the methods listed above.

[0143] A seventh example apparatus according to some embodiments may include a computer-readable medium storing instructions for causing one or more processors to perform any of the methods listed above.

[0144] An eighth example apparatus according to some embodiments may include at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform any of the methods listed above.

[0145] Example bitstreams according to some embodiments may include bitstreams generated according to any of the methods listed above.

[0146] This disclosure describes a wide variety of aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in detail and are generally described in a manner that may sound limiting, at least for the purpose of illustrating individual characteristics. However, this is for the purpose of clarity and does not limit the disclosure or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide further aspects. Furthermore, the aspects described can also be combined and interchanged with those described in earlier applications.

[0147] The aspects described and contemplated in this disclosure can be implemented in many different forms. While some embodiments are specifically illustrated, other embodiments are contemplated, and the discussion of particular embodiments does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, a computer-readable storage medium having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having a bitstream generated according to any of the described methods stored thereon.

[0148] In this disclosure, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0149] The terms HDR (High Dynamic Range) and SDR (Standard Dynamic Range) generally convey specific values ​​of dynamic range to those skilled in the art. However, additional embodiments are also contemplated, in which references to HDR are understood to mean "higher dynamic range" and references to SDR are understood to mean "lower dynamic range". Such additional embodiments are not bound by any specific value of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range".

[0150] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first," "second," etc., can be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply a sequence of modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or in a time period overlapping with the second decoding.

[0151] For example, various numerical values ​​may be used in this disclosure. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0152] The embodiments described herein can be implemented by computer software implemented by a processor or other hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. As a non-limiting example, the processor can be of any type suitable for the technical environment and can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0153] Various implementations involve decoding. As used in this disclosure, "decoding" can encompass all or part of a process performed, for example, on a received encoded sequence, to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes, or alternatively includes, processes performed by a decoder of the various implementations described in this disclosure, such as extracting an image from a tile (packed) image, determining an upsampling filter to use and then upsampling the image, and flipping the image back to its intended orientation.

[0154] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, based on the specific context of the description, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process.

[0155] Various implementations involve encoding. In a manner similar to the discussion above regarding "decoding," the term "encoding," as used herein, can encompass all or part of a process performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such a process also includes, or alternatively includes, processes performed by an encoder of the various implementations described herein.

[0156] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. It will be clear, based on the specific context of the description, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process.

[0157] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0158] Various embodiments refer to rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between bit rate and distortion is typically considered, often taking into account constraints on computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of bit rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, and a complete evaluation of their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of encoding costs and associated distortion.

[0159] The implementations and aspects described herein can be implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatus or program). An apparatus can be implemented, for example, in suitable hardware, software, and firmware. A method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. A processor also includes communication devices, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate communication of information between end-users.

[0160] The references to "an embodiment" or "an embodiment" or "an implementation" or "an implementation," and other variations thereof, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the appearance of the phrases "in an embodiment" or "in an embodiment" or "in an implementation" or "in an implementation," and any other variations appearing throughout this disclosure, do not necessarily refer to the same embodiment.

[0161] Furthermore, this disclosure may refer to "determining" each piece of information. Determining information may include, for example, one or more of estimated information, calculated information, predicted information, or information retrieved from memory.

[0162] Furthermore, this disclosure may refer to "accessing" individual pieces of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0163] Furthermore, this disclosure may refer to "receiving" individual pieces of information. Like "access," receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) one or more of them. Moreover, "receiving" is generally referred to in one manner or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0164] To understand, for example, in the cases of "A / B", "A and / or B", and "at least one of A and B", the use of any of the following " / ", "and / or", and "at least one" is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to as many listed items as possible.

[0165] Furthermore, as used herein, the term "signaling" specifically refers to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a particular one of several parameters used for region-based filter parameter selection for artifact removal filtering. In this way, in one embodiment, the same parameter is used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be done in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the term "signaling" has been referred to above, the term "signaling" can also be used as a noun herein.

[0166] The implementation can generate various signals that are formatted to carry information, such as information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described embodiments. For example, the signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0167] We have described numerous embodiments. Features of these embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim classes and types: • Adapt the residual at the encoder according to any of the embodiments discussed. • Includes one or more bitstreams or signals of the described syntax element or its variants. • Includes a bitstream or signal that conveys the syntax of information generated according to any of the described embodiments. • Insert elements into the signaling syntax elements that enable the decoder to adapt the residuals in a manner corresponding to that used by the encoder. • Create and / or transmit and / or receive and / or decode bit streams or signals including one or more of the described syntax elements or their variants. • Create and / or transmit and / or receive and / or decode according to any of the described embodiments. • A method, process, apparatus, medium for storing instructions, medium for storing data, or signal according to any of the described embodiments. • TVs, set-top boxes, mobile phones, tablets, or other electronic devices that perform filter parameter adaptation according to any of the described embodiments. • TVs, set-top boxes, mobile phones, tablets, or other electronic devices that perform filter parameter adaptation according to any of the described embodiments and display (e.g., using a monitor, screen, or other type of display) the resulting image. • TVs, set-top boxes, mobile phones, tablets, or other electronic devices that select (e.g., using a tuner) a channel to receive signals including encoded images and perform filter parameter adaptation according to any of the described embodiments. • TVs, set-top boxes, mobile phones, tablets, or other electronic devices that receive signals including encoded images over the air (e.g., using an antenna) and perform filter parameter adaptation according to any of the described embodiments.

[0168] Note that the various hardware elements in one or more embodiments described herein are referred to as "modules," which perform (i.e., execute, implement, etc.) the various functions described herein in connection with the respective modules. As used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more memory devices) that a person skilled in the art would consider suitable for a given implementation. Each described module may further include executable instructions for performing one or more functions described as being performed by the respective module, and note that these instructions may take the form of hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or the like, or include hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or the like, and may be stored in one or more suitable non-transitory computer-readable media, such as commonly referred to as RAM, ROM, etc.

[0169] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor storage devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROMs and digital versatile discs (DVDs)). The processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.< / dot> < / dot> < / dot>

Claims

1. A method comprising: Obtain the candidate avatar format data. Obtain the reference avatar format data for the avatar. The reference avatar format data includes one or more defined first sets; Perform the verification process for each of one or more definitions in the first set. The verification process includes: Retrieve the current definition from a first set of the one or more definitions; A comparison result is generated by determining whether the candidate avatar format data satisfies the current definition; and If the comparison result is false, then the verification process is terminated; and If the comparison result is true for each iteration of the verification process, it indicates that the candidate avatar format is valid.

2. The method according to claim 1, wherein, The first set includes at least one definition of the mesh topology of the avatar.

3. The method according to any one of claims 1-2, wherein, The first set includes at least one definition of the semantic topology of the incarnation.

4. The method according to any one of claims 1-3, wherein, The first set includes at least one definition regarding the transformation of the incarnation.

5. The method according to any one of claims 1-4, wherein, The first set includes at least one definition of the skeleton of the avatar.

6. The method according to any one of claims 1-5, wherein, The first set includes at least one definition regarding the material of the avatar.

7. The method according to any one of claims 1-6, wherein, The first set includes at least one definition regarding the interactivity of the avatar.

8. The method according to any one of claims 1-7, wherein, The first set includes at least one definition of a first subformat and at least one definition of a second subformat.

9. The method according to any one of claims 1-8, further comprising: Obtain the second reference avatar format data of the avatar. The second reference avatar format data includes one or more defined second sets; The verification process is repeated using the second reference avatar format data, and The candidate avatar format is indicated to be valid only when at least one of the reference avatar format data or the second reference avatar format data is indicated to be valid.

10. The method according to any one of claims 1-9, wherein, At least one of the reference avatar format data, the second reference avatar format data, and the candidate avatar format data satisfies the MPEG_node_avatar extension standard.

11. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform the method of any one of claims 1 to 10.

12. A method for verifying candidate avatar formats by comparing them with a reference avatar format, comprising: Compare one or more features of the candidate avatar format with one or more definitions of the reference avatar format; as well as If one or more features of the candidate avatar format satisfy one or more of the definitions, then the candidate avatar format is verified.

13. A method comprising: Perform a verification process for each of one or more definitions in the first set within the reference avatar format data. The reference avatar format data includes one or more defined first sets, and The verification process includes: Retrieve the current definition from a first set of one or more definitions; The comparison result is determined by comparing the current definition with a set of candidate avatar format data; and If the comparison result is false, then the verification process is terminated; and If the comparison result is true for each iteration of the verification process, it indicates that the candidate avatar format is valid.

14. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by a processor, are operable to cause the apparatus to perform the method of claim 13.

15. A method comprising: Obtain the reference avatar format data for the avatar. The reference avatar format data includes one or more defined first sets; Obtain the candidate incarnation format data of the incarnation; Perform the verification process for each of one or more definitions in the first set. The verification process includes: Retrieve the current definition from a first set of one or more definitions; The comparison result is determined by comparing the current definition with the candidate avatar format data; and If the comparison result is false, then the verification process is terminated; and If the comparison result is true for each iteration of the verification process, it indicates that the candidate avatar format is valid.

16. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform the method of claim 15.

17. A method comprising: Obtain the reference avatar format data for the avatar. The reference avatar format data includes one or more defined first sets; Obtain the candidate incarnation format data of the incarnation; Perform the verification process for each of one or more definitions in the first set. The verification process includes: Retrieve the current definition from a first set of one or more definitions; The comparison result is determined by comparing the current definition with the candidate avatar format data; If the comparison result is false, it indicates that the candidate avatar format is invalid; If the comparison result is false, then the verification process is terminated; If the comparison result is true and more definitions from the first set are yet to be retrieved, then return to the beginning of the verification process; and If the comparison result is true and all definitions from the first set have been retrieved, it indicates that the candidate avatar format is valid.

18. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform the method of claim 17.

19. An apparatus comprising at least one processor configured to perform the method of any one of claims 1-10, 12, 13, 15 and 17.

20. An apparatus comprising a computer-readable medium storing instructions for causing one or more processors to perform the method of any one of claims 1-10, 12, 13, 15 and 17.

21. An apparatus comprising at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform the method of any one of claims 1-10, 12, 13, 15 and 17.

22. A signal comprising a bit stream generated according to any one of claims 1-10, 12, 13, 15 and 17.

Images