Method and apparatus for associating attributes at different levels of hierarchical grid codec

Abstract

Dynamic grid encoding and decoding systems and methods. In an example decoding method, an encoded dynamic grid includes an encoded base grid, atlas information, and higher level mapping information associating a respective lower level attribute identifier with each of a plurality of higher level attributes. The encoded base grid is decoded, wherein the encoded base grid includes the encoded static grid, motion information, and lower level mapping information. The static grid includes a plurality of static grid attributes, and the lower level mapping information associates a lower level attribute identifier with each of the plurality of static grid attributes. The static mesh is decoded, including decoding a static mesh attribute mapped to at least one of the higher level attributes. In some embodiments, static grid attributes that are not mapped to any higher level attributes are not decoded.

Description

[0001] Cross-referencing This application claims priority to European Patent Application No. 23306552.3, filed on September 20, 2023, entitled “Methods and Apparatus for Associating Attributes at Different Levels of a Hierarchical Mesh Codec”, which is incorporated herein by reference in its entirety. Background Technology

[0002] This disclosure relates to systems and methods for encoding and decoding meshes. While this disclosure is not limited to any particular mesh encoding technique, some embodiments may utilize edgebreaker techniques. Edgebreaker is a technique capable of efficiently encoding the connectivity of triangular meshes. In its simplest implementation, a mesh encoded using edgebreaker is represented by an ordered sequence of symbols C, L, E, R, and S (referred to as a "CLERS" sequence). Generally, starting with an initial triangle, these symbols describe different ways of attaching new triangles, providing information about whether or how different edges of the new triangle are connected to one or more existing triangles.

[0003] Following the MPEG V-Mesh (now renamed V-DMC) proposal call, a solution proposed by Apple was selected as the basis for the MPEG V-Mesh Test Model (TM). The proposal is described in “m59281 - [V-CG] Apple's Dynamic Mesh Coding CfP Response”, Apple Inc., 2022, by K. Mammou, J. Kim, A. Tourapis, and D. Podborski. In this disclosure… Figure 1 and Figure 2 A summary of the diagrams from that proposal is provided in the document.

[0004] For some frames, the test model encodes the base mesh using a static mesh encoder, then subdivides it to obtain an approximation of the original mesh. Currently, the V-Mesh test model encodes the base mesh using a Google Draco implementation of an edgebreaker-based mesh encoder. Specifically, this implementation uses a "spirale reversi" version of the edgebreaker as described in M. Isenburg and J. Snoeyink's "Spirale Reversi: Reverse decoding of the edgebreaker encoding", Computational Geometry, vol. 20, pp. 39-52, 2001.

[0005] For V-DMC, the framework is developed by extending V3C. V3C is described in ISO / IEC 23090-5:2021, Informationtechnology — Coded representation of immersive media — Part 5: Visualvolumetric video-based coding (V3C) and video-based point cloud compression (V-PCC).

[0006] In MPEG, the V-DMC standard has begun working on a static mesh codec. Annex-I of ISO / IEC 23090-29 specifies the static mesh codec. The static mesh codec in Annex I is the default codec for compressing static meshes. This includes mesh geometry, mesh connectivity, and mesh properties. Mesh properties provide additional information about mesh vertices or mesh faces.

[0007] The V-DMC framework can involve different properties specified for dynamic mesh sequences. In V-DMC, the underlying static mesh codec can decode mesh properties on a per-face or per-vertex basis. These properties can provide additional information about the static mesh, such as color, texture coordinates, normals, reflectivity information, transparency information, and / or user-defined properties.

[0008] Attributes from the static mesh codec can be further used by the base mesh codec. The base mesh bitstream is further encapsulated within a V3C bitstream. The layered structure of the V-DMC bitstream is... Figure 6As shown in the diagram, the extension mechanism in the V3C bitstream can provide information about the type of attributes from the underlying base mesh codec that the V3C bitstream / frame will use. Furthermore, the base mesh bitstream can provide a mechanism to provide information about the type of mesh attributes of interest from the underlying static mesh codec. Summary of the Invention

[0009] According to some embodiments, a dynamic mesh encoding method includes: obtaining an encoded static mesh, the encoded static mesh including a plurality of static mesh attributes; encoding a base mesh based on the encoded static mesh and motion information, the encoded base mesh including lower-level mapping information, the lower-level mapping information assigning lower-level attribute identifiers to each static mesh attribute in at least a subset of the static mesh attributes; and encoding a dynamic mesh based on the encoded base mesh and atlas information, the encoded dynamic mesh including higher-level mapping information, the higher-level mapping information assigning one of the corresponding lower-level attribute identifiers to each of a plurality of higher-level attributes.

[0010] In some embodiments, one or more of the attributes are identified by an index. In some embodiments, one or more of the attributes are identified by any unique identifier.

[0011] In some embodiments, the lower-level attribute identifier is an index. In other embodiments, the lower-level attribute identifier is any unique identifier.

[0012] According to some embodiments, a dynamic mesh decoding method includes: obtaining an encoded dynamic mesh, the encoded dynamic mesh including an encoded base mesh, atlas information, and higher-level mapping information, the higher-level mapping information assigning corresponding lower-level attribute identifiers to each of a plurality of higher-level attributes; decoding the encoded base mesh, the encoded base mesh including an encoded static mesh, motion information, and lower-level mapping information, wherein the static mesh includes a plurality of static mesh attributes, and wherein the lower-level mapping information assigns lower-level attribute identifiers to each of the plurality of static mesh attributes; and decoding the encoded static mesh, including decoding at least a first static mesh attribute, wherein the first static mesh attribute is mapped to at least one of the higher-level attributes through the lower-level mapping information and the higher-level mapping information.

[0013] In some embodiments, at least some (or all) static mesh properties that are not mapped to higher-level properties are not decoded, or their values ​​may be discarded.

[0014] The example embodiments further include the following: a corresponding decoding technique; means including one or more processors configured to perform the methods herein; means including at least one processor and a computer-readable medium storing instructions for performing the methods herein; a computer-readable medium storing a grid encoded according to the methods herein; and a signal conveying the grid encoded according to the methods herein. Attached Figure Description

[0015] The following detailed description will be better understood when read in conjunction with the accompanying drawings, which illustrate examples of one or more embodiments of the present disclosure. However, it should be understood that the embodiments described herein are not limited to the precise arrangements and instrumentalities shown in the drawings.

[0016] Figure 1 This is a functional block diagram of an example grid coding system.

[0017] Figure 2 This is a functional block diagram of an example grid decoding system.

[0018] Figure 3 An example of an edgebreaker mesh codec according to some embodiments is illustrated.

[0019] Figure 4 This is a flowchart of the encoding process according to some embodiments.

[0020] Figure 5 This is a flowchart of the decoding process according to some embodiments.

[0021] Figure 6 The illustration schematically depicts the encapsulation of different sub-bitstreams in a bitstream that encodes a dynamic grid (such as a V-DMC bitstream).

[0022] Figure 7 The diagram illustrates the indices of different levels of grid attributes from the V-DMC bitstream.

[0023] Figure 8 The diagram illustrates the mapping of grid properties via unique arbitrary identifiers (UAIs) from different levels in the V-DMC bitstream.

[0024] Figure 9 The diagram illustrates the mapping of properties between the V3C extension and the base mesh codec via indexes, and the mapping of properties between the base mesh codec and the static mesh codec via UAI.

[0025] Figure 10 This is a block diagram of an example system that implements various aspects and embodiments. Detailed Implementation

[0026] An overview of grid geometry encoding.

[0027] Figure 3 The illustration shows an example of an edgebreaker grid codec that can be used for grid coding according to some embodiments. The top row represents the encoding lines, and the bottom row represents the decoding lines. Figure 3 As illustrated in the figures, the example embodiments include some or all of the following. During encoding, preprocessing 302 can be used to remove potential connectivity problems (non-manifold edges and vertices) that may exist on the input mesh. This cleanup is performed because the edgebreaker algorithm itself cannot handle meshes with such connectivity problems. In some embodiments, removing non-manifold edges and vertices involves copying multiple points. Some embodiments keep track of those copied vertices to merge them during decoding. This allows for a reduction in the number of points in the decoded mesh, but requires sending some additional information in the bitstream. In some embodiments, the preprocessing 302 further includes adding some dummy points to fill potential holes on the surface, since the edgebreaker algorithm alone does not handle holes. The example embodiment operates to fill holes before encoding and recreate holes after decoding. The example embodiment uses “virtual” dummy points and generates and encodes dummy triangles attached to these dummy points, but the 3D positions of those points are not encoded or decoded. In some embodiments, vertex attributes are quantized if needed. Those attributes can be provided to the already quantized encoder.

[0028] The example implementation uses a modified version of the edgebreaker algorithm to encode mesh connectivity, thereby generating a CLERS table (a table consisting of 'C', 'L', 'E', 'R', and 'S' symbols). This stage also generates some tables in memory for the attribute prediction stage. Vertex attributes are then predicted, starting with position attributes. Other attributes are then predicted, ultimately depending on the position prediction, which is the case for texture UV coordinates. Configuration and metadata are also provided in the bitstream; the CLERS table, some other connectivity data, and all attribute prediction residuals are entropy-encoded and added to the bitstream.

[0029] In the example decoding method, all entropy-encoded sub-bitstreams are decoded. Mesh connectivity is reconstructed using the CLERS table and the edgebreaker algorithm. Additional information can be used to manage the handles describing the topology. All other per-vertex positions are predicted using the mesh connectivity and a minimal set of vertex positions expressed in 3D coordinates. Attribute residuals are applied to correct the predictions, resulting in the reconstructed vertex positions. Other attributes are then decoded, possibly depending on the location of the decoded position, such as for UV coordinates. Connectivity of attributes using a separate index table is reconstructed using entropy-encoded per-edge binary seam information.

[0030] In the post-processing stage, virtual triangles are removed. If the encoder is configured to perform lossless encoding, the non-manifold problem can be recreated, and if the model has been quantized by the encoder, vertex attributes can be dequantized.

[0031] Figure 4 This is a flowchart of the encoding process that can be used in some embodiments.

[0032] Figure 5 This is a flowchart of the decoding process that can be used in some embodiments.

[0033] An overview of attribute relationships in grid coding.

[0034] The goal is to carefully map the mesh attributes encoded by the MPEG static mesh codec to attributes that will be exposed to the underlying mesh codec for further use. The bitstream encoded using the underlying mesh codec provides references to the attributes encoded in the static mesh bitstream. In the same spirit, in the V3C extensions, specifically the V3C Parameter Set V-DMC extension, attributes are encoded in the underlying mesh bitstream. The mesh attributes of the underlying static mesh bitstream are abstracted by the underlying mesh codec, which in turn exposes some attributes to the V3C high-level syntax extensions.

[0035] All base mesh properties with different base mesh property types may potentially be exposed in the V3C parameter set extension of V-DMC. However, the current semantics of the base mesh property types lack some of the required properties, as described in Section 1.b below.

[0036] Some implementations allow for a one-to-one correspondence between attributes defined in V3C extensions and the base mesh codec, as well as between the base mesh codec and the static mesh codec. For example, two ATTR_NORMALS attributes can correspond to information provided in two MESH_ATTR_NORMALS attributes in the static mesh codec, provided that the mapped attributes are equivalent. However, the current draft of ISO / IEC 23090-29 does not describe rules for equivalence.

[0037] Furthermore, the base mesh codec can select a subset of mesh attributes. For example, a static mesh bitstream may contain two MESH_ATTR_NORMAL attributes. On the other hand, the base mesh bitstream may only be interested in one normal attribute and require mapping ATTR_NORMAL to one of the MESH_ATTR_NORMAL attributes in the static mesh codec. Since the base mesh bitstream abstracts the static mesh codec, this functionality also enables V3C extensions via the base mesh bitstream to expose only a subset of the encoded attributes from the static mesh bitstream. There may be cases where the V3C extension might only use a subset of the base mesh attributes. Such flexibility can be achieved using appropriate selection mechanisms as described in the different embodiments described herein.

[0038] In some cases, only one attribute of one attribute type exists in a static mesh codec, but the underlying mesh bitstream signals to reference many of the same attributes, resulting in duplication. In some embodiments, such referencing is prohibited. In some embodiments, this is achieved by defining a range of the number of attributes to be referenced at higher levels of the V-DMC hierarchy, adhering to the number of attributes available at lower levels of the V-DMC hierarchy, as described below in Part 1.

[0039] The example implementation addresses the potential for inconsistent attribute mappings between the V3C extension and the underlying mesh codec. In the V3C extension of V-DMC, `vps_ext_bmesh_data_attribute_count` is one of the syntax elements. This syntax element indicates the number of attributes in the underlying mesh of an atlas with an atlas ID. In the underlying mesh sequence parameter set, `bmsps_mesh_attribute_count` indicates the number of attributes associated with a mesh encoded using a static mesh codec.

[0040] The range of vps_ext_bmesh_data_attribute_count is not currently specified. However, it is assumed that since vps_ext_bmesh_data_attribute_count references the base mesh attributes, the range of vps_ext_bmesh_data_attribute_count must be within the range of the number of attributes available in the base mesh, i.e., bmsps_mesh_attribute_count.

[0041] Some implementations provide equal attribute counts for the V3C extension and the underlying mesh codec. Two mentioned syntax elements can correspond to the same value, such that `vps_ext_bmesh_data_attribute_count = bmsps_mesh_attribute_count`. The corresponding attributes can have the same type. In this case, there is no need to map attributes, as this is an ideal one-to-one correspondence. However, it is possible for the referenced attributes to have different types, i.e., `vps_ext_bmesh_attribute_type` ≠ `bmsps_mesh_attribute_type_id`. This can lead to issues regarding bitstream consistency.

[0042] The example implementation addresses the discrepancy in attribute counts between the V3C extension and the base mesh codec. For instance, in some cases, the syntax element `vps_ext_bmesh_data_attribute_count` is not equal to `bmsps_mesh_attribute_count`. In cases where `vps_ext_bmesh_data_attribute_count` is greater than `bmsps_mesh_attribute_count`, there is a possibility of decoder crashing because the V3C extension might want to reference base mesh attributes that are not present in the base mesh bitstream. However, there might be cases where the V3C extension wants to reference attributes multiple times. Such cases are impractical and likely uninteresting to the application. However, the decoder will not crash.

[0043] On the other hand, if `vps_ext_bmesh_data_attribute_count` is less than `bmsps_mesh_attribute_count`, then a subset of the base mesh attributes is referenced by the V3C extension. The current syntax in the V3C extension lacks the ability to signal which attributes in the attribute list of the base mesh bitstream should be referenced. In some cases, multiple identical attributes may exist in the base mesh bitstream; for example, a base mesh bitstream containing two texture coordinate attributes and one color attribute. The current syntax in the V3C extension lacks the ability to signal how to select a subset (e.g., only one texture coordinate) from multiple identical attributes.

[0044] Some implementations address the issue of inconsistent attribute mapping between the base mesh codec and the static mesh codec. In the base mesh bitstream, `bmsps_mesh_attribute_count` is one of the syntax elements. Syntax elements indicate the number of attributes associated with a mesh. The mesh is provided by the static mesh codec. In the MPEG edge breaker static mesh codec, `mesh_attribute_count` indicates the number of encoded attributes. Currently, the range of `bmsps_mesh_data_attribute_count` is limited to 0 to 127 (inclusive). However, the range of `mesh_attribute_count` has not yet been specified.

[0045] Some implementations address problems that can arise even when the underlying mesh codec and the static mesh codec have equal attribute counts. For example, two mentioned syntax elements may correspond to the same value, where `bmsps_mesh_attribute_count` equals `mesh_attribute_count`. The corresponding attributes may have the same type. In this case, there is no need to map attributes, as this is an ideal one-to-one correspondence. However, there may be cases where the types of the referenced attributes differ, i.e., `bmsps_mesh_attribute_type_id` is different from `mesh_attribute_type`. This can lead to problems regarding bitstream consistency.

[0046] Some implementations address issues related to different attribute counts between the base mesh codec and the static mesh codec, for example, where the syntax element `bmsps_mesh_attribute_count` is not equal to `mesh_attribute_count`. In cases where `bmsps_mesh_attribute_count` is greater than `mesh_attribute_count`, there is a possibility of decoder crashing because the base mesh bitstream might want to reference static mesh attributes that are not present in the static mesh bitstream. However, there might be cases where the base mesh bitstream wants to reference mesh attributes multiple times. Such cases are impractical and likely uninteresting to the application. However, the decoder will not crash.

[0047] If bmsps_mesh_attribute_count is less than mesh_attribute_count, then a subset of the base mesh attributes is referenced by the base mesh bitstream. The current syntax in the base mesh SPS lacks the ability to signal which attributes in the attribute list of the base mesh bitstream should be referenced.

[0048] In some cases, multiple identical properties may exist in a static mesh bitstream. For example, a static mesh bitstream may contain two texture coordinate properties and one color property. The current syntax in the underlying mesh bitstream lacks the ability to signal how to select a subset (e.g., only one texture coordinate) from multiple identical properties.

[0049] The problem addressed in some embodiments.

[0050] The layered structure of the V-DMC codec and the complex and flexible use of its attributes can lead to inconsistent referencing of mesh attributes between different levels of the codec structure. For example, a base mesh codec might only be interested in the "texture coordinates" attribute from a static mesh. However, the static mesh bitstream may encapsulate other attributes as well as "texture coordinates." Similarly, a V3C codec might only be interested in the "normal" attribute from a base mesh codec. However, the base mesh codec may encapsulate additional attributes as well as "normals." This can lead to compatibility issues if the required attributes are missing from the underlying bitstream. The V-DMC specification does not provide a mechanism to ensure consistency in attribute usage and referencing across the codec's layered structure.

[0051] Furthermore, multiple properties of the same type may exist within a static mesh bitstream. However, the current V-DMC specification lacks information to signal to the underlying mesh codec which property(s) of the same type is of interest. The same applies between the underlying mesh codec and the V3C codec. The V-DMC specification does not provide a mechanism to indicate which(s) of the same type can be used by the parent codec.

[0052] Implementations of some example embodiments.

[0053] The example implementation provides a mapping mechanism for associating attributes from V3C extensions with a base mesh bitstream, and for associating a base mesh bitstream with a static mesh bitstream. Several different implementations are described herein.

[0054] §1. Some embodiments impose additional semantic constraints on the bitstream of the encoded lattice. In the current working draft, some semantics in the grammatical elements are not explicitly defined, or some items are missing. Some embodiments described herein involve additions to the grammar of existing data structures; in the examples below, some such additions are marked with a dagger (†).

[0055] §1.a. Some embodiments impose constraints on attribute counts in VPS extensions. For example, where vps_ext_bmesh_data_attribute_count[j] indicates the number of attributes in the base mesh of the atlas with atlas ID j, and where the value of vps_ext_mesh_data_attribute_count[j] is in the range of 0 to N (inclusive), a constraint can be imposed such that N equals bmsps_mesh_attribute_count - 1. In that case, the modified semantics of vps_ext_bmesh_data_attribute_count[j] might be expressed (read) as follows: vps_ext_bmesh_data_attribute_count[j] indicates the number of attributes in the base mesh of the atlas with atlas ID j. vps_ext_mesh_data_attribute_count[j] will be in the range of 0 to N (inclusive), where N is bmsps_mesh_attribute_count - 1.

[0056] §1.b. Some embodiments impose constraints on the count of mesh attributes in the underlying mesh. For example, where bmsps_mesh_attribute_count indicates the number of attributes associated with a mesh, and where the value of bmsps_mesh_attribute_count is in the range of 0 to N (inclusive), a constraint can be imposed such that N is the number of attributes provided in the mesh bitstream encoded using the static mesh codec identified by bmsps_intra_mesh_codec_id. In that case, the modified semantics of bmsps_mesh_attribute_count might be expressed as follows: bmsps_mesh_attribute_count indicates the number of attributes associated with a mesh. bmsps_mesh_attribute_count will be in the range of 0 to N (inclusive), where N is the number of attributes provided in the mesh bitstream encoded using the static mesh codec identified by bmsps_intra_mesh_codec_id.

[0057] §1.c. Some embodiments introduce texture coordinate attributes into the underlying mesh codec. For example, where bmsps_mesh_attribute_type_id[i] indicates the attribute type of the mesh with index i, the value bmsps_mesh_attribute_type_id[i] = 6 can be used to indicate that the attribute type is texture coordinates. The semantics of bmsps_mesh_attribute_count can be expressed as follows: bmsps_mesh_attribute_type_id[i] indicates the attribute type of the attribute with index i in the mesh. Table H-1 describes the list of supported attributes and their relationship to bmsps_mesh_attribute_type_id[i].

[0058] Table H-2 can be used to describe a list of supported attributes and their relationship to bmsps_mesh_attribute_type_id[i]. Table H-2 can be modified by adding rows marked with a dagger (†), as follows: Table H-1

[0059] In this embodiment, a new `bmsps_mesh_attribute_type_id` is introduced. The new `bmsps_mesh_attribute_type_id` is an identifier for texture coordinates. Texture coordinates are two-component and / or three-component attributes that provide information about the mapping pixels from the texture to the 3D mesh geometry. In practice, two-component texture coordinates are typically used.

[0060] §2. Some embodiments provide references to attributes using attribute indexes. A base mesh can have several attributes. Attributes provide additional information about the mesh. This additional information corresponds to non-locational information. Base mesh attributes are referenced by the V3C extension of V-DMC. The V3C framework uses the referenced base mesh attributes for processes such as reconstructing and decoding other components. Similarly, attributes from the static mesh bitstream are referenced by the base mesh codec. The base mesh codec uses the static mesh attributes for the reconstruction, decoding, or other purposes of other components.

[0061] Figure 7 The diagram illustrates the indices of different levels of grid attributes from the V-DMC bitstream. For example... Figure 7 As shown in the example, the V-DMC extension of the V3C codec references two of the three properties of the base mesh codec. Furthermore, the base mesh codec references three of the four properties of the static mesh codec.

[0062] In an exemplary embodiment, references are made by the higher-level structure using the index of the attribute. The index of the attribute corresponds to the position of the attribute in the attribute array of the underlying codec.

[0063] In an example embodiment of the mesh encoding method, an encoded static mesh 702 is obtained. The encoded static mesh 702 includes multiple static mesh attributes 704, 706, 708, and 710. A base mesh 712 is encoded based on the encoded static mesh 702 and motion information. The encoded base mesh includes lower-level mapping information, such as information associated with attributes 714, 716, and 718, which associates lower-level attribute identifiers with each static mesh attribute in at least a subset of the static mesh attributes. Figure 7In some embodiments, lower-level attribute identifiers can be indices. For example, base mesh mapping information 714 may include an index "0" indicating its association with a first static mesh attribute 704, base mesh mapping information 716 may include an index "1" indicating its association with a second static mesh attribute 706, and base mesh mapping information 718 may include an index "3" indicating its association with a fourth static mesh attribute 710. As seen in this example, not all attributes in the encoded static mesh 702 must be referenced in the encoded base mesh; for example, static mesh attribute 708 is not referenced in the mapping information of the encoded static mesh 702. The example encoding method further includes encoding a dynamic mesh 720 based on the encoded base mesh 712 and atlas information. The encoded dynamic mesh 720 includes higher-level mapping information for attributes 722, 724, which associates each of a plurality of higher-level attributes with one of the corresponding lower-level attribute identifiers. For example, in Figure 7 In the mapping information of dynamic mesh attribute 722, there is an index (e.g., "0") that identifies the base mesh attribute 714 (which is associated with the static mesh attribute 704); and the mapping information of dynamic mesh attribute 724 includes an index (e.g., "2") that identifies the base mesh attribute 718 (which is associated with the static mesh attribute 710).

[0064] In an example embodiment of the mesh decoding method, an encoded dynamic mesh 720 is obtained. The encoded dynamic mesh includes an encoded base mesh 712, atlas information, and higher-level mapping information for attributes 722, 724. The higher-level mapping information associates a corresponding lower-level attribute identifier with each of a plurality of higher-level attributes. For example, the mapping information for dynamic mesh attribute 722 includes an index (e.g., "0") identifying base mesh attribute 714, while the mapping information for dynamic mesh attribute 724 includes an index (e.g., "2") identifying base mesh attribute 718. The encoded base mesh 712 is decoded. The encoded base mesh 712 includes an encoded static mesh 702, motion information, and lower-level mapping information for attributes 714, 716, 718. The static mesh includes a plurality of static mesh attributes 704, 706, 708, 710. The lower-level mapping information associates a lower-level attribute identifier with each of a plurality of static mesh attributes. The lower-level attribute identifier may be an index. For example, the base mesh mapping information for attribute 714 may include an index "0" indicating its association with the first static mesh attribute 704, the base mesh mapping information for attribute 716 may include an index "1" indicating its association with the second static mesh attribute 706, and the base mesh mapping information for attribute 718 may include an index "3" indicating its association with the fourth static mesh attribute 710. The encoded static mesh 702 is decoded, including decoding at least the first static mesh attributes (such as attributes 704 and / or 710), wherein the first static mesh attributes are mapped to at least one of the higher-level attributes (in this example, attributes 722 or 724) through lower-level mapping information and higher-level mapping information.

[0065] §2.a. Some embodiments operate to identify the index of the underlying mesh attribute in the V-DMC stream. According to some embodiments, the referencing system provides mechanisms for selecting attribute sets from the underlying sub-bitstream / sub-sub-bitstream and allowing the use of multiple identical attribute types. In some embodiments, the syntax of vps_vdmc_extension can be modified by adding lines marked with a dagger (†), as follows:

[0066] In some examples, the semantics of vps_ext_bmesh_attribute_index[j][i] can be as follows: vps_ext_bmesh_attribute_index[j][i] is the index of the i-th attribute in the array of specified attributes, which is signaled through the underlying mesh for the i-th attribute type.

[0067] §2.b. Some embodiments operate to identify the index of a static mesh attribute in the base mesh bitstream. In the base mesh SPS, attribute_index / property_index is added to the attributes in the static mesh codec. The attribute_index can reference the implicit ID of the encoded attribute (an implicit ID based on the encoding order). In some embodiments, the syntax of bmesh_sequence_parameter_set_rbsp can be modified by adding a line marked with a dagger (†), as follows:

[0068] In some examples, the semantics of bmsps_mesh_attribute_index[i] can be as follows: bmsps_mesh_attribute_index[i] specifies the index of the i-th attribute in the array of attributes in the mesh for the i-th attribute type.

[0069] In some embodiments, the encoder encodes or otherwise obtains an encoded static mesh, wherein the encoded static mesh includes a plurality of static mesh attributes. The encoder encodes a base mesh based on the encoded static mesh and motion information. The encoded base mesh includes lower-level mapping information that assigns lower-level attribute identifiers to each static mesh attribute in at least a subset of the static mesh attributes. For example, a data structure such as bmesh_sequence_parameter_set_rbsp can be used to associate each lower-level attribute identifier (which may be an index i) with the corresponding static mesh attribute indicated by bmsps_mesh_attribute_index[i]. (The value of i may be implicit, for example, it may be inferred from the ordering of the values ​​bmsps_mesh_attribute_index[i] in the bitstream.) The encoder also encodes a dynamic mesh based on the encoded base mesh and atlas information. The encoded dynamic mesh includes higher-level mapping information that associates each higher-level attribute with a corresponding lower-level attribute identifier in at least a subset of the lower-level attribute identifiers. For example, a data structure such as `vps_vdmc_extension` may include information that associates each higher-level attribute (which may be associated with index `i`) with the corresponding lower-level attribute identifier `vps_ext_bmesh_attribute_index[j][i]`. (The value of `i` may be implicit, for example, it may be inferred from the order of the values ​​`vps_ext_bmesh_attribute_index[j][i]` in the bitstream.) Different values ​​of `j` may be associated with different atlases.

[0070] In some embodiments, the decoder obtains an encoded dynamic mesh, for example, in a received bitstream. The encoded dynamic mesh includes an encoded base mesh, atlas information, and higher-level mapping information that associates each of a plurality of higher-level attributes with a corresponding lower-level attribute identifier. For example, a data structure such as vps_vdmc_extension may include information that associates each higher-level attribute (which may be identified by an index i) with a corresponding lower-level attribute identifier vps_ext_bmesh_attribute_index[j][i]. (The value of i may be implicit, for example, it may be inferred from the ordering of the values ​​vps_ext_bmesh_attribute_index[j][i] in the bitstream.) Different values ​​of j may be associated with different atlases. The decoder decodes the encoded base mesh, wherein the encoded base mesh includes an encoded static mesh, motion information, and lower-level mapping information. The static mesh includes a plurality of static mesh attributes, and the lower-level mapping information assigns lower-level attribute identifiers to each of the plurality of static mesh attributes. For example, a data structure such as `bmesh_sequence_parameter_set_rbsp` can be used to associate each lower-level attribute identifier (which may be an index `i`) with a corresponding static mesh attribute indicated by `bmsps_mesh_attribute_index[i]`. (The value of `i` can be implicit, for example, it can be inferred from the order of the values ​​`bmsps_mesh_attribute_index[i]` in the bitstream.) The decoder decodes the encoded static mesh, including decoding at least a first static mesh attribute, wherein at least the first static mesh attribute has an assigned first lower-level attribute identifier, and the first lower-level attribute identifier is associated with at least a first higher-level attribute. (In the case where the first static mesh attribute has an assigned first lower-level attribute identifier and wherein the lower-level attribute identifier is associated with a higher-level attribute, the first static mesh attribute can be described as mapped to a higher-level attribute.) It may be noted that in some embodiments, the term "higher level" refers to the VDMC level, but the principles described herein are not limited to any particular implementation or nomenclature for mesh coding. Similarly, the term "lower level" in some embodiments refers to the base mesh level, and is not limited to any particular coding standard.

[0071] §3. Some embodiments perform referencing by using a unique attribute identifier. In example embodiments, a unique attribute identifier (UAI) may be used instead of an index position for referencing. A UAI is a mechanism that can be used to uniquely reference attributes. In some embodiments, this functionality is also used for attributes in the underlying grid bitstream.

[0072] Figure 8 The diagram illustrates how grid properties are mapped via unique attribute identifiers from different levels in the V-DMC bitstream. For example... Figure 8 As shown in the example, the V-DMC extension of the V3C codec references two of the three properties of the base mesh codec via the base mesh UAI. Furthermore, the base mesh codec references three of the four properties of the static mesh codec via the mesh UAI. In this embodiment, the references are performed by the higher structure using the UAI of the attribute codec in the lower level.

[0073] §3.a. Some embodiments operate by identifying attributes in the underlying mesh bitstream via unique identifiers in the V3C codec. At the V3C V-DMC extension, underlying mesh attributes are referenced by encoding a unique, arbitrary identifier for each attribute. In cases where there are multiple different types of attributes, each instance is also encoded with a corresponding UAI. The UAI allows identification of attribute items in the underlying mesh attribute list. The identifier can remain arbitrary to allow flexibility in not defining a strict mapping from UAI to attributes. In some embodiments, the syntax of vps_vdmc_extension can be modified by adding a line marked with a dagger (†), as follows:

[0074] In some examples, the semantics of vps_ext_bmesh_attribute_unique_id[j][i] can be as follows: vps_ext_bmesh_attribute_unique_id[j][i] specifies a unique attribute identifier for the i-th attribute encoded in the underlying mesh bitstream for the i-th attribute type in a atlas with atlasID j.

[0075] To support this functionality, in some embodiments, the underlying mesh codec currently specified in ISO / IEC 23090-29 is modified by adding a syntax element to the underlying mesh bitstream that specifies a UAI for each attribute present in the underlying mesh bitstream. For example, a syntax element named bmsps_attribute_unique_id can be introduced. This syntax element specifies an arbitrary unique identifier for each attribute in the underlying mesh bitstream. In some embodiments, the syntax of bmesh_sequence_parameter_set_rbsp can be modified by adding a line marked with a dagger (†), as follows:

[0076] In some examples, the semantics of bmsps_attribute_unique_id can be as follows: bmsps_attribute_unique_id specifies a unique attribute identifier for the i-th attribute encoded in the trellis bitstream for the i-th attribute type.

[0077] §3.b. Some embodiments operate to identify attributes in the mesh bitstream via unique identifiers in the underlying mesh codec. In some embodiments, the syntax of bmesh_sequence_parameter_set_rbsp can be modified by adding lines marked with a dagger (†), as follows:

[0078] In some examples, the semantics of bmsps_mesh_attribute_unique_id[i] can be as follows: bmsps_mesh_attribute_unique_id[i] specifies a unique identifier for the i-th attribute encoded in the mesh bitstream for the i-th attribute type.

[0079] In some embodiments, the encoder encodes or otherwise obtains an encoded static mesh, wherein the encoded static mesh includes a plurality of static mesh attributes. The encoder encodes a base mesh based on the encoded static mesh and motion information. The encoded base mesh includes lower-level mapping information that assigns lower-level attribute identifiers to each static mesh attribute in at least a subset of the static mesh attributes. For example, a data structure such as bmesh_sequence_parameter_set_rbsp can be used to associate each lower-level attribute identifier with a corresponding static mesh attribute indicated by an arbitrary unique identifier at the static mesh level, such as bmsps_mesh_attribute_unique_id, which can be an arbitrary unique attribute identifier at the base mesh level, such as bmsps_attribute_unique_id. The encoder also encodes a dynamic mesh based on the encoded base mesh and atlas information. The encoded dynamic mesh includes higher-level mapping information that assigns each higher-level attribute (which may use index i as a higher-level attribute identifier) ​​to a corresponding lower-level attribute identifier in at least a subset of the lower-level attribute identifiers. For example, a data structure such as `vps_vdmc_extension` may include information that associates each higher-level attribute (which may be identified by an index `i`) with a corresponding lower-level attribute identifier, which may be any unique attribute identifier at the underlying mesh level, such as `vps_ext_bmesh_attribute_unique_id[j][i]`. (The value of `i` may be implicit, for example, it may be inferred from the order of the values ​​`bmsps_mesh_attribute_index[i]` in the bitstream.) In some embodiments, the decoder obtains the encoded dynamic mesh, for example, in the received bitstream. The encoded dynamic mesh includes the encoded base mesh, atlas information, and higher-level mapping information that assigns each higher-level attribute to a corresponding lower-level attribute identifier. For example, a data structure such as vps_vdmc_extension may include information that associates each higher-level attribute (which may be identified by an index i) with a corresponding lower-level attribute identifier, which may be any unique attribute identifier at the base mesh level, such as vps_ext_bmesh_attribute_unique_id[j][i]. (The value of i may be implicit, for example, it may be inferred from the ordering of the values ​​bmsps_mesh_attribute_index[i] in the bitstream.) The decoder decodes the encoded base mesh, which includes the encoded static mesh, motion information, and lower-level mapping information. The static mesh includes multiple static mesh attributes, and the lower-level mapping information assigns lower-level attribute identifiers to each of the multiple static mesh attributes. For example, a data structure such as `bmesh_sequence_parameter_set_rbsp` can be used to associate each lower-level attribute identifier with a corresponding static mesh attribute indicated by an arbitrary unique identifier at the static mesh level, such as `bmsps_mesh_attribute_unique_id`, which can be an arbitrary unique attribute identifier at the base mesh level, such as `bmsps_attribute_unique_id`. The decoder decodes the encoded static mesh, including decoding at least a first static mesh attribute, wherein the first static mesh attribute has an assigned first lower-level attribute identifier, and the first lower-level attribute identifier is associated with at least a first higher-level attribute.

[0080] §4. Some embodiments use unique attribute identifiers to reference the static mesh and indexes to reference the underlying mesh codec. Such embodiments can be implemented as a combination of Parts 2 and 3. A potential advantage of such embodiments compared to those using unique identifiers at both levels is that attributes in V3C can follow an index-based referencing mechanism used for underlying mesh attributes. This reduces the need to introduce unique identifiers for underlying mesh attributes.

[0081] On the other hand, static mesh codecs (which can be MPEG-defined or non-MPEG-defined static mesh codecs, such as Draco) can provide additional information about attributes, such as a unique identifier associated with each attribute.

[0082] Figure 9The diagram illustrates the mapping of properties between the V3C extension and the base mesh codec via indexes, and between the base mesh codec and the static mesh codec via UAI. For example... Figure 9 As shown in the example, the V-DMC extension of the V3C codec references two of the three properties of the underlying mesh codec via its corresponding index. Furthermore, the underlying mesh codec references three of the four properties of the static mesh codec via the mesh UAI.

[0083] §4.a. Some embodiments identify attributes in the underlying mesh bitstream via indexes in the V3C codec. In some embodiments, the syntax of vps_vdmc_extension can be modified by adding lines marked with a dagger (†), as follows:

[0084] In some examples, the semantics of vps_ext_bmesh_attribute_index can be as follows: vps_ext_bmesh_attribute_index specifies the index of the i-th attribute encoded in the underlying mesh bitstream for the i-th attribute type of the atlas with atlasID j.

[0085] §4.b. Some embodiments operate to identify attributes in the mesh bitstream via unique identifiers in the underlying mesh codec. In some embodiments, the syntax of bmesh_sequence_parameter_set_rbsp can be modified by adding lines marked with a dagger (†), as follows:

[0086] In some examples, the semantics of bmsps_mesh_attribute_unique_id can be as follows: bmsps_mesh_attribute_unique_id specifies a unique identifier for the i-th attribute encoded in the mesh bitstream.

[0087] In some embodiments, the encoder encodes or otherwise obtains an encoded static mesh, wherein the encoded static mesh includes a plurality of static mesh attributes. The encoder encodes a base mesh based on the encoded static mesh and motion information. The encoded base mesh includes lower-level mapping information that assigns lower-level attribute identifiers to each static mesh attribute in at least a subset of the static mesh attributes. For example, a data structure such as bmesh_sequence_parameter_set_rbsp can be used to associate each lower-level attribute identifier (which may be an index i) with the corresponding static mesh attribute indicated by bmsps_mesh_attribute_index[i]. (The value of i may be implicit, for example, it may be inferred from the ordering of the values ​​bmsps_mesh_attribute_index[i] in the bitstream.) The encoder also encodes a dynamic mesh based on the encoded base mesh and atlas information. The encoded dynamic mesh includes higher-level mapping information that assigns each higher-level attribute to a corresponding lower-level attribute identifier in at least a subset of the lower-level attribute identifiers. For example, a data structure such as `vps_vdmc_extension` may include information that associates each higher-level attribute (which may be identified by an index `i`) with a corresponding lower-level attribute identifier, which may be any unique attribute identifier at the underlying mesh level, such as `vps_ext_bmesh_attribute_unique_id[j][i]`. (The value of `i` may be implicit, for example, it may be inferred from the order of the values ​​`bmsps_mesh_attribute_index[i]` in the bitstream.) In some embodiments, the decoder obtains an encoded dynamic mesh, for example, in a received bitstream. The encoded dynamic mesh includes an encoded base mesh, atlas information, and higher-level mapping information that assigns a corresponding lower-level attribute identifier to each of a plurality of higher-level attribute identifiers. For example, a data structure such as vps_vdmc_extension may include information that associates each higher-level attribute, which can be identified by index i, with a corresponding lower-level attribute identifier, which may be any unique attribute identifier at the base mesh level, such as vps_ext_bmesh_attribute_unique_id[j][i]. (The value of i may be implicit, for example, it may be inferred from the ordering of the values ​​bmsps_mesh_attribute_index[i] in the bitstream.) The decoder decodes the encoded base mesh, which includes an encoded static mesh, motion information, and lower-level mapping information. The static mesh includes a plurality of static mesh attributes, and the lower-level mapping information assigns a lower-level attribute identifier to each of the plurality of static mesh attributes. For example, a data structure such as `bmesh_sequence_parameter_set_rbsp` can be used to associate each lower-level attribute identifier (which may be an index `i`) with a corresponding static mesh attribute indicated by `bmsps_mesh_attribute_index[i]`. (The value of `i` can be implicit, for example, it can be inferred from the order of the values ​​`bmsps_mesh_attribute_index[i]` in the bitstream.) The decoder decodes the encoded static mesh, including decoding at least a first static mesh attribute, wherein at least the first static mesh attribute has an assigned first lower-level attribute identifier, and the first lower-level attribute identifier is associated with a higher-level attribute.

[0088] §5. Some embodiments operate to enable implicit or explicit attribute ordering. In some embodiments, the V3C extension includes references to attributes in the underlying mesh bitstream. The attributes described in the V3C extension have the same order in terms of attribute type as the attributes in the underlying mesh bitstream that are ordered in terms of attribute type. For example, in the V3C extension, attributes are described in the following order: ATTR_NORMAL, ATTR_REFLECTANCE, ATTR_TRANSPARENCY. In the underlying mesh bitstream, attributes are described in the following order: ATTR_NORMAL, ATTR_REFLECTANCE, ATTR_TRANSPARENCY. Such a V-DMC bitstream may not require signaling for mapping between attributes between different levels of the V-DMC hierarchy. This is because the Nth element in the V3C extension attribute array references the Nth element in the underlying mesh bitstream with the same attribute type, where the maximum value of N is bmsps_mesh_attribute_count. There may be a scenario where the V3C extension references fewer attributes than the number given by bmsps_mesh_attribute_count, but the attributes ordered in the V3C extension bitstream and the base mesh bitstream are the same in terms of attribute type.

[0089] §5.a. Some embodiments operate to signal explicit mapping of the V3C extension. Where the ordering of attributes differs between the V3C extension and the underlying mesh codec, the ID needs to be explicitly referenced at the V-DMC extension level (using an index as shown in Section 2.a or a UAI as shown in Section 3.a). In some embodiments, the syntax of vps_vdmc_extension can be modified by adding a line marked with a dagger (†), as follows:

[0090] In some examples, the semantics of vps_ext_bmesh_data_attribute_explicit_mapping_flag[j] can be as follows: vps_ext_bmesh_data_attribute_explicit_mapping_flag[j] specifies whether the mapping index of the attribute needs to reference the attribute in the underlying mesh codec of the atlas with altasID j.

[0091] In some examples, the semantics of vps_ext_bmesh_attribute_index can be as follows: vps_ext_bmesh_attribute_index specifies the index of the i-th attribute encoded in the underlying mesh bitstream for the i-th attribute type of the atlas with atlasID j.

[0092] Some implementations provide explicit mapping via UAI. In some embodiments, the syntax of vps_vdmc_extension can be modified by adding lines marked with a dagger (†), as follows:

[0093] In some examples, the semantics of vps_ext_bmesh_data_attribute_explicit_mapping_flag[j] can be as follows: vps_ext_bmesh_data_attribute_explicit_mapping_flag[j] specifies whether the mapping of the attribute UAI needs to reference the attribute in the underlying mesh codec of the atlas with altasID j.

[0094] In some examples, the semantics of vps_ext_bmesh_attribute_unique_id[j][i] can be as follows: vps_ext_bmesh_attribute_unique_id[j][i] specifies a unique attribute identifier for the i-th attribute encoded in the underlying mesh bitstream for the i-th attribute type in a atlas with atlasID j.

[0095] §5.b. Some embodiments operate to provide signaling for explicit mapping of attributes for the underlying lattice codec. Signaling can be used at the underlying lattice level to identify whether explicit mapping is provided for attributes in the lattice bitstream.

[0096] At the static mesh level, there can be more than one attribute with the same attribute type. Each attribute of the same type can correspond to different information. Each attribute item of the same attribute type is assigned an implicit id, which is the same as its sequential index stored in the attribute list. For example, the first attribute of each type can have an implicit id set to 0, the second attribute will have an id set to 1, and so on. In some embodiments, the syntax of bmesh_sequence_parameter_set_rbsp can be modified by adding a line marked with a dagger (†), as follows:

[0097] In some examples, the semantics of bmsps_mesh_attribute_explicit_mapping_flag can be as follows: bmsps_mesh_attribute_explicit_mapping_flag specifies whether the mapping index of the attribute needs to reference the attribute in the static mesh codec.

[0098] In some examples, the semantics of bmsps_mesh_attribute_index[i] can be as follows: bmsps_mesh_attribute_index[i] specifies the index of the i-th attribute in the array of attributes in the mesh for the i-th attribute type.

[0099] In some embodiments, the syntax of bmesh_sequence_parameter_set_rbsp can be modified by adding a line marked with a dagger (†), as follows:

[0100] In some examples, the semantics of bmsps_mesh_attribute_explicit_mapping_flag can be as follows: bmsps_mesh_attribute_explicit_mapping_flag specifies whether the mapping of attributes in the UAI needs to reference attributes in the static mesh codec.

[0101] In some examples, the semantics of bmsps_mesh_attribute_unique_id[i] can be as follows: bmsps_mesh_attribute_unique_id[i] specifies a unique identifier for the i-th attribute encoded in the mesh bitstream for the i-th attribute type.

[0102] §6. Some embodiments impose constraints on attribute mapping. In some such embodiments, attributes from the V3C extension are also explicitly mapped via the mechanisms listed in sections 2.a, 3.a, 4.a, and / or 5.a. Using mapping, attributes at the V3C extension refer to base mesh attributes that have the same or nearly similar semantics in terms of attribute type. For example, an attribute with vps_ext_bmesh_attribute_type_id: 0 = ATTR_TEXTURE can be mapped to a base mesh attribute with bmesh_attribute_type: 0 = ATTR_TEXTURE. Example embodiments provide constraints that prohibit mapping between attributes with different semantics between the V3C extension and the base mesh.

[0103] In the case of attribute mapping between a base mesh codec and a static mesh codec, some embodiments impose constraints such that the mapped attributes are equivalent. In some embodiments, the equivalence of attributes between the base mesh codec and the static mesh can be derived from the following rules: • Equivalent semantics, for example, the underlying mesh attribute type: MESH_ATTR_MATERIAL_ID in the underlying mesh bitstream is functionally equivalent to ATTR_MATERIAL_ID in the static mesh. • Equivalence based on the number of components: The number of components of a base mesh attribute can be equal to the number of components of a mapped attribute in a static mesh codec. For example, texture in a base mesh codec is a three-component attribute, which is equivalent to color in a static mesh codec.

[0104] • If MESH_ATTR_GENERIC is signaled in the static lattice bitstream, it is mapped to an equivalent attribute based on the number of components the MESH_ATTR_GENERIC attribute has. That is, a three-component ATTR_GENERIC attribute can be mapped to ATTR_TEXTURE in the underlying lattice bitstream.

[0105] In some embodiments, equivalence may also apply to non-MPEG static mesh codecs, such as Draco.

[0106] §7. Some embodiments provide a decoding process for the underlying lattice codec. In example embodiments, the consistent decoding process for the underlying lattice codec ensures that all lattice properties described in the syntax of the underlying lattice bitstream are decodeable and can be used for further reference.

[0107] In the example embodiment, attributes from the static grid referenced by the base grid bitstream are made available in the static grid bitstream. Grid attributes referenced by the base grid bitstream are decoded. Where the base grid bitstream references all grid attributes, all attributes of the static grid bitstream can be decoded.

[0108] In some embodiments, the decoder determines for at least one static mesh attribute whether the static mesh attribute maps to any higher-level attribute. If the static mesh attribute does not map to any higher-level attribute, the decoder may determine not to decode the attribute (or if it decodes the attribute, the decoder may discard the decoded values ​​instead of storing them in memory).

[0109] In some embodiments, the base grid bitstream may reference a subset of grid properties based on different mapping mechanisms as described in Parts 2, 3, 4 and 5.

[0110] In some embodiments, the base mesh decoder may instruct the static mesh codec to decode only a subset of the mesh attributes in the mesh attribute array, as described by the syntax in the base mesh bitstream. The referenced subset of decoded mesh attributes may then be made available in a storage buffer.

[0111] In some embodiments, an entry for each attribute in the mesh attributes is decoded and stored as an array-like structure in a storage buffer. The length of the decoded array can be equal to `mesh_attribute_count`. Where the base mesh decoder instructs the static mesh decoder to decode only a subset of the mesh attributes as described by the syntax in the base mesh bitstream, the decoded array can be equal to `bmsps_mesh_attribute_count`. In this case, the mesh attributes are decoded in the order they appear in the base mesh bitstream.

[0112] The example implementation addresses the issue of referencing and mapping attributes at different levels of the V-DMC hierarchy. Such an implementation enables bitstream consistency for sub-bitstreams and provides decoder consistency.

[0113] Example system hardware.

[0114] Example embodiments configured to implement the encoders and / or decoders (collectively, the encoders) described herein can use systems (such as...) Figure 10 This is achieved through a system (such as a .). Figure 10This is a block diagram illustrating examples of systems implementing various aspects and embodiments. System 1000 can be implemented as a device including the various components described below and configured to perform one or more aspects of the aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects of the aspects described in this document.

[0115] System 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing aspects such as those described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry as known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040 that may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0116] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents one or more modules that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Additionally, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated into processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0117] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the aspects described in this document may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing equations, formulas, operations, and operational logic.

[0118] In some embodiments, the memory within the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 1010 or encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory (such as RAM) is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding, a new standard developed by the Joint Video Experts Group JVET).

[0119] Inputs to the components of system 1000 can be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) component (COMP) input terminals (or a set of COMP input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Other examples include composite video.

[0120] In various embodiments, the input device of block 1130 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or down-converting it to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the components described above (and others), remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0121] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within the processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within the processor 1010. Demodulation, error correction, and demultiplexing streams are provided to various processing elements, including, for example, the processor 1010 and the encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0122] Various components of system 1000 can be provided within an integrated housing, in which various components can be interconnected and transmit data therebetween using a suitable connection arrangement 1140 (e.g., internal buses as known in the art, including inter-IC (I2C) buses, wiring and printed circuit boards).

[0123] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0124] In various embodiments, a wireless network such as Wi-Fi (e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)) is used to stream or otherwise provide data to system 1000. In these embodiments, Wi-Fi signals are received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 1000, delivering data via an HDMI connection to input block 1130. Still other embodiments use an RF connection to input block 1130 to provide streaming data to system 1000. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0125] System 1000 can provide output signals to various output devices, including display 1100, speaker 1110, and other peripheral devices 1120. Display 1100 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 1100 can be used in televisions, tablet computers, laptop computers, cellular phones (mobile phones), or other devices. Display 1100 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 1120 include one or more of a stand-alone digital video disc (or digital universal disc) (DVR, for both terms), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disk player performs the function of playing the output of system 1000.

[0126] In various embodiments, signaling (such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols enabling device-to-device control with or without user intervention) is used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120. Output devices can be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to system 1000 via communication interface 1050 using communication channel 1060. In electronic devices (such as, for example, televisions), display 1100 and speaker 1110 can be integrated into a single unit with other components of system 1000. In various embodiments, display interface 1070 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0127] For example, if the RF section of input 1130 is part of a separate set-top box, then display 1100 and speaker 1110 can alternatively be separated from one or more other components. In various embodiments where display 1100 and speaker 1110 are external components, output signals can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0128] The embodiments may be executed by computer software implemented by processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. As a non-limiting example, memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 1010 may be of any type suitable for the technical environment and may encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0129] Additional examples.

[0130] According to some embodiments, a method includes: obtaining an encoded static mesh, the encoded static mesh including a plurality of static mesh attributes; encoding a base mesh based on the encoded static mesh and motion information, the encoded base mesh including lower-level mapping information, the lower-level mapping information assigning lower-level attribute identifiers to each static mesh attribute in at least a subset of the static mesh attributes; and encoding a dynamic mesh based on the encoded base mesh and atlas information, the encoded dynamic mesh including higher-level mapping information, the higher-level mapping information assigning one of the corresponding lower-level attribute identifiers to each of a plurality of higher-level attributes.

[0131] In some embodiments, each of the static mesh attributes is identified by an index. In other embodiments, each of the static mesh attributes is identified by an arbitrary unique identifier.

[0132] In some embodiments, the lower-level attribute identifier is an index. In other embodiments, the lower-level attribute identifier is any unique identifier.

[0133] In some embodiments, the lower-level mapping information further identifies the attribute type of each of the static mesh attributes in a subset of the static mesh attributes. In some embodiments, the higher-level mapping information further identifies the attribute type associated with each of the higher-level attributes.

[0134] In some embodiments, the static mesh has a plurality of faces, and at least one of the static mesh properties is an attribute of each of the faces. In some embodiments, the static mesh has a plurality of vertices, and at least one of the static mesh properties is an attribute of each of the vertices.

[0135] According to some embodiments, a method includes: obtaining an encoded dynamic mesh, the encoded dynamic mesh including an encoded base mesh, atlas information, and higher-level mapping information, the higher-level mapping information assigning corresponding lower-level attribute identifiers to each of a plurality of higher-level attributes; decoding an encoded base mesh, the encoded base mesh including an encoded static mesh, motion information, and lower-level mapping information, wherein the static mesh includes a plurality of static mesh attributes, and wherein the lower-level mapping information assigns lower-level attribute identifiers to each of the plurality of static mesh attributes; and decoding an encoded static mesh, including decoding at least a first static mesh attribute, wherein the first static mesh attribute is mapped to at least one of the higher-level attributes through the lower-level mapping information and the higher-level mapping information.

[0136] In some embodiments, at least some (or all) static mesh properties that are not mapped to higher-level properties are not decoded, or their values ​​may be discarded.

[0137] Some embodiments include an apparatus comprising one or more processors configured to perform any of the methods described herein.

[0138] Some embodiments include at least one processor and a computer-readable medium storing instructions for performing any of the methods described herein.

[0139] Some embodiments include a computer-readable medium storing instructions for performing any of the methods described herein.

[0140] Some embodiments include a computer-readable medium storing a grid encoded according to any of the encoding methods described herein.

[0141] Some embodiments include signals that convey a mesh encoded according to any of the methods described herein.

[0142] Note that the various hardware elements in one or more of the described embodiments are referred to as “modules” that perform (i.e., execute, implement, and so on) the various functions described herein in connection with the respective modules. As used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more memory devices) that a person skilled in the art would consider suitable for a given implementation. Each described module may also include executable instructions for performing one or more functions described as being performed by the respective module, and note that those instructions may take the form of hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or such instructions, or include hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or such instructions, and may be stored in any suitable one or more non-transitory computer-readable media (such as commonly referred to as RAM, ROM, etc.).

[0143] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented as computer programs, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROMs and digital universal discs (DVDs)). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A grid coding method, comprising: Obtain an encoded static mesh, wherein the encoded static mesh includes multiple static mesh attributes; The base mesh is encoded based on the encoded static mesh and motion information. The encoded base mesh includes lower-level mapping information that associates lower-level attribute identifiers with each static mesh attribute in at least a subset of the static mesh attributes. as well as The dynamic mesh is encoded based on the base mesh and atlas information. The encoded dynamic mesh includes higher-level mapping information that associates each of a plurality of higher-level attributes with one of the corresponding lower-level attribute identifiers.

2. A mesh encoder device, comprising one or more processors, said one or more processors being configured to perform at least the following operations: Obtain an encoded static mesh, wherein the encoded static mesh includes multiple static mesh attributes; The base mesh is encoded based on coded static mesh and motion information, the encoded base mesh including lower-level mapping information that associates lower-level attribute identifiers with each static mesh attribute in at least a subset of static mesh attributes; and The dynamic mesh is encoded based on the base mesh and atlas information. The encoded dynamic mesh includes higher-level mapping information that associates each of a plurality of higher-level attributes with one of the corresponding lower-level attribute identifiers.

3. The method of claim 1 or the apparatus of claim 2, wherein each of the static mesh attributes is identified by an index.

4. The method according to claim 1 or claim 3 which is dependent on claim 1, or the apparatus according to claim 2 or claim 3 which is dependent on claim 2, wherein the lower-level attribute identifier is an index.

5. The method according to claim 1 or claims 3-4 which are dependent on claim 1, or the apparatus according to claim 2 or claims 3-4 which are dependent on claim 2, wherein the lower-level mapping information further identifies the attribute type of each of the static mesh attributes in a subset of the static mesh attributes.

6. The method according to claim 1 or claims 3-5 which are dependent on claim 1, or the apparatus according to claim 2 or claims 3-5 which are dependent on claim 2, wherein the higher-level mapping information further identifies the attribute type associated with each of the higher-level attributes.

7. A grid decoding method, comprising: Obtain an encoded dynamic mesh, the encoded dynamic mesh including an encoded base mesh, atlas information and higher-level mapping information, the higher-level mapping information associating corresponding lower-level attribute identifiers with each of a plurality of higher-level attributes; Decode the encoded base mesh, which includes an encoded static mesh, motion information, and lower-level mapping information, wherein the static mesh includes multiple static mesh attributes, and wherein the lower-level mapping information associates a lower-level attribute identifier with each of the multiple static mesh attributes; as well as Decoding the encoded static mesh includes decoding at least a first static mesh attribute, wherein the first static mesh attribute is mapped to at least one of the higher-level attributes through lower-level mapping information and higher-level mapping information.

8. A mesh decoder apparatus, comprising one or more processors, said one or more processors being configured to perform at least the following operations: Obtain an encoded dynamic mesh, the encoded dynamic mesh including an encoded base mesh, atlas information and higher-level mapping information, the higher-level mapping information associating corresponding lower-level attribute identifiers with each of a plurality of higher-level attributes; Decoding the encoded base mesh, which includes an encoded static mesh, motion information, and lower-level mapping information, wherein the static mesh includes multiple static mesh attributes, and wherein the lower-level mapping information associates a lower-level attribute identifier with each of the multiple static mesh attributes; and Decoding the encoded static mesh includes decoding at least a first static mesh attribute, wherein the first static mesh attribute is mapped to at least one of the higher-level attributes through lower-level mapping information and higher-level mapping information.

9. The method of claim 7 or the apparatus of claim 8, further comprising determining, for at least a second static mesh attribute, whether the second static mesh attribute maps to any higher-level attribute, and wherein, in response to determining that the second static mesh attribute does not map to any higher-level attribute, the second static mesh attribute is not decoded.

10. The method according to claim 7 or claim 9 which is a subset of claim 7, or the apparatus according to claim 8 or claim 9 which is a subset of claim 8, wherein each of the static grid properties is identified by an index.

11. The method according to claim 7 or claims 9-10 which are dependent on claim 7, or the apparatus according to claim 8 or claims 9-10 which are dependent on claim 8, wherein the lower-level attribute identifier is an index.

12. The method according to claim 7 or claims 9-11 which are dependent on claim 7, or the apparatus according to claim 8 or claims 9-11 which are dependent on claim 8, wherein the lower-level mapping information further identifies the attribute type of each of the static mesh attributes in a subset of the static mesh attributes.

13. The method according to claim 7 or claims 9-12 which are dependent on claim 7, or the apparatus according to claim 8 or claims 9-12 which are dependent on claim 8, wherein the higher-level mapping information further identifies the attribute type associated with each of the higher-level attributes.

14. The method according to claim 7 or any one of claims 9-13 dependent on claim 7, or the apparatus according to claim 8 or any one of claims 9-13 dependent on claim 8, wherein the static mesh has a plurality of faces, and wherein at least one of the static mesh properties is a property of each of the faces.

15. The method according to claim 7 or any one of claims 9-14 dependent on claim 7, or the apparatus according to claim 8 or any one of claims 9-14 dependent on claim 8, wherein the static mesh has a plurality of vertices, and wherein at least one of the static mesh properties is a property of each of the vertices.

Images