Signaling supplementary information related to attributes in v3c bitstream and basemesh bitstream

Abstract

Mesh encoding and decoding using custom attributes are proposed. In an example mesh encoding method, the following steps are performed: obtaining base mesh attribute information for at least one custom attribute; encoding each of the custom attributes in a respective atlas, each atlas having a respective atlas identifier; and signaling a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a respective label of the custom attribute and the respective atlas identifier.

Description

SIGNALING SUPPLEMENTARY INFORMATION RELATED TO ATTRIBUTES IN V3C BITSTREAM AND BASEMESH BITSTREAMCROSS-REFERENCE

[0001] This application claims the benefit of European Patent Application No. 24305101.8, filed 16 January 2024, the entire disclosure of which is incorporated herein by reference.BACKGROUND

[0002] Visual Volumetric Video-based Coding (V3C) standards are being developed for the coding of immersive scenes. A description of V3C is provided in ISO / IEC JTC 1 / SC29, "ISO / IEC 23090-5:2023, Coded representation of immersive media Part 5: Visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC)," November 2023. Within the V3C framework, a video-based dynamic mesh coding (V-DMC) is under development and is described in ISO / IEC 23090-29, S. 2. Secretariat, "WD 5.0 of V-DMC," 20 October 2023.

[0003] V3C provides for the encoding and decoding of attributes used in the immersive scene. Some such attributes are described in the standard materials, ISO / IEC 23090-5. However, a V3C encoder also has the ability to deliver an attribute or a set of attributes that does not correspond to the specific attribute types defined in ISO / IEC 23090-5. Conversely, a V3C decoder may receive a V3C bitstream that includes an attribute or a set of attributes that does not correspond to the attribute types defined in the ISO / IEC 23090-5.

[0004] Such attributes that are not defined in the standard materials may be tailored to specific needs of an application. Such attributes that are not specifically defined in ISO / IEC 23090-5 may be referred to as custom attributes. More specifically, such attributes that are not defined in ISO / IEC 23090-29 may be referred to as custom basemesh attributes.

[0005] Such custom attributes provide potential challenges for interoperability of a V3C bitstream to be signaled / decoded by a V3C encoder / decoder, since a conventional V3C encoder or decoder does not have the necessary information to correctly process the contents of the custom attributes. Even if the data of the custom attribute is encoded / decoded using compression schemes such as video codecs, the current codec does not provide sufficient signaling information to allow for the proper interpretation of the contents of the data for purposes such as reconstruction of the final volumetric asset.

[0006] ISO / IEC 23090-5 includes a list of attribute types. An attribute type is defined by ai_attribute_type_id syntax element. An implementation adds a custom attribute in the V3Cbitstream. The way to signal a custom attribute type in a V3C bitstream is with attribute with attribute type: ATTRJJNSPECIFIED. However, the information on what the ATTRJJNSPECIFIED attribute contains is unknown to a third party which receives this information. There is no mechanism to signal from a V3C encoder, what information is stored in the custom attribute with attribute type ATTRJJNSPECIFIED.

[0007] Furthermore, since V-DMC, ISO / IEC 23090-29, inherits the fundamentals and derives from ISO / IEC 23090-5. A V-DMC encoder / decoder may deliver / receive an attribute or a set of attributes that do not correspond to the attribute types defined in ISO / IEC 23090-29. ISO / IEC 23090-29 introduces a component called basemesh bitstream. A basemesh bitstream may also store attributes. These basemesh attributes provide additional information about the basemesh component. This information may be useful for processing and reconstruction of basemesh. A list of basemesh attributes is defined in the specification ISO / IEC 23090-29. However, it may be desirable for a V-DMC application to code a custom basemesh attribute. This may be signaled by vps_ext_bmesh_attributejype: ATTRJJNSPECIFIED in V3C V- DMC extension or bmsps_mesh_attributejypejd: ATTRJJNSPECIFIED in the basemesh bitstream.

[0008] In ISO / IEC 23090-5, for predefined attributes, the bitstream may indicate that the attribute consists of several dimensions (a number of channels). The syntax element ai_attribute_dimension_minus1 indicates the total number of dimensions of the attribute. Each attribute dimension may store different information. For example, an attribute with attribute type:ATTR_NORMAL may consist of three dimensions as described in sub-clause 8.4.4.5 in ISO / IEC 23090-5. Each dimension stores one of the components of the unit vector(x,y,z).

[0009] A V3C encoder / decoder may signal / decode a custom attribute with a particular dimension. However, for a V3C encoder there is no means to signal the dimensions of the custom attribute. Similarly, for a V3C decoder there is no means to interpret the decoded data of a custom attribute without the semantic information.SUMMARY

[0010] A mesh encoding method according to some embodiments comprises: selecting a data type for a basemesh attribute; encoding the basemesh attribute using the selected data type; and signaling, in a sequence parameter set, a syntax element having a value indicating the selected data type.

[0011] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type isunspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled in the sequence parameter set having a value indicating a data type of the respective basemesh attribute.

[0012] A mesh encoding method according to some embodiments comprises: selecting a data type for a basemesh attribute; encoding the basemesh attribute using the selected data type; and signaling, at a V3C level, a syntax element having a value indicating the selected data type.

[0013] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled at the V3C level having a value indicating a data type of the respective basemesh attribute.

[0014] A mesh encoding method according to some embodiments comprises: obtaining basemesh attribute information having a number of one or more sub-components; encoding the plurality of sub-components of the basemesh attribute; and signaling, in a sequence parameter set, a syntax element having a value indicating the number of sub-components of the basemesh attribute.

[0015] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled in the sequence parameter set having a value indicating the number of sub-components of the respective basemesh attribute.

[0016] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0017] A mesh encoding method according to some embodiments comprises: obtaining basemesh attribute information having a number of one or more sub-components; encoding the plurality of sub-components of the basemesh attribute; and signaling, in a V3C parameter set V-DMC extension, a syntax element having a value indicating the number of subcomponents of the basemesh attribute.

[0018] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemeshattributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled in the V3C parameter set V-DMC extension having a value indicating the number of sub-components of the respective basemesh attribute.

[0019] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0020] A mesh encoding method according to some embodiments comprises: obtaining basemesh attribute information for at least one custom attribute; encoding each of the custom attributes in a respective atlas, each atlas having a respective atlas identifier; and signaling a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a label of the custom attribute and the respective atlas identifier.

[0021] In some embodiments, the SEI message further includes, for each custom attribute, information indicating whether a label is provided for each of a plurality of dimensions of the custom attribute.

[0022] In some embodiments, for at least one of the custom attributes, the information indicating whether a label is provided for each of a plurality of dimensions indicates that a label is provided for each of a plurality of dimensions, and wherein the SEI message further includes a dimension label for each of the plurality of dimensions.

[0023] A mesh decoding method according to some embodiments comprises: reading, from a sequence parameter set, a syntax element having a value indicating a selected data type for a basemesh attribute; and decoding the basemesh attribute using the indicated data type.

[0024] Some embodiments further comprise decoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read from in the sequence parameter set having a value indicating a data type of the respective basemesh attribute.

[0025] A mesh decoding method according to some embodiments comprises: reading, at a V3C level, a syntax element having a value indicating a selected data type for a basemesh attribute; and decoding the basemesh attribute using the indicated data type.

[0026] Some embodiments further comprise decoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type isunspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read at the V3C level having a value indicating a data type of the respective basemesh attribute.

[0027] A mesh decoding method according to some embodiments comprises: reading, from a sequence parameter set, a syntax element having a value indicating a number of subcomponents of a basemesh attribute; and decoding the plurality of sub-components of the basemesh attribute.

[0028] A method according to some embodiments further comprises reading, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read from the sequence parameter set having a value indicating the number of sub-components of the respective basemesh attribute.

[0029] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0030] A mesh decoding method according to some embodiments comprises reading, from a V3C parameter set V-DMC extension, a syntax element having a value indicating a number of sub-components of the basemesh attribute; and decoding the plurality of sub-components of the basemesh attribute.

[0031] Some embodiments further comprise reading, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read from the V3C parameter set V-DMC extension having a value indicating the number of sub-components of the respective basemesh attribute.

[0032] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0033] A mesh decoding method according to some embodiments comprises: reading a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a label of the custom attribute and an atlas identifier associated with the custom attribute; and decoding each of the custom attributes from the respective atlas.

[0034] In some embodiments, the SEI message further includes, for each custom attribute, information indicating whether a label is provided for each of a plurality of dimensions of the custom attribute.

[0035] In some embodiments, for at least one of the custom attributes, the information indicating whether a label is provided for each of a plurality of dimensions indicates that a label is provided for each of a plurality of dimensions, and wherein the SEI message further includes a dimension label for each of the plurality of dimensions.

[0036] An apparatus according to some embodiments comprises one or more processors, the apparatus being configured to perform any of the methods described herein.

[0037] An apparatus according to some embodiments comprises at least one processor and a computer-readable medium storing instructions for performing any of the methods described herein.

[0038] A computer-readable medium according to some embodiments store instructions for performing any of the methods described herein.

[0039] A computer-readable medium according to some embodiments stores a mesh encoded according to any of the methods described herein.

[0040] A signal according to some embodiments conveys a mesh encoded according to any of the methods described herein.

[0041] A computer program product according to some embodiments includes instructions which, when the program is executed by one or more processors, cause the one or more processors to carry out any of the methods described herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The following detailed description will be better understood when read in conjunction with the appended drawings, in which there are shown examples of one or more of the multiple embodiments of the present disclosure. It should be understood, however, that the embodiments described herein are not limited to the precise arrangements and instrumentalities shown in the drawings.

[0043] FIG. 1 is a functional block diagram of an example mesh encoding system using intra frame encoding.

[0044] FIG. 2 is a functional block diagram of an example mesh decoding system using intra frame decoding.

[0045] FIG. 3 schematically illustrates encapsulation of a different sub-bitstreams in a bitstream encoding a dynamic mesh such as a V-DMC bitstream.

[0046] FIG. 4 is a block diagram of an example of a system in which various aspects and embodiments are implemented.DETAILED DESCRIPTIONOverview of dynamic mesh coding

[0047] FIG. 1 is a schematic block diagram of a mesh encoding process that may be employed in some embodiments. A source mesh model 302 is provided as an input mesh M(i) to the mesh encoding process. The source mesh model 302 is associated with a source texture map 304 that is proved as an input texture map A(i) to the encoding process. The input mesh is decimated at 306 to generate a base mesh m(i) with a reduced number of vertices, and a UV atlas is generated for the base mesh AT 308. The base mesh is quantized at 310 and encoded at 312, with the compressed base mesh data being multiplexed at 314 into a dynamic mesh bitstream. The compressed base mesh data is reconstructed at the encoder to generate reconstructed base mesh m’(i) with a static mesh decoder 316. The reconstructed base mesh is subdivided at 318 by adding new vertices. A subdivision surface fitting process is performed at 320 by comparing the subdivided base mesh with the input mesh M(i) to determine a set of displacements d(i) that deform the vertices of the subdivided base mesh to correspond more closely to the surfaces defined by the input mesh M(i). These displacements may be updated at 322 into updated displacements d’(i) based on difference between the original base mesh m(i) and the reconstructed base mesh m’(i). These updated displacements are encoded using a wavelet transform 324 that generates wavelet coefficients e’(i), which are quantized at 326 and packed at 328 into an image format. A time-varying series of images representing the wavelet coefficients may be encoded at 330 using conventional video encoding techniques, and the encoded video may be multiplexed at 314 with the data representing the compressed base mesh. At the encoder, the displacements are reconstructed from the encoded video through image unpacking 329, inverse quantization 331 , and inverse wavelet transform 332 to generate a reconstructed set of displacements d”(i). A reconstructed base mesh M”(i) is obtained through inverse quantization at 334 of the reconstructed quantized base mesh m’(i), and the reconstructed base mesh M”(i) is subdivided at 336. A reconstructed deformed mesh DM(i) is generated at 338 by applying the reconstructed set of displacements d”(i) to the reconstructed base mesh m’(i). The reconstructed deformed mesh DM(i) is used as a destination mesh model 341 for the purpose of attribute transfer 340.

[0048] Using the reconstructed deformed mesh DM(i) (destination mesh model 341), the input mesh M(i) (source mesh model 302), and the input texture map A(i) (source texture map 304), an attribute transfer process is performed to provide attribute values for a destination texture map A’(i) that is associated with the reconstructed deformed mesh DM(i). Pixels in the texture map A’(i) that are not associated with any triangle of the reconstructed deformed mesh DM(i) may be filled using a padding process 342. A color space conversion 344 may be performed, a time-varying series of texture maps A’(i) may be encoded using conventional video encoding techniques 346, and the encoded video may be multiplexed at 314 into a bitstream 350 with the data representing the displacements and the compressed base mesh. Patch information 348 may also be multiplexed in the bitstream.

[0049] FIG. 2 illustrates a mesh decoding method that may be performed in some embodiments. A compressed bitstream b(i) is demultiplexed into data representing patch information, data representing a static mesh, video data representing mesh displacements, and video data representing attributes. The data representing a static mesh is decoded by a static mesh decoder into a reconstructed quantized base mesh m’(i) and inverse quantized, resulting in a decoded base mesh m"(i). The video data representing mesh displacements is decoded. The decoded image is unpacked and inverse quantized. An inverse wavelet transform is applied, resulting in decoded displacements d”(i). The deformed mesh is reconstructed using the decoded base mesh m"(i) and the decoded displacements d”(i), resulting in a decoded mesh M"(i). The video data representing attributes is decoded, and color format / space conversion is applied, resulting in a decoded attribute map A"(i).

[0050] FIGs. 1 and 2 illustrate examples of intra mesh encoding and decoding. It should be noted that example embodiments described herein may also be implemented in the case of inter mesh encoding and decoding.

[0051] For V-DMC, a framework is developed by extending V3C. V3C is described at ISO / IEC 23090-5:2021 , Information technology — Coded representation of immersive media — Part 5: Visual volumetric video-based coding (V3C) and video-based point cloud compression (V- PCC).

[0052] In MPEG, the V-DMC standard has started to work on static mesh codec. The static mesh codec is specified in Annex-I of ISO / IEC 23090-29. The static mesh codec in Annex I is the default codec to compress static mesh. This includes mesh geometry, mesh connectivity and mesh attributes. The mesh attributes provide additional information about the mesh vertices or mesh faces.

[0053] The V-DMC framework may involve different attributes specified for a dynamic mesh sequence. In V-DMC, the underlying static mesh codec may decode mesh attributes per faceor per vertex. These attributes may provide additional information about the static mesh e.g., color, texture coordinates, normals, reflectance information, transparency information, and / or user-defined attributes.

[0054] The attributes from the static mesh codec may further be used by the basemesh codec. A basemesh bitstream is further encapsulated in a V3C bitstream. The hierarchical structure of a V-DMC bitstream is shown in FIG. 3. An extension mechanism in V3C bitstream may provide information about the type of attributes that a V3C bitstream / framework will use from the underlying basemesh codec. Additionally, the basemesh bitstream may provide a mechanism to provide information about the type of mesh attribute of interest from the underlying static mesh codec.Issues addressed in some embodiments

[0055] One issue addressed in some embodiments is that the current specification of ISO / IEC 23090-29 does not provide the ability to signal additional information for a custom basemesh attribute such as the number of components and data type for the components of a basemesh attribute.

[0056] It may be desirable for a V3C encoder / packager to deliver custom attribute(s) within V3C bitstream. The custom attribute(s) needs to be signaled in the bitstream. Within the ISO / IEC 23090-5 specification, a custom attribute can be signaled using an unspecified attribute type, e.g. ATTRJJNSPECIFIED. However, the ISO / IEC 23090-5 specification does not provide a mechanism for providing information regarding the properties of an attribute coded using the attribute type of ATTRJJNSPECIFIED. Example embodiments provide a mechanism for identifying the content stored in the attribute with attribute type ATTRJJNSPECIFIED with a standardized mechanism.Overview of example embodiments

[0057] In an example embodiment, a V3C encoder may signal (and a decoder may decode) additional information for the basemesh attribute. The information may include such information as the number of components and / or the data type of the components of the basemesh attribute.

[0058] In some embodiments, an encoder signals (and a decoder decodes) a supplemental enhancement information (SEI) message to embed / retrieve information about a custom attribute(s) used by an implementation.

[0059] Example embodiments may be used to provide additional information about a basemesh attribute. The current specification ISO / IEC 23090-29 does not provide a mechanism for identifying additional information about the basemesh attribute; instead,basemesh attribute components are signaled / decoded with a specific data type. In example embodiments, a basemesh attribute component type may be present in the V3C bitstream to provide information about the data type of the basemesh attribute component as shown below.

[0060] Furthermore, the number of components for a basemesh attribute may have different values depending on the basemesh attribute type. In example embodiments, the dimension of a basemesh attribute may be present in the V3C bitstream to provide information about the number of components of the basemesh attribute as shown in below.

[0061] Some embodiments provide a basemesh attribute component type. The subcomponents of the basemesh attribute may be coded using a specified data type. The basemesh encoder / decoder signals / decodes the basemesh attribute components with appropriate information regarding the data type for the basemesh attribute. A data type of the basemesh attribute may remain consistent across the number of sub-components / dimensions of the basemesh attribute. For example, a data type of the basemesh attribute may be FLOAT or UNSIGNED INTEGER for the all the sub-components / dimensions of the basemesh attribute.

[0062] An example basemesh encoder / decoder signals / decodes a syntax element which provides information on the data type for the component(s) of a basemesh attribute. In some cases, a syntax element (bmsps_mesh_attribute_dimension_data_type) may be signaled within the basemesh sequence parameter set to provide a value which identifies the data type for the components of a basemesh attribute within a basemesh sequence. In some cases, a syntax element (e.g. v3c_ext_bmesh_attribute_dimension_data_type) may be signaled at the V3C level to provide a value which identifies the data type for the components of a basemesh attribute for the entire V3C bitstream.

[0063] In some embodiments, the value of the syntax element (e.g. v3c_ext_bmesh_attribute_dimens ion_data_type / bmsps_mesh_attribute_dimens ion_data_type) is mapped to a Standardized mechanism specified outside of the ISO / IEC 23090-29. In some cases, the value of the syntax element may be mapped to a data type identifier. In some cases, the values of the syntax element follow a mapping as shown in the table below.

[0064] Some embodiments provide for signaling of basemesh attribute dimensions. A basemesh attribute may have several dimensions. The dimensions of a basemesh attribute specificy the number of sub-components that constitute the basemesh attribute. For example, a NORMAL is represented by three components (x,y,z). Therefore, the value of dimensions of a basemesh attribute of basemesh attribute type NORMAL will be three.

[0065] In an example embodiment, a V3C encoder / decoder may signal / decode a syntax element which provides the value for the dimension of a basemesh attribute. Such a syntax element can be present in the V3C bitstream at different level. The value provided by the syntax element is equal to the value of the number of components in the static mesh bitstream. In ISO / IEC 23090-29, the static mesh codec specifies the dimension of a mesh attribute with mesh_attribute_num_components_minus1.

[0066] In some embodiments, the dimensions of the basemesh attribute may remain consistent within a sequence of the basemesh bitstream. This allows for a flexible mechanism to signal a different value for the dimension of the basemesh attribute over the sequences in a basemesh bitstream. For example, in an example embodiment, a syntax element bmsps_attribute_dimension_minus1 is present in the sequence parameter set of the basemesh bitstream as shown in the syntax table below. A V3C encoder / decoder signals / decodes the value for the syntax element bmsps_attribute_dimension_minus1 to determine the number of dimensions for a basemesh attribute.

[0067] In some embodiments, the dimensions of the basemesh attribute may not change for the entire basemesh bitstream. Signaling the dimension of the basemesh attribute in every sequence is redundant and may cause the bitrate to increase. Therefore, a V3C encoder / decoder in some embodiments may signal / decode a syntax element at the V3C level. For example, a syntax element vps_ext_bmesh_attribute_dimension_minus1 may be present in the V3C parameter set V-DMC extension of the V3C bitstream as shown in the syntax table below. A V3C encoder / decoder signals / decodes the value of the syntax element vps_ext_bmesh_attribute_dimension_minus1 to determine the number of dimensions for a basemesh attribute.

[0068] Some embodiments provide dimensions for a basemesh attribute with an attribute type “ATTRJJNSPECIFIED.” In some embodiments, for a defined basemesh attribute, the specification may provide a default value for the dimension of the basemesh attribute. A 3C decoder decoding a basemesh attribute may use the default value of the dimension of the basemesh attribute to determine the number of sub-components of the basemesh attribute. However, the specification allows for a V3C encoder / decoder to signal / decode a generic or unspecified attribute with basemesh attribute type ATTRJJNSPECIFIED. In order to provide additional information about the dimensions of the generic attribute, in some embodiments, a basemesh encoder / decoder may signal / decode a syntax element which determines the dimension of the basemesh attribute with basemesh attribute type as ATTRJJNSPECIFIED. This syntax element, for example bmsps_attribute_dimension_minus1 , may be present in the basemesh bitstream at the sequence parameter set only when basemesh attribute type is ATTRJJNSPECIFIED. Such mechanism allows a V3C encoder / decoder to flexiblysignal / decode a change in the dimensions of the basemesh attribute with basemesh attribute type ATTRJJNSPECIFIED.

[0069] In some embodiments, a syntax element such as vps_ext_bmesh_attribute_dimension_minus1 is present at the V3C level for basemesh attribute with basemesh attribute type ATTRJJNSPECIFIED. Such mechanism allows a V3C encoder / decoder to signal / decode a value for the dimensions of a basemesh attribute with basemesh attribute type ATTRJJNSPECIFIED for the entire basemesh bitstream.

[0070] Some embodiments provide an SEI message with custom attribute information. A V3C encoder / decoder may signal / decode a custom attribute. A custom attribute is indicated withattribute type as ATTR_UNSPECIFIED. A V3C encoder / decoder signals the custom attribute with attribute type: ATTRJJNSPECIFIED.

[0071] A 3C encoder / decoder signals / decodes a Supplementary Essential Information (SEI) message within the bitstream. The SEI message provides information about the custom attribute such as contents of the attribute / The custom attribute SEI message may be called“Custom Attribute Information” SEI message.

[0072] The syntax of the custom attribute information SEI message in an example embodiment is shown in Table 1 .Table 1. Custom attribute information syntax table.

[0073] A V3C encoder / decoder according to some embodiments signals information for a list of custom attributes in the custom attribute information (CAI) SEI message, such as that of Table 1. In an example embodiment, the total number of custom attributes in Attribute Video Data unit of 3C bitstream is signaled via a syntax element, e.g. cai_attr_count_minus l, in the CAI SEI message. The total number of custom attributes, indicated for example bycai_attr_count_minus l, corresponds to the total of attributes across different atlases in V3C bitstream with attribute type: ATTRJJNSPECIFIED.

[0074] In an example embodiment, each attribute is associated with atlas within a V3C bitstream. For each custom attribute, the V3C encoder / decoder signals / identifies the atlas associated with the attribute, cai_attr_atlas_id indicates the identifier of the atlas which is associated with the attribute.

[0075] The index of the V3C attribute is used as reference to map the V3C attribute to the information related to custom attribute. cai_attr_index

[0001] indicates the index of the attribute data with attribute type ATTRJJNSPECIFIED carried in the Attribute Video Data unit.

[0076] In an example embodiment, each custom attribute is described with a label. The label may be provided via an ASN.1 object identifier value in the international object identifier tree. In such embodiments, the international object identifier may be assigned by a registration authority in accordance with Rec. ITU T X.660 | ISO / IEC 9834 1 or generated without registration using a universally unique identifier (UUID) as specified by Rec. ITU T X.667 | ISO / IEC 98348. The ASN.1 object identifier value of the label for a custom attribute is signaled with a syntax element such as cai_attr_label as shown in Table 1. The label may also be provided using a URI instead of an ASN.1 object identifier value. A null-terminated string encoded as UTF-8 characters as specified in ISO / IEC 10646 is used to store the value for the syntax element cai attr iabei. A label may describe the contents of the custom attribute stored in the V3C bitstream. The semantics of the label may be used to inform a V3C decoder how to interpret the information stored in the custom attribute. The semantics of the attribute may be sufficient to describe the contents for each of the dimensions of the custom attribute. In case of a video-based attribute, a label for the custom attribute can describe the contents for each of the channel (or dimensions) in the attribute video bitstream.

[0077] In some embodiments, a V3C encoder may also signal information for each dimension of the custom attribute. This may be performed when a custom attribute label is generic and calls for additional information to describe each dimension of the custom attributes. The dimensions may be described with ai_attr_dimens ion_minus i as shown above.

[0078] In an example embodiment, a syntax element describes whether per dimension information is available for a custom attribute via cai_attr_dimens ion_iabei_present. When the value of cai_attr_dimens ion_iabei_present is 1 then object identifier value for each dimension of the custom attribute is available in the CAI SEI message. Otherwise, no additional information per dimension is available and in which the information for each dimension is to be inferred based on the label of the custom attribute in cal attr label.

[0079] In an example embodiment, when cai_attr_dimens ion_label_present has value 1 , then for each dimension of the custom attribute, a V3C encoder / decoder signals / decodes an ASN.1 object identifier value which provides the semantics to the data stored in the different dimensions / channels of the attribute. A syntax element cai_attr_dimension_iabei specifies an ASN.1 object identifier value in the international object identifier tree. The international object identifier may be assigned by a registration authority in accordance with Rec. ITU T X.660 | ISO / IEC 9834 1 or may be generated without registration using a universally unique identifier (UUID) as specified by Rec. ITU T X.667 | ISO / IEC 9834 8, which the V3C encoder / decoder can use to provide semantics to the data in each dimension of the custom attribute. The label may also be provided using a URI instead of an ASN.1 object identifier value. A null-terminated string encoded as UTF-8 characters as specified in ISO / IEC 10646 may be used to store the value for the syntax element cai_attr_dimension_label .

[0080] In an example embodiment, a V3C bitstream may also contain basemesh components for a V-DMC application. A basemesh bitstream may further store a list of attributes. Some of the basemesh attributes are custom attributes for a V-DMC application. In one embodiment, to indicate the basemesh attributes with basemesh attribute type: ATTRJJNSPECIFIED, the Custom Attribute Information SEI message as described above can be used with some additions, as shown in Table 2.Table 2. CAI SEI message with Basemesh attribute information.

[0081] In another embodiment a new SEI message is specified for basemesh bitstream called “Custom Basemesh Attribute Information (CBAI).” A CBAI SEI message may be used to provide information about custom basemesh attribute, such as a basemesh attribute with basemesh attribute type ATTRJJNSPECIFIED.Table 3. Syntax for Custom Basemesh Attribute Information SEI message.

[0082] In an example embodiment, a V3C encoder / decoder or basemesh encoder / decoder signals information for a list of custom attributes with the Custom Basemesh Attribute Information (CBAI) SEI message. The total number of custom basemesh attributes in basemesh component of V3C bitstream is signaled via a syntax element, e.g., cbai_bmesh_attr_count_minus l, in the CBAI SEI message. The total number of custom basemesh attributes, which may be indicated by cbai_bmesh_attr_count_minus l, corresponds to the total of basemesh attributes across different atlases in basemesh bitstream with basemesh attribute type: ATTRJJNSPECIFIED.

[0083] In an example embodiment, each basemesh attribute is associated with an atlas within a V3C bitstream. For each custom basemesh attribute, the basemesh encoder / decoder signals / identifies the V3C atlas associated with the attribute. cbai_bmesh_attr_atias_id indicates the identifier of the V3C atlas which is associated with the basemesh attribute.

[0084] The index of the basemesh attribute may be used as a reference to map the basemesh attribute to the information related to a custom basemesh attribute. cbai_bmesh_attr_index [ i ] indicates the index of the basemesh attribute data with basemesh attribute type ATTRJJNSPECIFIED carried in the basemesh unit of V3C bitstream.

[0085] Each custom basemesh attribute is described with a label. The label may be provided for example via an ASN.1 object identifier value in the international object identifier tree. The international object identifier may be assigned by a registration authority in accordance with Rec. ITU T X.660 | ISO / IEC 9834 1 or may be generated without registration using a universally unique identifier (UUID) as specified by Rec. ITU T X.667 | ISO / IEC 9834-8. The ASN.1 object identifier value of the label for a custom basemesh attribute is signaled with a syntax element such as cbai_bmesh_attr_iabei as shown in Table 2 and Table 3. The label may also be provided using a URI instead of an ASN.1 object identifier value. A null- terminated string encoded as UTF-8 characters as specified in ISO / IEC 10646 is used to store the value for the syntax element cbai_bmesh_attrjabel. A label may describe the contents of the custom basemesh attribute stored in the basemesh bitstream. The semantics of the label can be useful to inform a basemesh decoder how to interpret the information stored in the custom basemesh attribute. The semantics of the basemesh attribute may be explicit to describe the contents for each of the dimensions of the custom basemesh attribute. Thesemantics provided by the label for the custom attribute can describe the contents for each of the basemesh attribute components (or dimensions) in the basemesh bitstream.

[0086] In some embodiments, a basemesh encoder may also signal information for each dimension of the custom basemesh attribute. This may be used when a custom basemesh attribute label is generic and calls for additional information to describe each dimension of the custom basemesh attributes. The dimensions can be described with a syntax element such as vps_ext_bmesh_attribute_dimension_minus 1 or bmsps_attribute_dimens ion_minus l as Shown herein.

[0087] In some embodiments, a syntax element describes whether per dimension information is available for a custom attribute via cbai_bmesh_attr_dimens ion_label_present. When the value Of cbai_bmesh_attr_dimension_label_present is 1 then Object identifier value for each dimension of the custom basemesh attribute is available in the CBAI SEI message. Otherwise, no additional information per dimension is available and the information for each dimension is to be inferred based on the label of the custom basemesh attribute in cbai_bmesh_attr_label.

[0088] In some embodiments, when cbai_bmesh_attr_dimens ion_label_present has value 1 , then for each dimension of the custom basemesh attribute, a basemesh encoder / decoder signals / decodes a ASN.1 object identifier value which provides the semantics to the data stored in the different dimensions / channels of the custom basemesh attribute. A syntax element cbai_bmesh_attr_dimens ion_iabei specifies an ASN.1 object identifier value in the international object identifier tree. The international object identifier may be assigned by a registration authority in accordance with Rec. ITU T X.660 | ISO / IEC 9834 1 or may be generated without registration using a universally unique identifier (UUID) as specified by Rec. ITU T X.667 | ISO / IEC 9834-8 which the basemesh encoder / decoder can use to provide semantics to the data in each dimension of the custom basemesh attribute. The label may also be provided using a URI instead of an ASN.1 object identifier value. A null- terminated string encoded as UTF-8 characters as specified in ISO / IEC 10646 is used to store the value for the syntax element cbai_bmesh_attr_dimension_label.

[0089] In example embodiments, a V3C encoder signals the number of components for a basemesh attribute to provide additional information related to the basemesh attribute. A 3C encoder may also signal a value to determine the data type for a basemesh attribute. The identification of the data type for a basemesh attribute can help the basemesh decoder to determine the exact format of basemesh attribute component(s).

[0090] In example embodiments, a V3C encoder signals a custom attribute information SEI message to provide additional information related to the custom attribute(s) indicated byattribute type: ATTRJJNSPECIFIED. A V3C decoder parses a custom attribute information SEI message to interpret the information carried in the custom attribute(s) indicated by attribute type: ATTRJJNSPECIFIED.

[0091] An encoder / decoder may use the CAI SEI message to signal / parse information related to custom attribute with attribute type: ATTRJJNSPECIFIED.Example system hardware.

[0092] Example embodiments of encoders and / or decoders (collectively coders) configured to implement embodiments described herein may be implemented using systems such as the system of FIG. 4. FIG. 4 is a block diagram of an example of a system in which various aspects and embodiments are implemented. System 1000 can be embodied as a device including the various components described below and is configured to perform one or more of the aspects described in this document. Examples of such devices, include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, the system 1000 is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more of the aspects described in this document.

[0093] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, the various aspects described in this document. Processor 1010 can include embedded memory, input output interface, and various other circuitries as known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device, and / or a non-volatile memory device). System 1000 includes a storage device 1040, which can include non-volatile memory and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and / or optical disk drive. The storage device 1040 can include an internal storage device, an attached storage device (includingdetachable and non-detachable storage devices), and / or a network accessible storage device, as non-limiting examples.

[0094] System 1000 includes an encoder / decoder module 1030 configured, for example, to process data to provide an encoded mesh or decoded mesh, and the encoder / decoder module 1030 can include its own processor and memory. The encoder / decoder module 1030 represents module(s) that can be included in a device to perform the encoding and / or decoding functions. As is known, a device can include one or both of the encoding and decoding modules. Additionally, encoder / decoder module 1030 can be implemented as a separate element of system 1000 or can be incorporated within processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0095] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to perform the various aspects described in this document can be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. In accordance with various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 can store one or more of various items during the performance of the processes described in this document. Such stored items can include, but are not limited to, the input mesh, the decoded mesh or portions of the decoded mesh, the bitstream, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0096] In some embodiments, memory inside of the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and to provide working memory for processing that is needed during encoding or decoding. In other embodiments, however, a memory external to the processing device (for example, the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory can be the memory 1020 and / or the storage device 1040, for example, a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of, for example, a television. In at least one embodiment, a fast external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations, such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or WC (Versatile Video Coding, a new standard being developed by JVET, the Joint Video Experts Team).

[0097] The input to the elements of system 1000 can be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a Component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples include composite video.

[0098] In various embodiments, the input devices of block 1130 have associated respective input processing elements as known in the art. For example, the RF portion can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF portion of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, down converters, demodulators, error correctors, and demultiplexers. The RF portion can include a tuner that performs various of these functions, including, for example, downconverting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. In one set-top box embodiment, the RF portion and its associated input processing element receives an RF signal transmitted over a wired (for example, cable) medium, and performs frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements performing similar or different functions. Adding elements can include inserting elements in between existing elements, such as, for example, inserting amplifiers and an analog-to-digital converter. In various embodiments, the RF portion includes an antenna.

[0099] Additionally, the USB and / or HDMI terminals can include respective interface processors for connecting system 1000 to other electronic devices across USB and / or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed-Solomon error correction, can be implemented, for example, within a separate input processing IC or within processor 1010 as necessary. Similarly, aspects of USB or HDMI interface processing can be implemented within separate interface ICs or within processor 1010 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 1010, and encoder / decoder1030 operating in combination with the memory and storage elements to process the datastream as necessary for presentation on an output device.

[0100] Various elements of system 1000 can be provided within an integrated housing, Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangement 1140, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0101] The system 1000 includes communication interface 1050 that enables communication with other devices via communication channel 1060. The communication interface 1050 can include, but is not limited to, a transceiver configured to transmit and to receive data over communication channel 1060. The communication interface 1050 can include, but is not limited to, a modem or network card and the communication channel 1060 can be implemented, for example, within a wired and / or a wireless medium.

[0102] Data is streamed, or otherwise provided, to the system 1000, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal of these embodiments is received over the communications channel 1060 and the communications interface 1050 which are adapted for Wi-Fi communications. The communications channel 1060 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over- the-top communications. Other embodiments provide streamed data to the system 1000 using a set-top box that delivers the data over the HDMI connection of the input block 1130. Still other embodiments provide streamed data to the system 1000 using the RF connection of the input block 1130. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network.

[0103] The system 1000 can provide an output signal to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes one or more of, for example, a touchscreen display, an organic lightemitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or other device. The display 1100 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop). The other peripheral devices 1120 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide afunction based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0104] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communications protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to system 1000 using the communications channel 1060 via the communications interface 1050. The display 1100 and speakers 1110 can be integrated in a single unit with the other components of system 1000 in an electronic device such as, for example, a television. In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0105] The display 1100 and speaker 1110 can alternatively be separate from one or more of the other components, for example, if the RF portion of input 1130 is part of a separate set- top box. In various embodiments in which the display 1100 and speakers 1110 are external components, the output signal can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0106] The embodiments can be carried out by computer software implemented by the processor 1010 or by hardware, or by a combination of hardware and software. As a nonlimiting example, the embodiments can be implemented by one or more integrated circuits. The memory 1020 can be of any type appropriate to the technical environment and can be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 1010 can be of any type appropriate to the technical environment, and can encompass one or more of microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting examples.Additional embodiments.

[0107] A mesh encoding method according to some embodiments comprises: selecting a data type for a basemesh attribute; encoding the basemesh attribute using the selected data type; and signaling, in a sequence parameter set, a syntax element having a value indicating the selected data type.

[0108] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemeshattributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled in the sequence parameter set having a value indicating a data type of the respective basemesh attribute.

[0109] A mesh encoding method according to some embodiments comprises: selecting a data type for a basemesh attribute; encoding the basemesh attribute using the selected data type; and signaling, at a V3C level, a syntax element having a value indicating the selected data type.

[0110] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled at the V3C level having a value indicating a data type of the respective basemesh attribute.

[0111] A mesh encoding method according to some embodiments comprises: obtaining basemesh attribute information having a number of one or more sub-components; encoding the plurality of sub-components of the basemesh attribute; and signaling, in a sequence parameter set, a syntax element having a value indicating the number of sub-components of the basemesh attribute.

[0112] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled in the sequence parameter set having a value indicating the number of sub-components of the respective basemesh attribute.

[0113] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0114] A mesh encoding method according to some embodiments comprises: obtaining basemesh attribute information having a number of one or more sub-components; encoding the plurality of sub-components of the basemesh attribute; and signaling, in a V3C parameter set V-DMC extension, a syntax element having a value indicating the number of subcomponents of the basemesh attribute.

[0115] Some embodiments further comprise encoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is signaled in the V3C parameter set V-DMC extension having a value indicating the number of sub-components of the respective basemesh attribute.

[0116] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0117] A mesh encoding method according to some embodiments comprises: obtaining basemesh attribute information for at least one custom attribute; encoding each of the custom attributes in a respective atlas, each atlas having a respective atlas identifier; and signaling a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a label of the custom attribute and the respective atlas identifier.

[0118] In some embodiments, the SEI message further includes, for each custom attribute, information indicating whether a label is provided for each of a plurality of dimensions of the custom attribute.

[0119] In some embodiments, for at least one of the custom attributes, the information indicating whether a label is provided for each of a plurality of dimensions indicates that a label is provided for each of a plurality of dimensions, and wherein the SEI message further includes a dimension label for each of the plurality of dimensions.

[0120] A mesh decoding method according to some embodiments comprises: reading, from a sequence parameter set, a syntax element having a value indicating a selected data type for a basemesh attribute; and decoding the basemesh attribute using the indicated data type.

[0121] Some embodiments further comprise decoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read from in the sequence parameter set having a value indicating a data type of the respective basemesh attribute.

[0122] A mesh decoding method according to some embodiments comprises: reading, at a V3C level, a syntax element having a value indicating a selected data type for a basemesh attribute; and decoding the basemesh attribute using the indicated data type.

[0123] Some embodiments further comprise decoding, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read at the V3C level having a value indicating a data type of the respective basemesh attribute.

[0124] A mesh decoding method according to some embodiments comprises: reading, from a sequence parameter set, a syntax element having a value indicating a number of subcomponents of a basemesh attribute; and decoding the plurality of sub-components of the basemesh attribute.

[0125] A method according to some embodiments further comprises reading, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read from the sequence parameter set having a value indicating the number of sub-components of the respective basemesh attribute.

[0126] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0127] A mesh decoding method according to some embodiments comprises reading, from a V3C parameter set V-DMC extension, a syntax element having a value indicating a number of sub-components of the basemesh attribute; and decoding the plurality of sub-components of the basemesh attribute.

[0128] Some embodiments further comprise reading, for each of a plurality of basemesh attributes, information indicating an attribute type, wherein, for at least one of the basemesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basemesh attributes having an unspecified attribute type, a syntax element is read from the V3C parameter set V-DMC extension having a value indicating the number of sub-components of the respective basemesh attribute.

[0129] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of sub-components of the basemesh attribute for each for a plurality of basemesh attributes.

[0130] A mesh decoding method according to some embodiments comprises: reading a supplemental enhancement information (SEI) message, the SEI message including, for eachcustom attribute, a label of the custom attribute and an atlas identifier associated with the custom attribute; and decoding each of the custom attributes from the respective atlas.

[0131] In some embodiments, the SEI message further includes, for each custom attribute, information indicating whether a label is provided for each of a plurality of dimensions of the custom attribute.

[0132] In some embodiments, for at least one of the custom attributes, the information indicating whether a label is provided for each of a plurality of dimensions indicates that a label is provided for each of a plurality of dimensions, and wherein the SEI message further includes a dimension label for each of the plurality of dimensions.

[0133] An apparatus according to some embodiments comprises one or more processors, the apparatus being configured to perform any of the methods described herein.

[0134] An apparatus according to some embodiments comprises at least one processor and a computer-readable medium storing instructions for performing any of the methods described herein.

[0135] A computer-readable medium according to some embodiments stores instructions for performing any of the methods described herein.

[0136] A computer-readable medium according to some embodiments stores a mesh encoded according to any of the methods described herein.

[0137] A signal according to some embodiments conveys a mesh encoded according to any of the methods described herein.

[0138] A computer program product according to some embodiments includes instructions which, when the program is executed by one or more processors, cause the one or more processors to carry out any of the methods described herein.

[0139] An apparatus according to some embodiments comprises one or more processors configured to perform any of the methods disclosed herein.

[0140] A computer-readable medium (which may be a non-transitory storage medium) according to some embodiments includes instructions for causing one or more processors to perform any of the methods described herein.

[0141] A computer program product according to some embodiments includes instructions which, when the program is executed by one or more processors, cause the one or more processors to carry out any of the methods described herein.

[0142] This disclosure describes a variety of aspects, including tools, features, embodiments, models, approaches, etc. Many of these aspects are described with specificity and, at least toshow the individual characteristics, are often described in a manner that may sound limiting. However, this is for purposes of clarity in description, and does not limit the disclosure or scope of those aspects. Indeed, all of the different aspects can be combined and interchanged to provide further aspects. Moreover, the aspects can be combined and interchanged with aspects described in earlier filings as well.

[0143] The aspects described and contemplated in this disclosure can be implemented in many different forms. While some embodiments are illustrated specifically, other embodiments are contemplated, and the discussion of particular embodiments does not limit the breadth of the implementations. At least one of the aspects generally relates to mesh encoding and decoding, and at least one other aspect generally relates to transmitting a bitstream generated or encoded. These and other aspects can be implemented as a method, an apparatus, a computer readable storage medium having stored thereon instructions for encoding or decoding mesh data according to any of the methods described, and / or a computer readable storage medium having stored thereon a bitstream generated according to any of the methods described.

[0144] In the present disclosure, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, the terms “image,” “picture” and “frame” may be used interchangeably. Usually, but not necessarily, the term “reconstructed” is used at the encoder side while “decoded” is used at the decoder side.

[0145] The terms HDR (high dynamic range) and SDR (standard dynamic range) often convey specific values of dynamic range to those of ordinary skill in the art. However, additional embodiments are also intended in which a reference to HDR is understood to mean “higher dynamic range” and a reference to SDR is understood to mean “lower dynamic range.” Such additional embodiments are not constrained by any specific values of dynamic range that might often be associated with the terms “high dynamic range” and “standard dynamic range.”

[0146] Various methods are described herein, and each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as “first”, “second”, etc. may be used in various embodiments to modify an element, component, step, operation, etc., such as, for example, a “first decoding” and a “second decoding”. Use of such terms does not imply an ordering to the modified operations unless specifically required. So, in this example, the first decoding need not be performed before the second decoding, and may occur, for example, before, during, or in an overlapping time period with the second decoding.

[0147] Various numeric values may be used in the present disclosure, for example. The specific values are for example purposes and the aspects described are not limited to these specific values.

[0148] Embodiments described herein may be carried out by computer software implemented by a processor or other hardware, or by a combination of hardware and software. As a nonlimiting example, the embodiments can be implemented by one or more integrated circuits. The processor can be of any type appropriate to the technical environment and can encompass one or more of microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting examples.

[0149] Various implementations involve decoding. “Decoding”, as used in this disclosure, can encompass all or part of the processes performed, for example, on a received encoded sequence in order to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by a decoder of various implementations described in this disclosure, for example, extracting a picture from a tiled (packed) picture, determining an upsampling filter to use and then upsampling a picture, and flipping a picture back to its intended orientation.

[0150] As further examples, in one embodiment “decoding” refers only to entropy decoding, in another embodiment “decoding” refers only to differential decoding, and in another embodiment “decoding” refers to a combination of entropy decoding and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or generally to the broader decoding process will be clear based on the context of the specific descriptions.

[0151] Various implementations involve encoding. In an analogous way to the above discussion about “decoding”, “encoding” as used in this disclosure can encompass all or part of the processes performed, for example, on an input mesh sequence in order to produce an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, for example, partitioning, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, such processes also, or alternatively, include processes performed by an encoder of various implementations described in this disclosure.

[0152] As further examples, in one embodiment “encoding” refers only to entropy encoding, in another embodiment “encoding” refers only to differential encoding, and in another embodiment “encoding” refers to a combination of differential encoding and entropy encoding.Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or generally to the broader encoding process will be clear based on the context of the specific descriptions.

[0153] When a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of a corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of a corresponding method / process.

[0154] Various embodiments refer to rate distortion optimization. In particular, during the encoding process, the balance or trade-off between the rate and distortion is usually considered, often given the constraints of computational complexity. The rate distortion optimization is usually formulated as minimizing a rate distortion function, which is a weighted sum of the rate and of the distortion. There are different approaches to solve the rate distortion optimization problem. For example, the approaches may be based on an extensive testing of all encoding options, including all considered modes or coding parameters values, with a complete evaluation of their coding cost and related distortion of the reconstructed signal after coding and decoding. Faster approaches may also be used, to save encoding complexity, in particular with computation of an approximated distortion based on the prediction or the prediction residual signal, not the reconstructed one. A mix of these two approaches can also be used, such as by using an approximated distortion for only some of the possible encoding options, and a complete distortion for other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete evaluation of both the coding cost and related distortion.

[0155] The implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented in, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, ora programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.

[0156] Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this disclosure are not necessarily all referring to the same embodiment.

[0157] Additionally, this disclosure may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

[0158] Further, this disclosure may refer to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0159] Additionally, this disclosure may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0160] It is to be appreciated that the use of any of the following 7”, “and / or”, and “at least one of’, for example, in the cases of “A / B”, “A and / or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended for as many items as are listed.

[0161] Also, as used herein, the word “signal” refers to, among other things, indicating something to a corresponding decoder. For example, in certain embodiments the encoder signals a particular one of a plurality of parameters for region-based filter parameter selection for de-artifact filtering. In this way, in an embodiment the same parameter is used at both the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter. Conversely, if the decoder already has the particular parameter as well as others, then signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter. By avoiding transmission of any actual functions, a bit savings is realized in various embodiments. It is to be appreciated that signaling can be accomplished in a variety of ways. For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun.

[0162] Implementations can produce a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal can be formatted to carry the bitstream of a described embodiment. Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

[0163] We describe a number of embodiments. Features of these embodiments can be provided alone or in any combination, across various claim categories and types.

[0164] Although features and elements are described above in particular combinations, each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used toimplement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

CLAIMS1. A mesh encoding method comprising: obtaining basemesh attribute information for at least one custom attribute; encoding each of the custom attributes in a respective atlas, each atlas having a respective atlas identifier; and signaling a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a respective label of the custom attribute and the respective atlas identifier.

2. A mesh encoding apparatus comprising one or more processors configured to perform at least: obtaining basemesh attribute information for at least one custom attribute; encoding each of the custom attributes in a respective atlas, each atlas having a respective atlas identifier; and signaling a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a respective label of the custom attribute and the respective atlas identifier.

3. The method of claim 1 or the apparatus of claim 2, wherein the label of each of the custom attributes is an ASN.1 object identifier value.

4. The method of claim 1 , or claim 3 as it depends from claim 1 , or the apparatus of claim 2, or claim 3 as it depends from claim 2, wherein the SEI message further includes, for each custom attribute, information indicating whether a label is provided for each of a plurality of dimensions of the custom attribute.

5. The method claim 4 as it depends from claim 1 , or the apparatus of claim 4 as it depends from claim 2, wherein, for at least one of the custom attributes, the information indicating whether a label is provided for each of a plurality of dimensions indicates that a label is provided for each of a plurality of dimensions, and wherein the SEI message further includes a respective dimension label for each of the plurality of dimensions.

6. The method of claim 5 as it depends from claim 1 , or the apparatus of claim 5 as it depends from claim 2, wherein each of the dimension labels is an ASN.1 object identifier value.

7. The method of claim 1 , or any of claims 3-6 as they depend from claim 1 , or the apparatus of claim 2, or any of claims 3-6 as they depend from claim 2, wherein the SEI message further includes, for each of the custom attributes, an attribute index identifying an index of the corresponding attribute in an attribute video data unit.

8. A mesh decoding method comprising: reading a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a label of the custom attribute and an atlas identifier associated with the custom attribute; and decoding the custom attributes from the respective atlas.

9. A mesh decoding apparatus comprising one or more processors configured to perform at least: reading a supplemental enhancement information (SEI) message, the SEI message including, for each custom attribute, a label of the custom attribute and an atlas identifier associated with the custom attribute; and decoding the custom attributes from the respective atlas.

10. The method of claim 8 or the apparatus of claim 9, wherein the label of each of the custom attributes is an ASN.1 object identifier value.11 . The method of claim 8, or claim 10 as it depends from claim 8, or the apparatus of claim 9, or claim 10 as it depends from claim 9, wherein the SEI message further includes, for each custom attribute, information indicating whether a label is provided for each of a plurality of dimensions of the custom attribute.

12. The method claim 11 as it depends from claim 8, or the apparatus of claim 11 as it depends from claim 9, wherein, for at least one of the custom attributes, the information indicating whether a label is provided for each of a plurality of dimensions indicates that a label is provided for each of a plurality of dimensions, and wherein the SEI message further includes a respective dimension label for each of the plurality of dimensions.

13. The method of claim 12 as it depends from claim 8, or the apparatus of claim 12 as it depends from claim 9, wherein each of the dimension labels is an ASN.1 object identifier value.

14. The method of claim 8, or any of claims 10-13 as they depend from claim 8, or the apparatus of claim 9, or any of claims 10-13 as they depend from claim 9, wherein the SEI message further includes, for each of the custom attributes, an attribute index identifying an index of the corresponding attribute in an attribute video data unit.

15. The method of claim 8, or any of claims 10-14 as they depend from claim 8, or the apparatus of claim 9, or any of claims 10-14 as they depend from claim 9, wherein the SEI message further includes an integer indicating a number of custom attributes for which information is provided in the SEI message.

Images