Signaling of attribute-related supplemental information in V3C bitstreams and base mesh bitstreams

Abstract

Grid encoding and decoding using custom attributes is presented. In an example grid encoding method, the following steps are performed: obtaining base grid attribute information for at least one custom attribute; encoding each custom attribute in a respective atlas, each atlas having a respective atlas identifier; and signaling a supplemental enhancement information (SEI) message, the SEI message including a respective label for the custom attribute and the respective atlas identifier for each custom attribute.

Description

[0001] Cross-references This application claims the benefit of European Patent Application No. 24305101.8, filed on January 16, 2024, the entire disclosure of which is incorporated herein by reference. Background Technology

[0002] The Visual Volumetric Video-Based Coding (V3C) standard is being developed for coding immersive scenes. A description of V3C is provided in ISO / IEC JTC 1 / SC29, “ISO / IEC 23090-5:2023, Coded representation of immersive media Part 5: Visual volumetric video-based coding (V3C) and video-based pointcloud compression (V-PCC),” issued in November 2023. Within the V3C framework, Video-Based Dynamic Mesh Coding (V-DMC) is under development and is described in ISO / IEC 23090-29, S.2 Secretariat, “WD 5.0 ​​of V-DMC,” issued on October 20, 2023.

[0003] V3C provides encoding and decoding of attributes used in immersive scenes. Some of these attributes are described in the standard material ISO / IEC 23090-5. However, the V3C encoder also has the ability to deliver attributes or sets of attributes that do not correspond to specific attribute types defined in ISO / IEC 23090-5. Conversely, the V3C decoder can receive V3C bitstreams that include attributes or sets of attributes that do not correspond to attribute types defined in ISO / IEC 23090-5.

[0004] Such properties not defined in standard materials can be customized to meet the specific needs of an application. Such properties not specifically defined in ISO / IEC 23090-5 are referred to as custom properties. More specifically, such properties not defined in ISO / IEC 23090-29 are referred to as custom basemesh properties.

[0005] Such custom attributes present potential challenges to the interoperability of V3C bitstreams that need to be signaled / decoded by V3C encoders / decoders, because conventional V3C encoders or decoders do not have the necessary information to correctly process the content of custom attributes. Even when using compression schemes such as video codecs to encode / decode data with custom attributes, current codecs do not provide sufficient signaling information to take into account the correct interpretation of the data content for purposes such as reconstructing the final volumetric assets.

[0006] ISO / IEC 23090-5 includes a list of attribute types. Attribute types are defined by the `ai_attribute_type_id` syntax element. This allows for the addition of custom attributes to the V3C bitstream. Custom attribute types are transmitted via signals in the V3C bitstream using attributes with the attribute type `ATTR_UNSPECIFIED`. However, information about what the `ATTR_UNSPECIFIED` attribute contains is unknown to the receiving third party. There is no mechanism to transmit via signals from the V3C encoder what information is stored in a custom attribute with the attribute type `ATTR_UNSPECIFIED`.

[0007] Furthermore, since V-DMC inherits the basic principles from ISO / IEC 23090-29 and originates from ISO / IEC 23090-5, V-DMC encoders / decoders can deliver / receive attributes or sets of attributes that do not correspond to the attribute types defined in ISO / IEC 23090-29. ISO / IEC 23090-29 introduces components called the base mesh bitstream. The base mesh bitstream can also store attributes. These base mesh attributes provide additional information about the base mesh components. This information can be useful for processing and reconstructing the base mesh. A list of base mesh attributes is defined in the ISO / IEC 23090-29 specification. However, it may be expected that V-DMC applications will encode custom base mesh attributes. This can be transmitted via signals in the V3C V-DMC extension: vps_ext_bmesh_attribute_type: ATTR_UNSPECIFIED or in the base mesh bitstream: bmsps_mesh_attribute_type_id: ATTR_UNSPECIFIED.

[0008] In ISO / IEC 23090-5, for predefined attributes, a bitstream can indicate that the attribute consists of several dimensions (the number of channels). The syntax element `ai_attribute_dimension_minus1` indicates the total number of dimensions of the attribute. Each attribute dimension can store different information. For example, an attribute with attribute type `ATTR_NORMAL` can consist of three dimensions as described in subclause 8.4.4.5 of ISO / IEC 23090-5. Each dimension stores one of the components of a unit vector (x, y, z).

[0009] V3C encoders / decoders can signal / decode custom attributes with specific dimensions. However, for V3C encoders, there is no means to signal the dimensions of custom attributes. Similarly, for V3C decoders, there is no means to interpret decoded data containing custom attributes that lack semantic information. Summary of the Invention

[0010] A grid encoding method according to some embodiments includes: selecting a data type for a basic grid property; encoding the basic grid property using the selected data type; and signaling a syntax element having a value indicating the selected data type in a sequence parameter set.

[0011] Some embodiments also include encoding information indicating the attribute type for each of a plurality of base grid attributes, wherein, for at least one of the base grid attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base grid attributes having an unspecified attribute type, a syntax element having a value indicating the data type of the corresponding base grid attribute is signaled in a sequence parameter set.

[0012] A mesh encoding method according to some embodiments includes: selecting a data type for a basic mesh attribute; encoding the basic mesh attribute using the selected data type; and signaling a syntax element having a value indicating the selected data type at the V3C level.

[0013] Some embodiments further include: for each of a plurality of base mesh attributes, encoding information indicating an attribute type, wherein for at least one of the base mesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein for each of the base mesh attributes having an unspecified attribute type, a syntax element is signaled at the V3C level, having a value indicating the data type of the corresponding base mesh attribute.

[0014] A mesh encoding method according to some embodiments includes: obtaining basic mesh attribute information having a number of one or more sub-components; encoding multiple sub-components of the basic mesh attribute; and signaling a syntax element in a sequence parameter set, the syntax element having a value indicating the number of sub-components of the basic mesh attribute.

[0015] Some embodiments further include: for each of a plurality of basic grid attributes, encoding information indicating an attribute type, wherein for at least one of the basic grid attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein for each of the basic grid attributes having an unspecified attribute type, a syntax element is signaled in a sequence parameter set having a value indicating the number of subcomponents of the corresponding basic grid attribute.

[0016] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of subcomponents of each of a plurality of base grid properties.

[0017] A mesh encoding method according to some embodiments includes: obtaining basic mesh attribute information having a number of one or more sub-components; encoding multiple sub-components of the basic mesh attribute; and signaling a syntax element in a V3C parameter set V-DMC extension, the syntax element having a value indicating the number of sub-components of the basic mesh attribute.

[0018] Some embodiments also include encoding information indicating the attribute type for each of a plurality of base mesh attributes, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a signaling syntax element is transmitted in the V3C parameter set V-DMC extension, having a value indicating the number of subcomponents of the corresponding base mesh attribute.

[0019] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of subcomponents of each of a plurality of base mesh attributes.

[0020] A mesh encoding method according to some embodiments includes: obtaining basic mesh attribute information for at least one custom attribute; encoding each custom attribute in a corresponding atlas, each atlas having a corresponding atlas identifier; and signaling supplementary enhancement information (SEI) messages, wherein for each custom attribute, the SEI message includes a label for the custom attribute and a corresponding atlas identifier.

[0021] In some embodiments, for each custom attribute, the SEI message also includes information indicating whether a label is provided for each of the multiple dimensions of the custom attribute.

[0022] In some embodiments, for at least one of the custom attributes, information indicating whether a label is provided for each of the multiple dimensions indicates that a label is provided for each of the multiple dimensions, and wherein the SEI message also includes dimension labels for each of the multiple dimensions.

[0023] A mesh decoding method according to some embodiments includes: reading a syntax element from a set of sequence parameters, the syntax element having a value of a selected data type indicating an underlying mesh attribute; and decoding the underlying mesh attribute using the indicated data type.

[0024] Some embodiments also include decoding information indicating the attribute type for each of a plurality of base grid attributes, wherein, for at least one of the base grid attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base grid attributes having an unspecified attribute type, a syntax element having a value indicating the data type of the corresponding base grid attribute is read from a set of sequence parameters.

[0025] A mesh decoding method according to some embodiments includes: reading a syntax element at the V3C level, the syntax element having a value of a selected data type indicating an underlying mesh attribute; and decoding the underlying mesh attribute using the indicated data type.

[0026] Some embodiments also include decoding information indicating the attribute type for each of a plurality of base mesh attributes, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a syntax element is read at the V3C level, which has a value indicating the data type of the corresponding base mesh attribute.

[0027] A mesh decoding method according to some embodiments includes: reading a syntax element from a set of sequence parameters, the syntax element having a value indicating the number of sub-components of a basic mesh property; and decoding the multiple sub-components of the basic mesh property.

[0028] The method according to some embodiments further includes, for each of a plurality of basic mesh attributes, reading information indicating an attribute type, wherein, for at least one of the basic mesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basic mesh attributes having an unspecified attribute type, a syntax element is read from a set of sequence parameters having a value indicating the number of subcomponents of the corresponding basic mesh attribute.

[0029] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of subcomponents of each of a plurality of base grid properties.

[0030] According to some embodiments, the mesh decoding method includes reading a syntax element from a V3C parameter set V-DMC extension, the syntax element having a value indicating the number of sub-components of a basic mesh property; and decoding the multiple sub-components of the basic mesh property.

[0031] Some embodiments also include, for each of a plurality of base mesh attributes, reading information indicating the attribute type, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a syntax element is read from the V3C parameter set V-DMC extension, which has a value indicating the number of subcomponents of the corresponding base mesh attribute.

[0032] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of subcomponents of each of a plurality of base mesh attributes.

[0033] According to some embodiments, the mesh decoding method includes: reading supplementary enhancement information (SEI) messages, for each custom attribute, the SEI message including a label for the custom attribute and an atlas identifier associated with the custom attribute; and decoding each custom attribute from the corresponding atlas.

[0034] In some embodiments, for each custom attribute, the SEI message also includes information indicating whether a label is provided for each of the multiple dimensions of the custom attribute.

[0035] In some embodiments, for at least one of the custom attributes, information indicating whether a label is provided for each of the multiple dimensions indicates that a label is provided for each of the multiple dimensions, and wherein the SEI message also includes dimension labels for each of the multiple dimensions.

[0036] An apparatus according to some embodiments includes one or more processors configured to perform any of the methods described herein.

[0037] An apparatus according to some embodiments includes at least one processor and a computer-readable medium storing instructions for performing any of the methods described herein.

[0038] According to some embodiments, a computer-readable medium stores instructions for performing any of the methods described herein.

[0039] Computer-readable media, according to some embodiments, store a grid encoded according to any method described herein.

[0040] According to some embodiments, signal transmission is a mesh encoded according to any method described herein.

[0041] A computer program product according to some embodiments includes instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described herein. Attached Figure Description

[0042] The following detailed description will be better understood when read in conjunction with the accompanying drawings, which illustrate examples of one or more of the various embodiments of this disclosure. However, it should be understood that the embodiments described herein are not limited to the precise arrangements and means shown in the drawings.

[0043] Figure 1 This is a functional block diagram of an example grid coding system using intra-frame coding.

[0044] Figure 2 This is a functional block diagram of an example mesh decoding system using intra-frame decoding.

[0045] Figure 3 The diagram schematically illustrates the encapsulation of different sub-bitstreams in a bitstream that encodes a dynamic grid, such as a V-DMC bitstream.

[0046] Figure 4 This is a block diagram of an example system that implements various aspects and embodiments. Detailed Implementation

[0047] Overview of Dynamic Mesh Coding Figure 1This is a schematic block diagram of a mesh encoding process that may be employed in some embodiments. A source mesh model 302 is provided as the input mesh M(i) for the mesh encoding process. The source mesh model 302 is associated with a source texture map 304, which is shown to be the input texture map A(i) for the encoding process. At 306, the input mesh is decimate to generate a base mesh m(i) with a reduced number of vertices, and at 308, a UV atlas is generated from the base mesh. The base mesh is quantized at 310 and encoded at 312, where the compressed base mesh data is multiplexed into a dynamic mesh bitstream at 314. The compressed base mesh data is reconstructed at the encoder to generate a reconstructed base mesh m'(i) using a static mesh decoder 316. At 318, the reconstructed base mesh is subdivided by adding new vertices. At 320, a subdivision surface fitting process is performed, which determines a set d(i) of displacements d(i) that deform the vertices of the subdivided base grid to more closely correspond to the surface defined by the input grid M(i) by comparing the subdivided base grid with the input grid M(i). These displacements can be updated to newer displacements d'(i) at 322 based on the difference between the original base grid m(i) and the reconstructed base grid m'(i). These updated displacements are encoded using a wavelet transform 324 that generates wavelet coefficients e'(i), which is quantized at 326 and packed into an image format at 328. At 330, a time-varying image series representing the wavelet coefficients can be encoded using conventional video coding techniques, and at 314, the encoded video can be multiplexed with data representing the compressed base grid. At the encoder, displacements are reconstructed from the encoded video via image unpacking 329, inverse quantization 331, and inverse wavelet transform 332 to generate the reconstructed displacement set d''(i). The reconstructed base mesh M''(i) is obtained by inverse quantization of the reconstructed quantized base mesh m'(i) at 334, and the reconstructed base mesh M''(i) is subdivided at 336. At 338, the reconstructed deformable mesh DM(i) is generated by applying the reconstructed displacement set d''(i) to the reconstructed base mesh m'(i). The reconstructed deformable mesh DM(i) is used as the destination mesh model 341 for the purpose of attribute transfer 340.

[0048] Using the reconstructed deformable mesh DM(i) (destination mesh model 341), the input mesh M(i) (source mesh model 302), and the input texture map A(i) (source texture map 304), an attribute transfer process is performed to provide attribute values ​​for the destination texture map A'(i) associated with the reconstructed deformable mesh DM(i). A padding process 342 can be used to fill pixels in texture map A'(i) that are not associated with any triangles in the reconstructed deformable mesh DM(i). A color space conversion 344 can be performed, the time-varying series texture map A'(i) can be encoded using conventional video coding techniques 346, and the encoded video can be multiplexed at 314 into a bitstream 350 with data representing displacement and a compressed underlying mesh. Patch information 348 can also be multiplexed in the bitstream.

[0049] Figure 2 A grid decoding method that can be implemented in some embodiments is shown. Compressed bitstream. b(i) The data is demultiplexed into data representing patch information, data representing the static mesh, video data representing mesh displacement, and video data representing attributes. The data representing the static mesh is decoded by the static mesh decoder into a reconstructed quantized base mesh. m '( i Then, inverse quantization is performed to obtain the underlying grid for decoding. m ''( i The video data representing the grid displacement is decoded. The decoded image is unpacked and inversely quantized. The inverse wavelet transform is applied to obtain the decoded displacement d''(i). The decoded underlying grid is then used. m ''( i The deformed mesh is reconstructed using the decoded displacement d''(i) and the decoded displacement d''(i), thus obtaining the decoded mesh. M ''( i The video data representing the attributes is decoded, and color format / space conversion is applied to obtain the decoded attribute map. A ''( i ).

[0050] Figure 1 and Figure 2 An example of intra-frame grid coding and decoding is shown. It should be noted that the example embodiments described herein can also be implemented in the case of inter-frame grid coding and decoding.

[0051] For V-DMC, the framework is developed by extending V3C. V3C is described in ISO / IEC 23090-5:2021, Informationtechnology — Coded representation of immersive media — Part 5: Visualvolumetric video-based coding (V3C) and video-based point cloud compression (V-PCC).

[0052] In MPEG, the V-DMC standard has already begun working on static mesh codecs. Static mesh codecs are specified in Annex I of ISO / IEC 23090-29. The static mesh codec in Annex I is the default codec for compressing static meshes. This includes mesh geometry, mesh connectivity, and mesh properties. Mesh properties provide additional information about mesh vertices or mesh faces.

[0053] The V-DMC framework can involve different properties specified for dynamic mesh sequences. In V-DMC, the underlying static mesh codec can decode the mesh properties for each face or each vertex. These properties can provide additional information about the static mesh, such as color, texture coordinates, normals, reflectivity information, transparency information, and / or user-defined properties.

[0054] Attributes from the static mesh codec can be further used by the base mesh codec. The base mesh bitstream is further encapsulated within a V3C bitstream. The layered structure of the V-DMC bitstream is... Figure 3 As shown in the diagram, the extension mechanism in the V3C bitstream can provide information about the types of attributes that the V3C bitstream / frame will use from the underlying base mesh codec. Additionally, the base mesh bitstream can provide mechanisms for obtaining information about the types of mesh attributes of interest from the underlying static mesh codec.

[0055] Problems solved in some embodiments One problem addressed in some embodiments is that the current ISO / IEC 23090-29 specification does not provide the ability to signal additional information about custom underlying mesh properties, such as the number of components of the underlying mesh property and the data type used for the components.

[0056] A V3C encoder / packager may be needed to deliver one or more custom attributes within the V3C bitstream. One or more custom attributes may need to be signaled within the bitstream. Within the ISO / IEC 23090-5 specification, unspecified attribute types, such as ATTR_UNSPECIFIED, can be used to signal custom attributes. However, the ISO / IEC 23090-5 specification does not provide a mechanism for providing information about the nature of attributes encoded using the ATTR_UNSPECIFIED attribute type. An example embodiment provides a mechanism for identifying the content stored in an attribute having the attribute type ATTR_UNSPECIFIED for the standardized mechanism.

[0057] Overview of Example Implementations In an example embodiment, the V3C encoder can transmit (and the decoder can decode) additional information about the underlying mesh properties using signals. This information may include things like the number of components of the underlying mesh properties and / or the data type of the components.

[0058] In some embodiments, the encoder transmits (and the decoder decodes) supplemental enhancement information (SEI) messages to embed / retrieve information about one or more custom attributes used by the implementation.

[0059] The example implementation can be used to provide additional information about the underlying mesh properties. The current specification ISO / IEC 23090-29 does not provide a mechanism for identifying additional information about the underlying mesh properties; instead, the underlying mesh property components are transmitted / decoded as signals of a specific data type. In the example implementation, the underlying mesh property component type may exist in the V3C bitstream to provide information about the data type of the underlying mesh property component, as shown below.

[0060] Furthermore, depending on the type of the underlying grid attribute, the number of components of the underlying grid attribute can have different values. In the example embodiment, the dimension of the underlying grid attribute can exist in the V3C bitstream to provide information about the number of components of the underlying grid attribute, as shown below.

[0061] Some embodiments provide a base mesh attribute component type. Sub-components of the base mesh attribute can be encoded using a specified data type. The base mesh encoder / decoder uses appropriate information about the data type of the base mesh attribute to signal / decode the base mesh attribute components. The data type of the base mesh attribute can remain consistent across multiple sub-components / dimensions of the base mesh attribute. For example, the data type of the base mesh attribute can be FLOAT or UNSIGNED INTEGER for all sub-components / dimensions of the base mesh attribute.

[0062] The example base mesh encoder / decoder uses a signaled / decoding syntax element that provides information about the data type of one or more components of the base mesh attribute. In some cases, a signaled syntax element (bmsps_mesh_attribute_dimension_data_type) can be used within the base mesh sequence parameter set to provide values ​​identifying the data type of the components of the base mesh attribute within the base mesh sequence. In some cases, a signaled syntax element (e.g., v3c_ext_bmesh_attribute_dimension_data_type) can be used at the V3C level to provide values ​​identifying the data type of the components of the base mesh attribute for the entire V3C bitstream.

[0063] In some embodiments, the values ​​of syntax elements (e.g., v3c_ext_bmesh_attribute_dimension_data_type / bmsps_mesh_attribute_dimension_data_type) are mapped to a standardization mechanism specified outside of ISO / IEC 23090-29. In some cases, the values ​​of syntax elements may be mapped to data type identifiers. In some cases, the values ​​of syntax elements follow the mappings shown in the table below. bmsps_mesh_attribute_dimension_data_type or v3c_ext_bmesh_attribute_dimension_data_type describe 0 unsigned integers 1 Signed integers 2 Unsigned short integer 3 Signed short integer 4 floating point 5……14 Reserved for future use 15 not specified

[0064] Some embodiments provide signaling for the dimensions of the basic mesh attribute. A basic mesh attribute can have several dimensions. The dimension of a basic mesh attribute specifies the number of sub-components that make up the basic mesh attribute. For example, NORMAL is represented by three components (x, y, z). Therefore, the value of the dimension of the basic mesh attribute of type NORMAL will be three.

[0065] In an example embodiment, a V3C encoder / decoder can use signal transmission / decoding to provide a syntax element that provides the dimension value of the underlying mesh attribute. Such syntax elements can exist in the V3C bitstream at different levels. The value provided by the syntax element is equal to the number of components in the static mesh bitstream. In ISO / IEC 23090-29, the static mesh codec specifies the dimension of the mesh attribute as mesh_attribute_num_component_minus1.

[0066] In some embodiments, the dimensions of the underlying grid attribute can remain consistent throughout the sequence of the underlying grid bitstream. This takes into account a flexible mechanism for signaling different values ​​of the dimensions of the underlying grid attribute across the sequence of the underlying grid bitstream. For example, in an example embodiment, the syntax element bmsps_attribute_dimension_minus1 exists in the sequence parameter set of the underlying grid bitstream, as shown in the syntax table below. The V3C encoder / decoder signals / decodes the value of the syntax element bmsps_attribute_dimension_minus1 to determine the number of dimensions of the underlying grid attribute. bmesh_sequence_parameter_set_rbsp(){ descriptor ... u（7） for( i = 0; i < bmsps_mesh_attribute_count; i++ ) { bmsps_mesh_attribute_dimension_minus1 u（6） ... } ... }

[0067] In some embodiments, the dimensions of the underlying mesh attributes may remain unchanged for the entire underlying mesh bitstream. Signaling the dimensions of the underlying mesh attributes in each sequence is redundant and may increase the bit rate. Therefore, in some embodiments, the V3C encoder / decoder may signal / decode syntax elements at the V3C level. For example, the syntax element vps_ext_bmesh_attribute_dimension_minus1 may exist in the V3C parameter set V-DMC extension of the V3C bitstream, as shown in the syntax table below. The V3C encoder / decoder signals / decodes the value of the syntax element vps_ext_bmesh_attribute_dimension_minus1 to determine the number of dimensions of the underlying mesh attributes. vps_vdmc_extension() { descriptor for( j=0; j<vps_atlas_count_minus1+1; j++ ){ . . . for( i = 0; i < vps_ext_bmesh_data_attribute_count[ j ]; i++ ) { vps_ext_bmesh_attribute_dimension_minus1 u(6) . . . } . . . } }

[0068] Some embodiments provide the dimensions of a base grid attribute with the attribute type "ATTR_UNSPECIFIED". In some embodiments, the specification may provide default values ​​for the dimensions of the base grid attribute for the defined base grid attribute. A V3C decoder decoding the base grid attribute can use the default values ​​for the dimensions of the base grid attribute to determine the number of subcomponents of the base grid attribute. However, the specification takes into account the possibility that the V3C encoder / decoder may signal / decode generic or unspecified attributes with the base grid attribute type ATTR_UNSPECIFIED. To provide additional information about the dimensions of generic attributes, in some embodiments, the base grid encoder / decoder may signal / decode a syntax element that determines the dimensions of the base grid attribute with the base grid attribute type ATTR_UNSPECIFIED. This syntax element, such as bmsps_attribute_dimension_minus1, may only be present in the base grid bitstream at the sequence parameter set when the base grid attribute type is ATTR_UNSPECIFIED. This mechanism allows the V3C encoder / decoder to flexibly signal / decode changes in the dimensions of base grid attributes with the base grid attribute type ATTR_UNSPECIFIED. bmesh_sequence_parameter_set_rbsp(){ descriptor ... u（7） for( i = 0; i < bmsps_mesh_attribute_count; i++ ) { If (bmsps_mesh_attribute_type == ATTR_UNSPECIFIED) { bmsps_mesh_attribute_dimension_minus1 u（6） } . . . } . . . }

[0069] In some embodiments, for base mesh attributes with base mesh attribute type ATTR_UNSPECIFIED, syntax elements such as vps_ext_bmesh_attribute_dimension_minus1 exist at the V3C level. This mechanism allows the V3C encoder / decoder to signal / decode the dimension values ​​of base mesh attributes with base mesh attribute type ATTR_UNSPECIFIED for the entire base mesh bitstream. vps_vdmc_extension(){ descriptor for( j=0; j<vps_atlas_count_minus1+1; j++ ){ . . . for( i = 0; i < vps_ext_bmesh_data_attribute_count[ j ]; i++ ) { If(vps_ext_bmesh_attribute_type_id == ATTR_UNSPECIFIED ) { vps_ext_bmesh_attribute_dimension_minus1 u(6) } . . . } . . . } }

[0070] Some embodiments provide SEI messages with custom attribute information. V3C encoders / decoders can signal / decode custom attributes. Custom attributes are indicated by the attribute type ATTR_UNSPECIFIED. V3C encoders / decoders signal custom attributes with the attribute type ATTR_UNSPECIFIED.

[0071] V3C encoders / decoders use Supplementary Basic Information (SEI) messages within the signal transmission / decoding bitstream. SEI messages provide information about custom attributes, such as the content of the attribute / custom attribute SEI messages, which may be referred to as "Custom Attribute Information" SEI messages.

[0072] Table 1 shows the syntax of the custom attribute information SEI message in the example embodiment.

[0073] Table 1. Syntax table for custom attribute information. custom_attribute_information( payloadSize ) { descriptor cai_attr_count_minus1 u(7) cai_reserved u(1) for (i=0; i<cai_count_minus1; i++ ) { cai_attr_label[i] oid(v) cai_attr_atlas_id[i] u(6) atlasID = cai_atlas_id[i] cai_attr_index[i] u(7) attrIdx = cai_attribute_index[i] cai_attr_reserved u(3) cai_attr_dimension_label_present u(1) if (cai_attr_dimension_label_present) { NumberOfDimensions = ai_attribute_dimension_minus1[atlasID][attrIdx] for( j = 0 ; j<NumberOfDimensions + 1; j++) { cai_attr_dimension_label[ i ][ j ] oid(v) } } } }

[0074] According to some embodiments, the V3C encoder / decoder signals a list of custom attributes in a Custom Attribute Information (CAI) SEI message, such as the information in Table 1. In an example embodiment, the total number of custom attributes in the attribute video data units of the V3C bitstream is signaled via a syntax element in the CAI SEI message, such as cai_attr_count_minus1. For example, the total number of custom attributes indicated by cai_attr_count_minus1 corresponds to the total number of attributes across different atlases in a V3C bitstream with attribute type: ATTR_UNSPECIFIED.

[0075] In the example embodiment, each attribute is associated with an atlas within the V3C bitstream. For each custom attribute, the V3C encoder / decoder signals / identifies the atlas associated with the attribute. `cai_attr_atlas_id` indicates the identifier of the atlas associated with the attribute.

[0076] The index of the V3C attribute is used as a reference to map the V3C attribute to information related to the custom attribute. cai_attr_index[i] indicates the index of the attribute data with attribute type ATTR_UNSPECIFIED carried in the attribute video data unit.

[0077] In the example embodiment, each custom attribute is described using a label. The label can be provided via an ASN.1 object identifier value from the International Object Identifier Tree. In such embodiments, the International Object Identifier can be assigned by a registration authority according to Rec.ITU T X.660|ISO / IEC 9834 1 or, in the absence of registration, generated using a Universally Unique Identifier (UUID) specified by Rec.ITU T X.667|ISO / IEC 9834 8. The ASN.1 object identifier value of the label for the custom attribute is signaled using a syntax element such as cai_attr_label as shown in Table 1. The label can also be provided using a URI instead of an ASN.1 object identifier value. A null-terminated string encoded as UTF-8 characters, as specified in ISO / IEC 10646, is used to store the value of the syntax element cai_attr_label. The label can describe the content of the custom attribute stored in the V3C bitstream. The semantics of the label can be used to inform the V3C decoder how to interpret the information stored in the custom attribute. The semantics of the attribute may be sufficient to describe the content of each dimension of the custom attribute. In the case of video-based attributes, labels for custom attributes can describe the content of each channel (or dimension) in the attributed video bitstream.

[0078] In some embodiments, the V3C encoder can also transmit information about each dimension of a custom attribute via signals. This can be done when the custom attribute label is generic and additional information is needed to describe each dimension of the custom attribute. Dimensions can be described using ai_attr_dimension_minus1 as shown above.

[0079] In the example implementation, the syntax element describes whether information for each dimension is available for the custom attribute via `cai_attr_dimension_label_present`. When the value of `cai_attr_dimension_label_present` is 1, the object identifier value for each dimension of the custom attribute is available in the CAI SEI message. Otherwise, no additional information is available for each dimension, and the information for each dimension is inferred based on the label of the custom attribute in `cai_attr_label`.

[0080] In the example implementation, when `cai_attr_dimension_label_present` has a value of 1, the V3C encoder / decoder signals / decodes an ASN.1 object identifier value for each dimension of the custom attribute, providing semantics to the data stored in different dimensions / channels of the attribute. The syntax element `cai_attr_dimension_label` specifies the ASN.1 object identifier value in the International Object Identifier Tree. International object identifiers can be assigned by a registration authority according to Rec.ITU TX.660|ISO / IEC 9834 1, or can be generated using a Universally Unique Identifier (UUID) specified in Rec.ITU TX.667|ISO / IEC 9834 8 if no registration is made. The V3C encoder / decoder can use this to provide semantics to the data in each dimension of the custom attribute. A URI can also be used instead of an ASN.1 object identifier value to provide the label. A null-terminated string encoded as UTF-8 characters, as specified in ISO / IEC 10646, can be used to store the value of the syntax element `cai_attr_dimension_label`.

[0081] In the example embodiment, the V3C bitstream may also include a base mesh component for V-DMC applications. The base mesh bitstream may also store a list of attributes. Some base mesh attributes are custom attributes for V-DMC applications. In one embodiment, to indicate a base mesh attribute with the base mesh attribute type ATTR_UNSPECIFIED, the custom attribute information SEI message as described above may be used with some additions, as shown in Table 2.

[0082] Table 2. CAI SEI messages with basic mesh attribute information. custom_attribute_information( payloadSize ) { descriptor cai_attr_count_minus1 u(7) cai_bmesh_attr_count_minus1 u(7) cai_reserved u(1) for (i=0; i<cai_count_minus1 + 1; i++ ) { cai_attr_label [ i ] oid(v) cai_attr_atlas_id[ i ] u(6) atlasID = cai_atlas_id[ i ] cai_attr_index[ i ] u(7) attrIdx = cai_attribute_index[i] cai_attr_reserved u(3) cai_attr_dimension_label_present [ i ] u(1) if ( cai_attr_dimension_label_present ) { NumberOfDimensions = ai_attribute_dimension_minus1[atlasID][attrIdx] for( j = 0 ; j<NumberOfDimensions + 1; j++) { cai_attr_dimension_uri[ i ][ j ] st(v) } } } For (j =0; j<cai_bmesh_attr_count_minus1 + 1 ; j++ ) { cai_bmesh_attr_label [ j ] oid(v) cai_bmesh_attr_atlas_id[ j ] u(6) atlasID = cai_bmesh_attr_atlas_id[ j ] cai_bmesh_attr_index[ j ] u(7) cai_bmesh_attr_dimensions_lebel_present[ j ] u(1) If (cai_bmesh_attr_dimension_label_present ) { Number_of_bmesh_attr_dimensions = vps_ext_bmesh_attribute_dimension_minus1 / bmsps_attribute_dimension_minus1 + 1 for( k<0 ; k = Number_of_bmesh_attr_dimensions ; k++) { cai_bmesh_attr_dimension_label [j] [k] oid(v) } } } }

[0083] In another embodiment, a new SEI message is specified for the base grid bitstream, called "Custom Base Grid Attribute Information (CBAI)". The CBAI SEI message can be used to provide information about custom base grid attributes, such as base grid attributes with the base grid attribute type ATTR_UNSPECIFIED.

[0084] Table 3. Syntax of SEI messages for custom basic mesh attribute information. custom_bmesh_attribute_information( payloadSize ) { descriptor cbai_attr_basemesh_count_minus1 u(7) For (j =0; j<cbai_bmesh_attr_count_minus1 + 1 ; j++ ) { cbai_bmesh_attr_label [ j ] oid(v) cbai_bmesh_attr_atlas_id[ j ] u(6) atlasID = cai_bmesh_attr_atlas_id[ j ] cbai_bmesh_attr_index[ j ] u(7) cbai_bmesh_attr_dimensions_label_present[ j ] u(1) If (cbai_bmesh_attr_dimension_label_present ) { Number_of_bmesh_attr_dimensions = vps_ext_bmesh_attribute_dimension_minus1 / bmsps_attribute_dimension_minus1 + 1 for( k<0 ; k = Number_of_bmesh_attr_dimensions ; k++) { cbai_bmesh_attr_dimension_label [j] [k] oid(v) } } } }

[0085] In an example embodiment, the V3C encoder / decoder or base mesh encoder / decoder uses a Custom Base Mesh Attribute Information (CBAI) SEI message to signal information about a list of custom attributes. The total number of custom base mesh attributes in the base mesh component of the V3C bitstream is signaled via syntax elements in the CBAI SEI message, such as cbai_bmesh_attr_count_minus1. The total number of custom base mesh attributes indicated by cbai_bmesh_attr_count_minus1 corresponds to the total number of base mesh attributes across different atlases in a base mesh bitstream with the base mesh attribute type: ATTR_UNSPECIFIED.

[0086] In the example embodiment, each base mesh attribute is associated with an atlas within the V3C bitstream. For each custom base mesh attribute, the base mesh encoder / decoder signals / identifies the V3C atlas associated with the attribute. `cbai_bmesh_attr_atlas_id` indicates the identifier of the V3C atlas associated with the base mesh attribute.

[0087] The index of the base mesh attribute can be used as a reference to map the base mesh attribute to information related to the custom base mesh attribute. cbai_bmesh_attr_index[i] indicates the index of the base mesh attribute data with base mesh attribute type ATTR_UNSPECIFIED carried in the base mesh cell of the V3C bitstream.

[0088] Each custom base mesh attribute is described by a label. Labels can be provided, for example, via an ASN.1 object identifier value from the International Object Identifier Tree. International Object Identifiers can be assigned by a registration authority according to Rec.ITU T X.660|ISO / IEC 9834-1 or can be generated using a Universally Unique Identifier (UUID) specified in Rec.ITU T X.667|ISO / IEC 9834-8 if no registration is made. The ASN.1 object identifier value of the label for the custom base mesh attribute is signaled using syntax elements such as `cbai_bmesh_attr_label`, as shown in Tables 2 and 3. Labels can also be provided using a URI instead of an ASN.1 object identifier value. A null-terminated string encoded as UTF-8 characters, as specified in ISO / IEC 10646, is used to store the value of the syntax element `cbai_bmesh_attr_label`. Labels can describe the contents of the custom base mesh attributes stored in the base mesh bitstream. The semantics of the labels can be used to inform the base mesh decoder how to interpret the information stored in the custom base mesh attributes. The semantics of a base grid attribute can be explicit to describe the content of each dimension of a custom base grid attribute. The semantics provided by the labels of custom attributes can describe the content of each base grid attribute component (or dimension) in the base grid bitstream.

[0089] In some embodiments, the base mesh encoder can also signal information about each dimension of a custom base mesh attribute. This can be used when the custom base mesh attribute label is generic and additional information is needed to describe each dimension of the custom base mesh attribute. Dimensions can be described using syntax elements such as vps_ext_bmesh_attribute_dimension_minus1 or bmsps_attribute_dimension_minus1, as illustrated herein.

[0090] In some embodiments, the syntax element describes whether information for each dimension is available for the custom attribute via `cbai_bmesh_attr_dimension_label_present`. When the value of `cbai_bmesh_attr_dimension_label_present` is 1, the object identifier value for each dimension of the custom base mesh attribute is available in the CBAI SEI message. Otherwise, no additional information is available for each dimension, and the information for each dimension is inferred based on the label of the custom base mesh attribute in `cbai_bmesh_attr_label`.

[0091] In some embodiments, when `cbai_bmesh_attr_dimension_label_present` has a value of 1, the base mesh encoder / decoder signals / decodes an ASN.1 object identifier value for each dimension of the custom base mesh attribute. This object identifier value provides semantics to the data stored in different dimensions / channels of the custom base mesh attribute. The syntax element `cbai_bmesh_attr_dimension_label` specifies the ASN.1 object identifier value in the International Object Identifier Tree. The International Object Identifier can be assigned by a registration authority according to Rec.ITU T X.660|ISO / IEC 9834 1 or can be generated using a Universally Unique Identifier (UUID) specified in Rec.ITU T X.667|ISO / IEC 9834-8 if no registration is made. The base mesh encoder / decoder can use this to provide semantics to the data in each dimension of the custom base mesh attribute. A URI can also be used instead of an ASN.1 object identifier value to provide the label. As specified in ISO / IEC 10646, a null-terminated string encoded as UTF-8 characters is used to store the value of the syntax element cbai_bmesh_attr_dimension_label.

[0092] In an example embodiment, the V3C encoder signals the number of components of the base mesh attribute to provide additional information related to the base mesh attribute. The V3C encoder can also signal values ​​to determine the data type of the base mesh attribute. Identifying the data type of the base mesh attribute helps the base mesh decoder determine the exact format of one or more base mesh attribute components.

[0093] In an example embodiment, the V3C encoder signals a Custom Attribute Information (SEI) message to provide additional information related to one or more custom attributes indicated by attribute type: ATTR_UNSPECIFIED. The V3C decoder parses the SEI message to interpret the information carried in one or more custom attributes indicated by attribute type: ATTR_UNSPECIFIED.

[0094] The encoder / decoder can use CAI SEI messages to signal / parse information related to custom attributes with the attribute type: ATTR_UNSPECIFIED.

[0095] Example system hardware.

[0096] Example embodiments of encoders and / or decoders (collectively referred to as coders) configured to implement the embodiments described herein may use, for example, Figure 4This is achieved through systems like [system name missing]. Figure 4 This is a block diagram illustrating examples of systems implementing various aspects and embodiments therein. System 1000 may be embodied as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described in this document.

[0097] System 1000 includes at least one processor 1010 configured to execute instructions loaded thereon for implementing various aspects, such as those described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0098] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide an encoded or decoded grid, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Alternatively, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated into processor 1010 as a combination of hardware and software known to those skilled in the art.

[0099] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described in this document may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input grids, decoded grids or portions of decoded grids, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0100] In some embodiments, the memory within the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or storage device 1040, such as dynamically volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Universal Video Coding, a new standard being developed by JVET, the Joint Video Experts Group).

[0101] Input to the components of system 1000 can be provided through various input devices as shown in box 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) component (COMP) input terminals (or a collection of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Other examples include composite video.

[0102] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or band-limiting a signal to a certain frequency band), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower frequency band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and / or (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0103] Additionally, USB and / or HDMI terminals may include corresponding interface processors for connecting System 1000 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or within processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processor 1010. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in combination with memory and storage elements to process the data streams as needed for presentation on the output device.

[0104] Various components of system 1000 can be provided within an integrated housing, in which various components can be interconnected and transmit data between them using a suitable connection arrangement 1140, such as an internal bus known in the art, including inter-IC (I2C) bus, wiring and printed circuit board.

[0105] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0106] In various embodiments, data is streamed or otherwise provided to system 1000 using a wireless network such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals from these embodiments are received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 1000, delivering data via an HDMI connection of input box 1130. Still other embodiments use an RF connection of input box 1130 to provide streaming data to system 1000. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0107] System 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 1100 can be used in a television, tablet computer, laptop computer, cellular phone (mobile phone), or other device. The display 1100 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop computer). In various embodiments, other peripheral devices 1120 include one or more of a standalone digital video disc (or digital multifunction disc) (DVD, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments utilize one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000.

[0108] In various embodiments, signaling such as AV.Link, Consumer Electronic Control (CEC), or other communication protocols enabling device-to-device control with or without user intervention is used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120. Output devices can be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to system 1000 via communication interface 1050 using communication channel 1060. Display 1100 and speaker 1110 can be integrated into a single unit with other components of system 1000 in electronic devices such as, for example, televisions. In various embodiments, display interface 1070 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0109] Display 1100 and speaker 1110 may alternatively be separated from one or more other components, for example, if the RF section of input 1130 is part of a separate set-top box. In various embodiments where display 1100 and speaker 1110 are external components, output signals may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0110] These embodiments can be executed by computer software implemented by processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. Memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. As a non-limiting example, processor 1010 can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0111] Additional examples.

[0112] A grid encoding method according to some embodiments includes: selecting a data type for a basic grid attribute; encoding the basic grid attribute using the selected data type; and signaling a syntax element having a value indicating the selected data type in a sequence parameter set.

[0113] Some embodiments also include encoding information indicating the attribute type for each of a plurality of base grid attributes, wherein, for at least one of the base grid attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base grid attributes having an unspecified attribute type, a syntax element having a value indicating the data type of the corresponding base grid attribute is signaled in a sequence parameter set.

[0114] A mesh encoding method according to some embodiments includes: selecting a data type for a basic mesh attribute; encoding the basic mesh attribute using the selected data type; and signaling a syntax element having a value indicating the selected data type at the V3C level.

[0115] Some embodiments also include encoding information indicating the attribute type for each of a plurality of base mesh attributes, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a syntax element is signaled at the V3C level, having a value indicating the data type of the corresponding base mesh attribute.

[0116] A mesh encoding method according to some embodiments includes: obtaining basic mesh attribute information having a number of one or more sub-components; encoding multiple sub-components of the basic mesh attribute; and signaling a syntax element in a sequence parameter set, the syntax element having a value indicating the number of sub-components of the basic mesh attribute.

[0117] Some embodiments also include encoding information indicating the attribute type for each of a plurality of basic mesh attributes, wherein, for at least one of the basic mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the basic mesh attributes having an unspecified attribute type, a syntax element is signaled in a sequence parameter set having a value indicating the number of subcomponents of the corresponding basic mesh attribute.

[0118] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of subcomponents of each of a plurality of basic grid attributes.

[0119] A mesh encoding method according to some embodiments includes: obtaining basic mesh attribute information having a number of one or more sub-components; encoding multiple sub-components of the basic mesh attribute; and signaling a syntax element in a V3C parameter set V-DMC extension, the syntax element having a value indicating the number of sub-components of the basic mesh attribute.

[0120] Some embodiments also include encoding information indicating the attribute type for each of a plurality of base mesh attributes, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a signaling syntax element is transmitted in the V3C parameter set V-DMC extension, having a value indicating the number of subcomponents of the corresponding base mesh attribute.

[0121] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of subcomponents of each of a plurality of base mesh attributes.

[0122] A mesh encoding method according to some embodiments includes: obtaining basic mesh attribute information for at least one custom attribute; encoding each custom attribute in a corresponding atlas, each atlas having a corresponding atlas identifier; and signaling supplementary enhancement information (SEI) messages, wherein for each custom attribute, the SEI message includes a label for the custom attribute and a corresponding atlas identifier.

[0123] In some embodiments, for each custom attribute, the SEI message also includes information indicating whether a label is provided for each of the multiple dimensions of the custom attribute.

[0124] In some embodiments, for at least one of the custom attributes, information indicating whether a label is provided for each of the multiple dimensions indicates that a label is provided for each of the multiple dimensions, and wherein the SEI message also includes a dimension label for each of the multiple dimensions.

[0125] A mesh decoding method according to some embodiments includes: reading a syntax element from a set of sequence parameters, the syntax element having a value of a selected data type indicating an underlying mesh attribute; and decoding the underlying mesh attribute using the indicated data type.

[0126] Some embodiments also include decoding information indicating the attribute type for each of a plurality of base grid attributes, wherein, for at least one of the base grid attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base grid attributes having an unspecified attribute type, a syntax element having a value indicating the data type of the corresponding base grid attribute is read from a set of sequence parameters.

[0127] A mesh decoding method according to some embodiments includes: reading a syntax element at the V3C level, the syntax element having a value of a selected data type indicating an underlying mesh attribute; and decoding the underlying mesh attribute using the indicated data type.

[0128] Some embodiments also include decoding information indicating the attribute type for each of a plurality of base mesh attributes, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a syntax element is read at the V3C level, which has a value indicating the data type of the corresponding base mesh attribute.

[0129] A mesh decoding method according to some embodiments includes: reading a syntax element from a set of sequence parameters, the syntax element having a value indicating the number of sub-components of a basic mesh property; and decoding the multiple sub-components of the basic mesh property.

[0130] The method according to some embodiments further includes, for each of a plurality of basic mesh attributes, reading information indicating an attribute type, wherein, for at least one of the basic mesh attributes, the information indicating an attribute type indicates that the attribute type is unspecified; and wherein, for each of the basic mesh attributes having an unspecified attribute type, a syntax element is read from a set of sequence parameters having a value indicating the number of subcomponents of the corresponding basic mesh attribute.

[0131] In some embodiments, the sequence parameter set includes a syntax element having a value indicating the number of subcomponents of each of a plurality of basic mesh attributes.

[0132] According to some embodiments, the mesh decoding method includes reading a syntax element from a V3C parameter set V-DMC extension, the syntax element having a value indicating the number of sub-components of a basic mesh property; and decoding the multiple sub-components of the basic mesh property.

[0133] Some embodiments also include, for each of a plurality of base mesh attributes, reading information indicating the attribute type, wherein, for at least one of the base mesh attributes, the information indicating the attribute type indicates that the attribute type is unspecified; and wherein, for each of the base mesh attributes having an unspecified attribute type, a syntax element is read from the V3C parameter set V-DMC extension, which has a value indicating the number of subcomponents of the corresponding base mesh attribute.

[0134] In some embodiments, the V3C parameter set V-DMC extension includes a syntax element having a value indicating the number of subcomponents of each of a plurality of base mesh attributes.

[0135] According to some embodiments, the mesh decoding method includes: reading supplementary enhancement information (SEI) messages, for each custom attribute, the SEI message including a label for the custom attribute and an atlas identifier associated with the custom attribute; and decoding each custom attribute from the corresponding atlas.

[0136] In some embodiments, for each custom attribute, the SEI message also includes information indicating whether a label is provided for each of the multiple dimensions of the custom attribute.

[0137] In some embodiments, for at least one of the custom attributes, information indicating whether a label is provided for each of the multiple dimensions indicates that a label is provided for each of the multiple dimensions, and wherein the SEI message also includes a dimension label for each of the multiple dimensions.

[0138] An apparatus according to some embodiments includes one or more processors configured to perform any of the methods described herein.

[0139] An apparatus according to some embodiments includes at least one processor and a computer-readable medium storing instructions for performing any of the methods described herein.

[0140] According to some embodiments, a computer-readable medium stores instructions for performing any of the methods described herein.

[0141] Computer-readable media, according to some embodiments, store a grid encoded according to any method described herein.

[0142] According to some embodiments, signal transmission is a mesh encoded according to any method described herein.

[0143] A computer program product according to some embodiments includes instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0144] An apparatus according to some embodiments includes one or more processors configured to perform any of the methods disclosed herein.

[0145] A computer-readable medium (which may be a non-transitory storage medium) according to some embodiments includes instructions for causing one or more processors to perform any of the methods described herein.

[0146] A computer program product according to some embodiments includes instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0147] This disclosure describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in a specific manner, and many of these aspects, which at least show individual characteristics, are generally described in a way that may sound restrictive. However, this is for the purpose of clarity and does not limit the disclosure or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with those described in earlier filings.

[0148] The aspects described and contemplated in this application can be implemented in many different forms. While some embodiments are specifically shown, other embodiments are contemplated, and the discussion of particular embodiments does not limit the breadth of implementation. At least one aspect generally relates to lattice encoding and decoding, and at least one other aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding lattice data according to any of the described methods, and / or computer-readable storage media having bitstreams generated according to any of the described methods stored thereon.

[0149] In this disclosure, the terms "reconstructed" and "decoded" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Generally, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0150] The terms HDR (High Dynamic Range) and SDR (Standard Dynamic Range) generally convey specific values ​​of dynamic range to those skilled in the art. However, additional embodiments are also intended to include references to HDR as meaning "higher dynamic range" and references to SDR as meaning "lower dynamic range." Such additional embodiments are not bound by any specific values ​​of dynamic range that may generally be associated with the terms "high dynamic range" and "standard dynamic range."

[0151] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, terms such as "first," "second," etc., may be used in various examples to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or in a time period overlapping with the second decoding.

[0152] For example, various numerical values ​​may be used in this disclosure. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0153] The embodiments described herein can be executed by computer software implemented by a processor or other hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. As a non-limiting example, the processor can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0154] Various implementations involve decoding. As used herein, “decoding” can encompass all or part of a process performed, for example, on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also include, or alternatively include, processes performed by a decoder of the various implementations described herein, such as extracting an image from a tiled (packed) image, determining an upsampling filter to use and then upsampling the image, and flipping the image back to its intended orientation.

[0155] As another example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another example, "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, based on the specific context of the description, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process.

[0156] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” the term “encoding” as used in this disclosure can encompass all or part of a process performed, for example, on an input lattice sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also include, or alternatively include, processes performed by encoders of the various implementations described in this disclosure.

[0157] As another example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. It will be clear, based on the specific context of the description, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process.

[0158] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding apparatus. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0159] Various embodiments involve rate distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often taking into account computational complexity constraints. Rate distortion optimization is generally formulated as minimizing a rate distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate distortion optimization problem. For example, these approaches can be based on a comprehensive evaluation of the encoding cost and associated distortion of the reconstructed signal after encoding and decoding, through extensive testing of all encoding options, including all considered mode or encoding parameter values. Faster methods can also be used to save encoding complexity, particularly by utilizing computations based on approximate distortion based on the predicted or predicted residual signal rather than the reconstructed signal. A hybrid of these approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any technique from a variety of methods to perform optimization, but optimization is not necessarily a comprehensive evaluation of encoding cost and associated distortion.

[0160] The implementations and aspects described herein can be implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if discussed only in the context of a single implementation form (e.g., discussed only as a method), the implementation of the features in question can also be implemented in other forms (e.g., apparatus or program). An apparatus can be implemented, for example, in appropriate hardware, software, and firmware. A method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. A processor also includes communication devices, such as, for example, a computer, a cellular phone, a portable / personal digital assistant (“PDA”), and other devices that facilitate information communication between end users.

[0161] References to “an embodiment” or “an embodiment” or “an implementation” or “implementation” and their variations mean that a particular feature, structure, characteristic, and such described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases “in an embodiment” or “in an embodiment” or “in an implementation” or “in an implementation” appearing throughout this disclosure and any other variations do not necessarily refer to the same embodiment.

[0162] Additionally, this disclosure may involve "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, and information retrieved from memory.

[0163] Furthermore, this disclosure may involve “accessing” various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0164] Additionally, this disclosure may involve "receiving" various types of information. Like "access," receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) from one or more sources. Furthermore, "receiving" is generally referred to in one or more ways during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0165] It should be understood that, for example, the use of any of the above " / ", "and / or", and "...at least one of A and B" in the cases of "A / B", "A and / or B", and "at least one of A and B" is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of "A, B and / or C" and "at least one of A, B, and C", this wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to as many items as listed above.

[0166] Furthermore, as used herein, the word “signal” refers, among other things, to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a specific parameter among a plurality of parameters for region-based filter parameter selection for artifact removal filtering. In this way, in embodiments, the same parameter is used at both the encoder and decoder sides. Thus, for example, the encoder may transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, and the like are used to signal information to the corresponding decoder. While the foregoing refers to the verb form of the word “signal,” the word “signal” may also be used herein as a noun.

[0167] The implementation can generate various signals that are formatted to carry information, for example, that can be stored or transmitted. This information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bit stream of the described embodiment. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0168] We have described several embodiments. The features of these embodiments may be provided individually or in any combination across various claim classes and types.

[0169] Although the features and elements have been described above in specific combinations, each feature or element may be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital versatile discs (DVDs). The processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A grid coding method, comprising: Obtain basic mesh attribute information for at least one custom attribute; Each custom attribute is encoded in the corresponding atlas, and each atlas has a corresponding atlas identifier; as well as Supplemental Enhancement Information (SEI) messages are transmitted using signals. Each SEI message includes a corresponding label for the custom attribute and a corresponding atlas identifier for each custom attribute.

2. A lattice coding apparatus, comprising one or more processors, said one or more processors being configured to perform at least: Obtain basic mesh attribute information for at least one custom attribute; Each custom attribute is encoded in a corresponding atlas, and each atlas has a corresponding atlas identifier; and Supplemental Enhancement Information (SEI) messages are transmitted using signals. Each SEI message includes a corresponding label for the custom attribute and a corresponding atlas identifier for each custom attribute.

3. The method of claim 1 or the apparatus of claim 2, wherein the label of each custom attribute is an ASN.1 object identifier value.

4. The method of claim 1, or claim 3 where it is dependent on claim 1, or the apparatus of claim 2, or claim 3 where it is dependent on claim 2, wherein the SEI message further includes information indicating whether a label is provided for each of the multiple dimensions of the custom attribute for each custom attribute.

5. The method according to claim 4 when it is dependent on claim 1, or the apparatus according to claim 4 when it is dependent on claim 2, wherein, For at least one of the custom attributes, information indicating whether labels are provided for each of the multiple dimensions indicates that labels are provided for each of the multiple dimensions, and wherein the SEI message also includes the corresponding dimension label for each of the multiple dimensions.

6. The method according to claim 5 when it is dependent on claim 1, or the apparatus according to claim 5 when it is dependent on claim 2, wherein each dimension label is an ASN.1 object identifier value.

7. The method of claim 1, or any one of claims 3-6 when it is dependent on claim 1, or the apparatus of claim 2, or any one of claims 3-6 when it is dependent on claim 2, wherein the SEI message further includes an attribute index for each custom attribute, identifying an index of the corresponding attribute in the attribute video data unit.

8. A grid decoding method, comprising: Read the Supplemental Enhancement Information (SEI) message, which includes the label of the custom attribute and the atlas identifier associated with the custom attribute for each custom attribute; as well as Decode custom attributes from the corresponding atlas.

9. A mesh decoding apparatus comprising one or more processors, said one or more processors being configured to perform at least: Read the Supplemental Enhancement Information (SEI) message, which includes a label for each custom attribute and an atlas identifier associated with that attribute; and Decode custom attributes from the corresponding atlas.

10. The method of claim 8 or the apparatus of claim 9, wherein the label of each custom attribute is an ASN.1 object identifier value.

11. The method of claim 8, or claim 10 where it is dependent on claim 8, or the apparatus of claim 9, or claim 10 where it is dependent on claim 9, wherein the SEI message further includes information indicating whether a label is provided for each of the plurality of dimensions of the custom attribute for each custom attribute.

12. The method of claim 11 when it is dependent on claim 8, or the apparatus of claim 11 when it is dependent on claim 9, wherein for at least one of the custom attributes, information indicating whether a label is provided for each of the plurality of dimensions indicates that a label is provided for each of the plurality of dimensions, and wherein the SEI message further includes a corresponding dimension label for each of the plurality of dimensions.

13. The method of claim 12 when it is dependent on claim 8, or the apparatus of claim 12 when it is dependent on claim 9, wherein each dimension label is an ASN.1 object identifier value.

14. The method of claim 8, or any one of claims 10-13 thereto when it is dependent on claim 8, or the apparatus of claim 9, or any one of claims 10-13 thereto when it is dependent on claim 9, wherein the SEI message further includes an attribute index for each custom attribute, identifying an index of the corresponding attribute in the attribute video data unit.

15. The method of claim 8, or any one of claims 10-14 thereto when it is dependent on claim 8, or the apparatus of claim 9, or any one of claims 10-14 thereto when it is dependent on claim 9, wherein the SEI message further includes an integer indicating the number of custom attributes for which information is provided in the SEI message.

Images